In this video from HPCKP'19, Christian Kniep from QNIB Solutions presents: Containerized Convergence of Big Data and Big Compute
"Since the early days of High Performance Computing operations and usage of such systems was highly dependent on the vendor, the discipline, the diversity of use-cases and the community it was embedded in," writes Rich Brueckner
"The use-cases and communities of Big Data, Big Compute are converging, due to commoditization in hard- (x86) and software (Linux), the growing importance of big enterprise IT and the advent of HPC characteristics in AI/ML workloads.
With the introduction of containers as distribution artifact and Kubernetes as orchestration substrate, this convergence might experienced it's final push to take place. This talk will dissect the convergence by refreshing the audiences' memory on what containerization is about, segueing into why AI/ML workloads are leading to fully fledged HPC applications eventually and how this will inform the way forward. In conclusion Christian will discuss the three main challenges `Hardware Access`, `Data Access` and `Distributed Computing` in container technology and how they can be tackled by the power of open source, while focusing on the first..."
Read More ...