ISC2016 Workshop: Linux Containers to Optimise IT Infrastructure for HPC & BigData

This years 'Linux Container' workshop at the ISC 2016 is called: Docker: Linux Containers to Optimise IT Infrastructure for HPC & BigData.

It was held after the International Supercomputing Conference in Frankfurt on June 23rd at the Marriott hotel.

• Official agenda page: http://www.isc-hpc.com/isc16_ap/

Unlike last year the focus was to provide actionable knowledge about the world of Linux Containers, discuss problems and possible solutions.

Workshop History

• ISC2022: 8th Annual High Performance Container Workshop
• ISC2021: 7th Annual High Performance Container Workshop
• ISC2020: 6th Annual High Performance Container Workshop
• ISC2019: 5th Annual High Performance Container Workshop
• ISC2018 Workshop: High Performance Container Workshop
• ISC2017 Workshop: Linux Containers to Optimise IT Infrastructure for HPC & BigData
• ISC2016 Workshop: Linux Containers to Optimise IT Infrastructure for HPC & BigData
• ISC2015: Docker: User-Friendly Application & Service Containers for HPC Environments

Agenda

The agenda for the Workshop is as follows.

Details

Linux Containers in a Nutshell (Holger)

Container-based virtualization has many advantages when it comes to packaging and distributing applications and their dependencies and thus helps reducing the complexity of operations. This talk gives a brief introduction to Linux containers and especially their driving force "Docker", their suitability for scientific computation workflows and resource-intensive HPC simulations. As the progress of the Docker platform is fast paced and constantly evolving, the talk will also shed some light on the latest and greatest developments, including multi-host networking and the new high performance container daemon "containerd".

What drives docker in Non-HPC and how to catch up? (Christian)

Docker benefits the non-HPC environments with its boost to software development by providing fast feedback loops and an easy setup of development environment. This part will dive into why that is and shed some light on why HPC environments haven't benefited yet.

Missing Pieces: Singularity, RDMA and others

In this part Christian is touching on Singularity, a container platform built around the notion of "Mobility of Compute". With Singularity you can build executable containers based on your host system and define what happens when that container is launched. Processes inside the container can be single binaries, or a complex of binaries, scripts and data.

Second he will talk about Mellanox effort to bring RDMA Namespaces and CGroups into the Linux kernel, which allows a fine grained resource usage and scheduling of InfiniBand devices.

Furthermore Christian will give a small outlook what is next... :)

UberCloud - Tackling the Software Providers with Containers (Wolfgang)

UberCloud has developed software containers to host software for on-premise and in the HPC Cloud. These production ready containers bundle libraries and software tools as well as ISV software and even whole engineering and scientific workflows. By eliminating the need to install software and configure the high performance computing environment, the time for packaging and accessing ISV software in the cloud has been reduced dramatically.

Docker Volumes w/ BeeGFS (Jasper)

RedCoolBeans has developed a plugin for Docker to allow containers to store data on a volume in a BeeGFS cluster. This presentation will discuss Docker volumes in general, and the plugin for BeeGFS in particular.

Conclusion, Q&A (.*)

The workshop is wrapped up by providing some wiggle room to take questions and provide an outlook (wishlist) how Linux Containers evolve in the foreseeable future.

Motivation

For an idea of what the workshop is going to touch on the following video recording of the host's talk in Lugano provides some guidance:

The issues are talked about from 0:22:30 on.

Comments