The High Performance Container Workshop series assembles thought leaders to provide the 'state of containers' and the latest trends.

Virtual Workshop 2020

As ISC20 moved to a virtual event, we went virtual as well.

Instead of having one long sitting, we broke the workshop into three sessions that shared a similar structure:

  • Firstly, a hand full experts and thought leaders provided their insights and expertise in a short and crisp 10min lightning talk. This filled the first half (~45min) and went back to back, without time to do Q&A.
  • Second - and this was really the important and insightful part - all speakers were available for a panel and virtual Q&A.

Time Slots

Segment Date CEST ET PT
Runtime 6/16 5PM 11AM 8AM
Build 6/16 7PM 13AM 10AM
Distribute 6/17 5PM 11AM 8AM
Orchestrate/Schedule 6/17 7PM 13AM 10AM
HPC Specific 6/18 5PM 11AM 8AM
Use-cases / Outlook 6/18 7PM 13AM 10AM

Agenda

Runtime

Recorded: (6/16 - 5PM CEST)

The first segment build from the ground up by introducing container runtimes and why HPC did not adopt standard runtimes. Afterwards the leading project presented the current state of the art, the panelist concluded the segment by discussing the area with the community.

# Duration Title Speaker Company Links
1 5min Introduction and Scope Christian Kniep AWS YouTube
2 10min Container Runtime Overview Andrew Young Sandia Labs YouTube
3 10min The podman runtime Valentin Rothberg Red Hat YouTube
4 10min State of Singularity Ian Kaneshiro CtrlCmd YouTube
5 10min State of SARUS Lucas Benedicic CSCS YouTube
45min Live Panel / Q&A YouTube

Build

Recorded: (6/16 - 7PM CEST)

This segment focused on building of images as an artifact, how recipes looked like and what the end-user might worry about when defining the artifact.

# Duration Title Speaker Company Links
1 5min Introduction and Scope Christian Kniep AWS YouTube
2 10min State of BuildKit Nicholas Dille Haufe.Group YouTube
3 10min Buildah, a tool to build OCI images Carlos E Arango Red Hat YouTube
4 10min State of Singularity Build Greg Kurzter CtrlCmd YouTube
5 10min Build Tools (Spack/Easybuild/...) Kenneth Hoste Ghent University YouTube
6 5min HPC Container Maker Update CJ Newburn NVIDIA YouTube
7 10min End-user artifact to specify an image? Johannes Köster University of Düsseldorf YouTube
45min Live Panel / Q&A YouTube

Distribute

Recorded: (6/17 - 5PM CEST)

Once an image is build it needs to be distributed - this segment focused on how that can be done in a scalable and secure manner.

# Duration Title Speaker Company Links
1 5min Introduction and Scope Christian Kniep AWS YouTube
2 10min OCI Image Spec Carlos E Arango Red Hat YouTube
3 10min Singularity Image Format Ian Kaneshiro CtrlCmd YouTube
4 10min Skopeo Distribution Tool Valentin Rothberg RedHat YouTube
5 10min Trust when distributing images Brandon Lum IBM YouTube
6 10min Container Registries Phil Estes IBM YouTube
7 5min Findability of Bioinformatic Container Images Johannes Köster University of Düsseldorf YouTube
8 5min Biocontainers and Conda Björn Grüning University of Freiburg YouTube
45min Live Panel / Q&A YouTube

Orchestration/Scheduling

Recorded: (6/17 - 7PM CEST)

Starting with a simple scheduler like SLURM this segment built up to more complex schedulers (K8s) and workflow managers (Nextflow, Argo, Airflow).

# Duration Title Speaker Company Links
1 5min Introduction and Scope Christian Kniep AWS YouTube
2 10min Simple Container Orchestration Holger Gantikow ATOS YouTube
3 10min Orchestration/HPC with Kubernetes Carlos E. Arango Red Hat YouTube
5 10min Portable workflow orchestration at scale with Nextflow Paolo Di Tommaso Seqera Labs YouTube
6 10min Workflow Schedulers (e.g. Argo, Airflow) Bill Sparks HPE YouTube
45min Live Panel / Q&A YouTube

HPC Specific

Recorded: (6/18 - 5PM CEST)

Approaching the meet on the bone we discussed the particularities of HPC. Device integration, POSIX file-systems, MPI(/PMI) and scale in general.

# Duration Title Speaker Company Links
1 5min Introduction and Scope Christian Kniep AWS YouTube
2 10min HPC Device Integration (kernel-/userland) CJ Newburn NVIDIA YouTube
3 10min How do container scale? (hpc systems vs cluster) Bill Sparks HPE YouTube
4 10min High- and lowlights when moving HPC apps? Saber Feki, Saber Feki, Nagarajan Kathiresan, Mohsin Shaikh, Samuel Kortas, Amr Abdelazim KAUST YouTube
5 10min MPI / PMI Update Josh Hursey IBM YouTube
45min Live Panel / Q&A YouTube

Use-cases and Outlook

Recorded: (6/18 - 7PM CEST)

We looked back on 6 years of this workshop, got a glimps into how big centers run containers successfully and how HPC use-cases evolved.

# Duration Title Speaker Company Links
1 5min Introduction and Scope Christian Kniep AWS YouTube
2 15min 6y HPCW Containerization ReCap Burak Yenier TheUberCloud YouTube
3 10min NERSC Overview and Outlook Shane Canon NERSC YouTube
4 10min NERSC Overview and Outlook Lucas Benedicic CSCS YouTube
4 10min Orchestration for the Edge CJ Newburn NVIDIA YouTube
5 10min ECP Supercontainer Project Andrew Younge Sandia YouTube
45min Live Panel / Q&A YouTube
- Trends and Projects to be aware of?
- What is next? Compute models?
- Why didn't k8s take over yet?
- How can the HPC community schmooze the non-HPC?

Workshop ReCap

Recorded: (6/25 - 5PM CEST)

# Duration Title Links
45min ReCap Panel 1w after stream YouTube