ISC2020: 6th Annual High Performance Container Workshop
The High Performance Container Workshop series assembles thought leaders to provide the 'state of containers' and the latest trends.
Virtual Workshop 2020
As ISC20 moved to a virtual event, we went virtual as well.
Instead of having one long sitting, we broke the workshop into three sessions that shared a similar structure:
- Firstly, a hand full experts and thought leaders provided their insights and expertise in a short and crisp 10min lightning talk. This filled the first half (~45min) and went back to back, without time to do Q&A.
- Second - and this was really the important and insightful part - all speakers were available for a panel and virtual Q&A.
Time Slots
Segment | Date | CEST | ET | PT |
---|---|---|---|---|
Runtime | 6/16 | 5PM | 11AM | 8AM |
Build | 6/16 | 7PM | 13AM | 10AM |
Distribute | 6/17 | 5PM | 11AM | 8AM |
Orchestrate/Schedule | 6/17 | 7PM | 13AM | 10AM |
HPC Specific | 6/18 | 5PM | 11AM | 8AM |
Use-cases / Outlook | 6/18 | 7PM | 13AM | 10AM |
Agenda
Runtime
Recorded: (6/16 - 5PM CEST)
The first segment build from the ground up by introducing container runtimes and why HPC did not adopt standard runtimes. Afterwards the leading project presented the current state of the art, the panelist concluded the segment by discussing the area with the community.
# | Duration | Title | Speaker | Company | Links |
---|---|---|---|---|---|
1 | 5min | Introduction and Scope | Christian Kniep | AWS | YouTube |
2 | 10min | Container Runtime Overview | Andrew Young | Sandia Labs | YouTube |
3 | 10min | The podman runtime | Valentin Rothberg | Red Hat | YouTube |
4 | 10min | State of Singularity | Ian Kaneshiro | CtrlCmd | YouTube |
5 | 10min | State of SARUS | Lucas Benedicic | CSCS | YouTube |
45min | Live Panel / Q&A | YouTube |
Build
Recorded: (6/16 - 7PM CEST)
This segment focused on building of images as an artifact, how recipes looked like and what the end-user might worry about when defining the artifact.
# | Duration | Title | Speaker | Company | Links |
---|---|---|---|---|---|
1 | 5min | Introduction and Scope | Christian Kniep | AWS | YouTube |
2 | 10min | State of BuildKit | Nicholas Dille | Haufe.Group | YouTube |
3 | 10min | Buildah, a tool to build OCI images | Carlos E Arango | Red Hat | YouTube |
4 | 10min | State of Singularity Build | Greg Kurzter | CtrlCmd | YouTube |
5 | 10min | Build Tools (Spack/Easybuild/...) | Kenneth Hoste | Ghent University | YouTube |
6 | 5min | HPC Container Maker Update | CJ Newburn | NVIDIA | YouTube |
7 | 10min | End-user artifact to specify an image? | Johannes Köster | University of Düsseldorf | YouTube |
45min | Live Panel / Q&A | YouTube |
Distribute
Recorded: (6/17 - 5PM CEST)
Once an image is build it needs to be distributed - this segment focused on how that can be done in a scalable and secure manner.
# | Duration | Title | Speaker | Company | Links |
---|---|---|---|---|---|
1 | 5min | Introduction and Scope | Christian Kniep | AWS | YouTube |
2 | 10min | OCI Image Spec | Carlos E Arango | Red Hat | YouTube |
3 | 10min | Singularity Image Format | Ian Kaneshiro | CtrlCmd | YouTube |
4 | 10min | Skopeo Distribution Tool | Valentin Rothberg | RedHat | YouTube |
5 | 10min | Trust when distributing images | Brandon Lum | IBM | YouTube |
6 | 10min | Container Registries | Phil Estes | IBM | YouTube |
7 | 5min | Findability of Bioinformatic Container Images | Johannes Köster | University of Düsseldorf | YouTube |
8 | 5min | Biocontainers and Conda | Björn Grüning | University of Freiburg | YouTube |
45min | Live Panel / Q&A | YouTube |
Orchestration/Scheduling
Recorded: (6/17 - 7PM CEST)
Starting with a simple scheduler like SLURM this segment built up to more complex schedulers (K8s) and workflow managers (Nextflow, Argo, Airflow).
# | Duration | Title | Speaker | Company | Links |
---|---|---|---|---|---|
1 | 5min | Introduction and Scope | Christian Kniep | AWS | YouTube |
2 | 10min | Simple Container Orchestration | Holger Gantikow | ATOS | YouTube |
3 | 10min | Orchestration/HPC with Kubernetes | Carlos E. Arango | Red Hat | YouTube |
5 | 10min | Portable workflow orchestration at scale with Nextflow | Paolo Di Tommaso | Seqera Labs | YouTube |
6 | 10min | Workflow Schedulers (e.g. Argo, Airflow) | Bill Sparks | HPE | YouTube |
45min | Live Panel / Q&A | YouTube |
HPC Specific
Recorded: (6/18 - 5PM CEST)
Approaching the meet on the bone we discussed the particularities of HPC. Device integration, POSIX file-systems, MPI(/PMI) and scale in general.
# | Duration | Title | Speaker | Company | Links |
---|---|---|---|---|---|
1 | 5min | Introduction and Scope | Christian Kniep | AWS | YouTube |
2 | 10min | HPC Device Integration (kernel-/userland) | CJ Newburn | NVIDIA | YouTube |
3 | 10min | How do container scale? (hpc systems vs cluster) | Bill Sparks | HPE | YouTube |
4 | 10min | High- and lowlights when moving HPC apps? | Saber Feki, Saber Feki, Nagarajan Kathiresan, Mohsin Shaikh, Samuel Kortas, Amr Abdelazim | KAUST | YouTube |
5 | 10min | MPI / PMI Update | Josh Hursey | IBM | YouTube |
45min | Live Panel / Q&A | YouTube |
Use-cases and Outlook
Recorded: (6/18 - 7PM CEST)
We looked back on 6 years of this workshop, got a glimps into how big centers run containers successfully and how HPC use-cases evolved.
# | Duration | Title | Speaker | Company | Links |
---|---|---|---|---|---|
1 | 5min | Introduction and Scope | Christian Kniep | AWS | YouTube |
2 | 15min | 6y HPCW Containerization ReCap | Burak Yenier | TheUberCloud | YouTube |
3 | 10min | NERSC Overview and Outlook | Shane Canon | NERSC | YouTube |
4 | 10min | NERSC Overview and Outlook | Lucas Benedicic | CSCS | YouTube |
4 | 10min | Orchestration for the Edge | CJ Newburn | NVIDIA | YouTube |
5 | 10min | ECP Supercontainer Project | Andrew Younge | Sandia | YouTube |
45min | Live Panel / Q&A | YouTube | |||
- Trends and Projects to be aware of? | |||||
- What is next? Compute models? | |||||
- Why didn't k8s take over yet? | |||||
- How can the HPC community schmooze the non-HPC? |
Workshop ReCap
Recorded: (6/25 - 5PM CEST)
# | Duration | Title | Links |
---|---|---|---|
45min | ReCap Panel 1w after stream | YouTube |
Workshop History
- ISC2022: 8th Annual High Performance Container Workshop
- ISC2021: 7th Annual High Performance Container Workshop
- ISC2020: 6th Annual High Performance Container Workshop
- ISC2019: 5th Annual High Performance Container Workshop
- ISC2018 Workshop: High Performance Container Workshop
- ISC2017 Workshop: Linux Containers to Optimise IT Infrastructure for HPC & BigData
- ISC2016 Workshop: Linux Containers to Optimise IT Infrastructure for HPC & BigData
- ISC2015: Docker: User-Friendly Application & Service Containers for HPC Environments