ISC2021: 7th Annual High Performance Container Workshop
The workshop took place on July 2nd.
The High Performance Container Workshop series assembles thought leaders to provide the 'state of containers' and the latest trends.
Virtual HPCW 2021
As ISC2021 moved to a virtual event, we went virtual as well with a mix of pre-recorded short videos and a live session. Last year we dove deep into each aspect (runtime, build, distribute, schedule, HPC specifics, outlook) in three sittings. We highly recommend going through the youtube recordings. They can be found in last years post: qnib.org/2020/06/17/isc2020-hpcw.
2021
In 2021, we had a 4h panel session which revolves around use-cases and personas. The goal was to provide guidance on how to approach containers today and plan for the future.
Personas/Use-cases
| # | Short | Representative | Affiliation |
|---|---|---|---|
| 1 | Application Poweruser | Carsten Kutzner | Max Planck Institute Göttingen |
| 2 | Education Usecase | Michael Kuhn | Otto von Guericke University Magdeburg |
| 3 | HPC site SysOps | David Brayford | Leibniz Supercomputing Centre (LRZ) |
| 4 | Bootstrap HPC | Umesh Upadhyaya | HPCNepal |
| 5 | Complex Software Stack | Abdulrahman Azab | University of Oslo |
#1 Application Poweruser
Carsten provided a use-case that portraits an HPC application power user. How to provide a sane environment to researchers within a group? How to pick the right image? How to build images? How to be flexible when it comes to the underlying system (workstations, department clusters and supers)?
#2 Education Use-case
This use-case we previewed for the panelist was revolving around a university setup for teaching undergrads/graduate and post-doc students how to approach HPC applications. Development of I/O benchmarks in a setup that does not need a massive amount of setup and is able to leverage laptops/workstations, local clusters of the department (with control over sysops) as well as compute clusters without control of what is installed.
#3 HPC site SysOps
David portraited a system admin from a big HPC site. In this case LRZ. Stuff they control are system configuration. And the goal is to allow the users to run their code containerized. Both for powerusers and for novice users without much background knowledge. How to build containers for users? how to build with them? how to onboard containers from other sites?
#4 Bootstrap HPC
The third use-case portraits a new and upcoming HPC chapter in Nepal to support local HPC usage. Focus was on ease of use w/o the baggage of 'traditional' HPC. Run on laptops, but also move to clusters eventually.
#5 Complex Software Stack
This use-case put the finger on how to use containers with complex software stacks and workflows. Workflows in which a user would have a script with multiple different stacks being used. What is our guidance on that?
Agenda
We still touched on the following aspects (like last year), but with the goal to tie this back to the personas and discuss how to tackle those problems best.
| # | CEST | Title | Speaker | Company |
|---|---|---|---|---|
| 1 | 2:00PM | Workshop Introduction | Christian Kniep | AWS |
| Use-cases | ||||
| #1 Application Poweruser | Carsten Kutzner | MPI Göttingen | ||
| #2 Education Use-case | Michael Kuhn | Otto von Guericke University Magdeburg | ||
| #3 HPC site SysOps | David Brayford | Leibniz Supercomputing Centre (LRZ) | ||
| #4 Bootstrap HPC | Bootstrap HPC | Umesh Upadhyaya | ||
| #5 Complex Software Stack | Complex Software Stack | Abdulrahman Azab | ||
| 2 | 2:30PM | Runtime Segment | ||
| Runtime Overview | Christian Kniep | AWS | ||
| #1 Application Poweruser | Carsten Kutzner | MPI Göttingen | ||
| #2 Education Use-case | Michael Kuhn | Otto von Guericke University Magdeburg | ||
| #3 HPC site SysOps | David Brayford | Leibniz Supercomputing Centre (LRZ) | ||
| #4 Bootstrap HPC | Bootstrap HPC | Umesh Upadhyaya | ||
| #5 Complex Software Stack | Complex Software Stack | Abdulrahman Azab | ||
| 3 | 3:00PM | Build&Distribute Segment | ||
| Build&Distribute Overview | CJ Newburn | NVIDIA | ||
| ISV Codes | Burak Yenier / Holger Gantikow | UberCloud / Atos | ||
| #1 Application Poweruser | Carsten Kutzner | MPI Göttingen | ||
| #2 Education Use-case | Michael Kuhn | Otto von Guericke University Magdeburg | ||
| #3 HPC site SysOps | David Brayford | Leibniz Supercomputing Centre (LRZ) | ||
| #4 Bootstrap HPC | Bootstrap HPC | Umesh Upadhyaya | ||
| #5 Complex Software Stack | Complex Software Stack | Abdulrahman Azab | ||
| 4 | 3:30PM | Scheduler Segment | ||
| Scheduler Overview | Eduardo Arango | RedHat | ||
| #1 Application Poweruser | Carsten Kutzner | MPI Göttingen | ||
| #2 Education Use-case | Michael Kuhn | Otto von Guericke University Magdeburg | ||
| #3 HPC site SysOps | David Brayford | Leibniz Supercomputing Centre (LRZ) | ||
| #4 Bootstrap HPC | Bootstrap HPC | Umesh Upadhyaya | ||
| #5 Complex Software Stack | Complex Software Stack | Abdulrahman Azab | ||
| 4 | 4:00PM | HPC Specific Segment | ||
| HPC Specifc Topics Discussion | Christian Kniep | AWS | ||
| Singularity Registry (HPC) | Vanessa Sochat | LLNL | ||
| #1 Application Poweruser | Carsten Kutzner | MPI Göttingen | ||
| #2 Education Use-case | Michael Kuhn | Otto von Guericke University Magdeburg | ||
| #3 HPC site SysOps | David Brayford | Leibniz Supercomputing Centre (LRZ) | ||
| #4 Bootstrap HPC | Bootstrap HPC | Umesh Upadhyaya | ||
| #5 Complex Software Stack | Complex Software Stack | Abdulrahman Azab | ||
| 6 | 4:30PM | Use-case ReCap Segment | ||
| ReCap Introduction | Christian Kniep | AWS | ||
| Input Lighting Talks | ||||
| #1 Application Poweruser | Carsten Kutzner | MPI Göttingen | ||
| #2 Education Use-case | Michael Kuhn | Otto von Guericke University Magdeburg | ||
| #3 HPC site SysOps | David Brayford | Leibniz Supercomputing Centre (LRZ) | ||
| #4 Bootstrap HPC | Bootstrap HPC | Umesh Upadhyaya | ||
| #5 Complex Software Stack | Complex Software Stack | Abdulrahman Azab | ||
| 7 | 5:00PM | Live Q&A | ||
| 7 | 6:00PM | End |
Workshop History
- ISC2022: 8th Annual High Performance Container Workshop
- ISC2021: 7th Annual High Performance Container Workshop
- ISC2020: 6th Annual High Performance Container Workshop
- ISC2019: 5th Annual High Performance Container Workshop
- ISC2018 Workshop: High Performance Container Workshop
- ISC2017 Workshop: Linux Containers to Optimise IT Infrastructure for HPC & BigData
- ISC2016 Workshop: Linux Containers to Optimise IT Infrastructure for HPC & BigData
- ISC2015: Docker: User-Friendly Application & Service Containers for HPC Environments