ISC2020: 6th Annual High Performance Container Workshop

The High Performance Container Workshop series assembles thought leaders to provide the ‘state of containers’ and the latest trends.

Virtual Workshop 2020

As ISC20 moved to a virtual event, we went virtual as well.

Instead of having one long sitting, we broke the workshop into three sessions that shared a similar structure:

Firstly, a hand full experts and thought leaders provided their insights and expertise in a short and crisp 10min lightning talk. This filled the first half (~45min) and went back to back, without time to do Q&A.
Second - and this was really the important and insightful part - all speakers were available for a panel and virtual Q&A.

Time Slots

Segment	Date	CEST	ET	PT
Runtime	6/16	5PM	11AM	8AM
Build	6/16	7PM	13AM	10AM
Distribute	6/17	5PM	11AM	8AM
Orchestrate/Schedule	6/17	7PM	13AM	10AM
HPC Specific	6/18	5PM	11AM	8AM
Use-cases / Outlook	6/18	7PM	13AM	10AM

Agenda

Runtime

Recorded: (6/16 - 5PM CEST)

The first segment build from the ground up by introducing container runtimes and why HPC did not adopt standard runtimes. Afterwards the leading project presented the current state of the art, the panelist concluded the segment by discussing the area with the community.

#	Duration	Title	Speaker	Company	Links
1	5min	Introduction and Scope	Christian Kniep	AWS	YouTube
2	10min	Container Runtime Overview	Andrew Young	Sandia Labs	YouTube
3	10min	The podman runtime	Valentin Rothberg	Red Hat	YouTube
4	10min	State of Singularity	Ian Kaneshiro	CtrlCmd	YouTube
5	10min	State of SARUS	Lucas Benedicic	CSCS	YouTube
	45min	Live Panel / Q&A			YouTube

Build

Recorded: (6/16 - 7PM CEST)

This segment focused on building of images as an artifact, how recipes looked like and what the end-user might worry about when defining the artifact.

#	Duration	Title	Speaker	Company	Links
1	5min	Introduction and Scope	Christian Kniep	AWS	YouTube
2	10min	State of BuildKit	Nicholas Dille	Haufe.Group	YouTube
3	10min	Buildah, a tool to build OCI images	Carlos E Arango	Red Hat	YouTube
4	10min	State of Singularity Build	Greg Kurzter	CtrlCmd	YouTube
5	10min	Build Tools (Spack/Easybuild/…)	Kenneth Hoste	Ghent University	YouTube
6	5min	HPC Container Maker Update	CJ Newburn	NVIDIA	YouTube
7	10min	End-user artifact to specify an image?	Johannes Köster	University of Düsseldorf	YouTube
	45min	Live Panel / Q&A			YouTube

Distribute

Recorded: (6/17 - 5PM CEST)

Once an image is build it needs to be distributed - this segment focused on how that can be done in a scalable and secure manner.

#	Duration	Title	Speaker	Company	Links
1	5min	Introduction and Scope	Christian Kniep	AWS	YouTube
2	10min	OCI Image Spec	Carlos E Arango	Red Hat	YouTube
3	10min	Singularity Image Format	Ian Kaneshiro	CtrlCmd	YouTube
4	10min	Skopeo Distribution Tool	Valentin Rothberg	RedHat	YouTube
5	10min	Trust when distributing images	Brandon Lum	IBM	YouTube
6	10min	Container Registries	Phil Estes	IBM	YouTube
7	5min	Findability of Bioinformatic Container Images	Johannes Köster	University of Düsseldorf	YouTube
8	5min	Biocontainers and Conda	Björn Grüning	University of Freiburg	YouTube
	45min	Live Panel / Q&A			YouTube

Orchestration/Scheduling

Recorded: (6/17 - 7PM CEST)

Starting with a simple scheduler like SLURM this segment built up to more complex schedulers (K8s) and workflow managers (Nextflow, Argo, Airflow).

#	Duration	Title	Speaker	Company	Links
1	5min	Introduction and Scope	Christian Kniep	AWS	YouTube
2	10min	Simple Container Orchestration	Holger Gantikow	ATOS	YouTube
3	10min	Orchestration/HPC with Kubernetes	Carlos E. Arango	Red Hat	YouTube
5	10min	Portable workflow orchestration at scale with Nextflow	Paolo Di Tommaso	Seqera Labs	YouTube
6	10min	Workflow Schedulers (e.g. Argo, Airflow)	Bill Sparks	HPE	YouTube
	45min	Live Panel / Q&A			YouTube

HPC Specific

Recorded: (6/18 - 5PM CEST)

Approaching the meet on the bone we discussed the particularities of HPC. Device integration, POSIX file-systems, MPI(/PMI) and scale in general.

#	Duration	Title	Speaker	Company	Links
1	5min	Introduction and Scope	Christian Kniep	AWS	YouTube
2	10min	HPC Device Integration (kernel-/userland)	CJ Newburn	NVIDIA	YouTube
3	10min	How do container scale? (hpc systems vs cluster)	Bill Sparks	HPE	YouTube
4	10min	High- and lowlights when moving HPC apps?	Saber Feki, Saber Feki, Nagarajan Kathiresan, Mohsin Shaikh, Samuel Kortas, Amr Abdelazim	KAUST	YouTube
5	10min	MPI / PMI Update	Josh Hursey	IBM	YouTube
	45min	Live Panel / Q&A			YouTube

Use-cases and Outlook

Recorded: (6/18 - 7PM CEST)

We looked back on 6 years of this workshop, got a glimps into how big centers run containers successfully and how HPC use-cases evolved.

#	Duration	Title	Speaker	Company	Links
1	5min	Introduction and Scope	Christian Kniep	AWS	YouTube
2	15min	6y HPCW Containerization ReCap	Burak Yenier	TheUberCloud	YouTube
3	10min	NERSC Overview and Outlook	Shane Canon	NERSC	YouTube
4	10min	NERSC Overview and Outlook	Lucas Benedicic	CSCS	YouTube
4	10min	Orchestration for the Edge	CJ Newburn	NVIDIA	YouTube
5	10min	ECP Supercontainer Project	Andrew Younge	Sandia	YouTube
	45min	Live Panel / Q&A			YouTube
		- Trends and Projects to be aware of?
		- What is next? Compute models?
		- Why didn’t k8s take over yet?
		- How can the HPC community schmooze the non-HPC?

Workshop ReCap

Recorded: (6/25 - 5PM CEST)

#	Duration	Title	Links
	45min	ReCap Panel 1w after stream	YouTube