logo glicid


1. Introduction

In many domains, hypervisor-based virtualization using virtual machines (VMs) has become commonplace, offering customizable software environments with portability and security. However, High-Performance Computing (HPC) has been cautious about this approach due to performance overhead introduced by the layers between applications and hardware. A more efficient alternative in HPC is operating-system level virtualization through containers. Unlike VMs, where applications run with a complete guest OS and a hypervisor, containers execute directly on the host OS, sharing its kernel and some software. This minimizes execution overheads, potentially reaching near native-OS performance. Containers, being leaner, faster to deploy, and requiring less storage space, present a compelling solution compared to VMs.

Containers bring a series of potential benefits to HPC, such as improved software environment flexibility and portability, as well as enhanced research reproducibility. Containers bring a series of potential benefits to HPC, such as improved software environment flexibility and portability, as well as enhanced research reproducibility. Portability is arguably one of the main advantages containers bring to HPC. They allow the application a high degree of independence from the underlying platform. This limitates the need to install and configure these dependencies, as well as the application itself, on every system where it needs to be executed. Besides advantages in terms of system management, this feature reduces or eliminates the need for rebuilding and reconfiguration when attempting to run the application on a new system.

The portability of containers allows the same software environment to be used to repeat experiments on different systems, by different researchers. Therefore, containers can provide enhanced reproducibility of execution for scientific application. This is an important aspect in attaining reproducible of research results. Especially in view of the large array of scientific and engineering fields that rely on computational data processing and simulation.

2. Apptainer: A Secure Alternative to Docker

Despite being considered a de-facto standard, Docker never really gained traction in HPC. One reason being that it was designed to support microservices, thus placing a heavy focus on environment isolation. This contrasts with the sharing approach that is most common in HPC. In addition, its containers are launched by a daemon that has administrative privileges, which are transferred to any application executed inside a container. This raises serious concerns regarding the possibility of privilege escalation, especially in HPC systems, which are usually multi-tenant. As a result, Docker is not supported on the GLiCID cluster. Instead, Singularity has been utilized as a Docker alternative for several years on Liger and Waves. However, Singularity has evolved into three distinct projects: Singularity, SingularityCE, and Apptainer. Among these, Apptainer has replaced Singularity on the GLiCID.

Apptainer serves as a secure alternative to Docker and is specifically designed for high-performance computing. It maintains compatibility with all Docker images and offers support for GPUs and MPI applications.

  • Docker Hub

  • Singularity Container Hub

  • Quay.io

  • GitHub Container Registry

  • Amazon Elastic Container Registry (ECR)

  • Google Container Registry (GCR)

  • Microsoft Azure Container Registry (ACR)

  • IBM Cloud Container Registry

  • GitLab Container Registry

4. Apptainer on GLiCID

To use Apptainer on GLiCID, ensure your Guix package manager is up-to-date. As GLiCID utilizes Guix for package management on its cluster, run the following command in your terminal after logging into GLiCID(Nautilus for this tutorial):

guix pull

Please be patient as this process may take some time to update Guix. Once completed, load the Apptainer module, which has already been installed on Nautilus:

module load apptainer

This action loads the Apptainer module, allowing you to commence downloading, interacting, and building containers.

5. Apptainer Command Line Interface

Apptainer offers a convenient command-line interface (CLI) for container operations. Execute single-line commands to search, build, or run containers. Verify the version of the Apptainer or Singularity command in use with the following --version option:

apptainer --version

For this demonstration, we are using 1.1.6 version.

You can check the available options and subcommands using --help:

apptainer --help

It will give the description of the apptainer and various options using apptainer. There are lots of different options but in this tutorial, we will focus on shell, run, exec, build, and sign and verify.

6. Downloading and Interacting with Containers

Downloading an image from the container library is straightforward. To download the 'alpine' image from Sylabs Cloud, you can search image you want to download directly from the terminal. For example, to download alpine container image, run:

apptainer search alpine

This command can take around a minute to search for the given image from the Sylabs Cloud and you should see an output something like this:

Found 55 container images for amd64 matching "centos7":






Now, to download any one of them run:

apptainer pull library://godloved/secure/centos7:20200514.0.0

The image will be downloaded and saved locally as a .sif file.

6.1. Using Docker images with Apptainer

Docker Hub is one of the largest libraries for container images, and any image hosted on the hub can be easily downloaded. Fortunately, Apptainer is also compatible with Docker images and can be downloaded using the docker:// URL as reference. First, visit Docker Hub to search for the Docker container image you need and copy the image name from the Docker Hub page. For example to download centos7, run:

apptainer pull docker://centos:centos7

After downloading, interact with the container using run, shell, and exec commands.

6.2. Python Image

If you don’t have Python on your PC, you can download the official Python image from the Docker Hub by running:

apptainer pull docker://python

A file with .sif extension will be downloaded. To use it, you can simply shell into the Python image using:

apptainer shell python_latest.sif

For more information, please visit Apptainer Documentation.