2022-11-23: The Singularity software was renamed to Apptainer in November 2021. This change has now trickled all the way down to the CentOS software repositories and is now deployed on C4. We will soon suggest you to use the apptainer command, but the alias singularity will continue to work for the time being.
The C4 cluster supports Linux containers via the Singularity software. A Linux container is an efficient and powerful virtualization method for running isolated Linux systems (“containers”) on any Linux system including CentOS used by our cluster.
Singularity, and Linux containers in general, provides solutions to common problems and hurdles that HPC users often face, e.g.
Because you can create and customize your own containers, and because Singularity also supports Docker containers, you have immediate access to a very large number of Singularity and Docker containers available via repositories such as:
All tasks for using Linux containers, such as downloading, building, and running containers, is done via the singularity client and supported on C4. The most common command calls are:
singularity run <image> - run predefined script within containersingularity exec <image> - execute any command within containersingularity shell <image> - run bash shell within containersingularity build <path> - build a Singularity from local specificationssingularity build <url> - import Singularity or Docker container available onlineFor full details, see singularity --help, man singularity, and the Singularity website.
As an illustration on how to use Linux containers with Singularity, we will use the Docker container rocker/r-base available on Docker Hub. This particular container provides the latest release of the R software in an Ubuntu OS environment. Containers available from Singularity Hub, Biocontainers, and elsewhere, can be downloaded and used analogously.
To use this rocker/r-base container, we first pull it down to a Singularity image file ~/lxc/rocker_r-base.sif as:
[alice@c4-dev1 ~]$ mkdir lxc
[alice@c4-dev1 ~]$ cd lxc/
[alice@c4-dev1 lxc]$ singularity build rocker_r-base.sif docker://rocker/r-base
INFO: Starting build...
Getting image source signatures
Copying blob sha256:4363cc52203477cd66948034ae4a1db71cbfd27fddb648dd9c590161de1f8634
Copying blob sha256:84b4f34362bc84334554760f8f2546095e5fef74322efaa7979fd1121ae927e5
Copying blob sha256:5a06f0a021cad74bf9b4aa18538585dbef393e3e227fbb960f8bc327e6547581
Copying blob sha256:642a4d4f99ab3f2cf657380497eeede11a9d0263428287296bdc8f71de78795f
Copying blob sha256:d36c605538a66ebff6ae01e73f30069b5bd8e8292ffd7d275cd65f4804b2edae
Copying blob sha256:b6393690e1508d90cb0d3cb9a31dc8679ecc46a949d796e2447b91b4af45e159
Copying config sha256:97c4fe6614a8a627eceeeb91e2fabcffd9fceb89b6e36d526462de4fefbcaab5
Writing manifest to image destination
Storing signatures
2021/01/06 10:40:47 info unpack layer: sha256:4363cc52203477cd66948034ae4a1db71cbfd27fddb648dd9c590161de1f8634
2021/01/06 10:40:51 info unpack layer: sha256:84b4f34362bc84334554760f8f2546095e5fef74322efaa7979fd1121ae927e5
2021/01/06 10:40:51 info unpack layer: sha256:5a06f0a021cad74bf9b4aa18538585dbef393e3e227fbb960f8bc327e6547581
2021/01/06 10:40:52 info unpack layer: sha256:642a4d4f99ab3f2cf657380497eeede11a9d0263428287296bdc8f71de78795f
2021/01/06 10:40:52 info unpack layer: sha256:d36c605538a66ebff6ae01e73f30069b5bd8e8292ffd7d275cd65f4804b2edae
2021/01/06 10:40:52 info unpack layer: sha256:b6393690e1508d90cb0d3cb9a31dc8679ecc46a949d796e2447b91b4af45e159
INFO: Creating SIF file...
INFO: Build complete: rocker_r-base.sif
[alice@c4-dev1 lxc]$ ls -l rocker_r-base.sif
-rwxr-xr-x 1 alice boblab 296992768 Jan 6 10:41 rocker_r-base.sif
The above may take a minute or two to complete.
After this, we can run R within this container using:
[alice@c4-dev1 lxc]$ singularity run rocker_r-base.sif
R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> sum(1:10)
[1] 55
> q()
Save workspace image? [y/n/c]: n
[alice@c4-dev1 lxc]$
Exactly what is “run” is defined by the so called “runscript” of the Singularity container, or the “CMD” if imported from a Docker container. An alternative way to launch R within this container is by explicitly executing R, e.g.
[alice@c4-dev1 lxc]$ singularity exec rocker_r-base.sif R --quiet
> sum(1:10)
[1] 55
> q("no")
[alice@c4-dev1 lxc]$
Note that, the Singularity image is marked as an executable, which means you can run it as any other executable, e.g.
[alice@c4-dev1 lxc]$ ./rocker_r-base.sif
R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> sum(1:10)
[1] 55
> q("no")
[alice@c4-dev1 lxc]$
To launch a shell within this container, and to also convince yourselves that the container runs Ubuntu (and not CentOS as on the C4 host system), do:
[alice@c4-dev1 lxc]$ singularity shell rocker_r-base.sif
Singularity rocker_r-base.sif:~/lxc> head -3 /etc/os-release
PRETTY_NAME="Debian GNU/Linux buster/sid"
NAME="Debian GNU/Linux"
ID=debian
Singularity r-base.sif:~/lxc> Rscript --version
R scripting front-end version 4.0.3 (2020-10-10)
Singularity r-base.sif:~/lxc> exit
[alice@c4-dev1 lxc]$ head -3 /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
When running a container, only a few of the folders available “outside” are available “inside” the container. By default, you have access to the current working directory (= $PWD) and your home folder (= $HOME). In contrast, without further specifications, you will not have access to standard folders such as local /scratch and global /c4/scratch. Similarly, lab folders such as /boblab are not available from inside the container.
[alice@c4-dev1 lxc]$ singularity shell rocker_r-base.sif
Singularity> ls /scratch
ls: cannot access '/scratch': No such file or directory
Singularity> ls /c4/scratch
ls: cannot access '/c4/scratch': No such file or directory
Singularity> ls /boblab
ls: cannot access '/boblab': No such file or directory
Singularity> echo $TMPDIR
/scratch/alice
Singularity> ls "$TMPDIR"
ls: cannot access '/scratch/alice': No such file or directory
To make also these folders available within the container, we can use singularity option --bind. In its simplest form, we can just list the folders we want to make available, e.g.
[alice@c4-dev1 lxc]$ singularity shell --bind /scratch,/c4/scratch,/boblab rocker_r-base.sif
Singularity> ls /scratch
alice
Singularity> ls /c4/scratch
alice
Singularity> ls /boblab
data1 data2
See singularity help instance start for more details and other ways to mount and rename folders within the container.
When it comes to the scheduler, there is nothing special about Singularity per se - the Singularity software can be used as any other software on the cluster. As a proof of concept, here is how to calculate the sum of one to ten using R within the above Linux container at the command line:
[alice@c4-dev1 lxc]$ singularity exec rocker_r-base.sif Rscript -e "sum(1:10)"
[1] 55
[alice@c4-dev1 lxc]$
To run this as a batch job, we need to create a job script.
[alice@c4-dev1 lxc]$ cat demo-singularity.sh
#!/usr/bin/bash
#SBATCH --job-name=demo-singularity
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --mem=100M
#SBATCH --time=00:05:00
#SBATCH --output=%x_%j.out
./rocker_r-base.sif Rscript -e "sum(1:10)"
## End-of-job summary, if running as a job
[[ -n "$SLURM_JOB_ID" ]] && sstat --format="JobID,AveCPU,MaxRSS,MaxPages,MaxDiskRead,MaxDiskWrite" -j "$SLURM_JOB_ID"
And now submit with sbatch:
[alice@c4-dev1 lxc]$ sbatch demo-singularity.sh
Submitted batch job 1657
Check results:
[alice@c4-dev1 lxc]$ cat demo-singularity_5987.out
[1] 55
JobID AveCPU MaxRSS MaxPages MaxDiskRead MaxDiskWrite
------------ ---------- ---------- -------- ------------ ------------
5987.batch 00:00.000 3708K 0 6151153 7102
Q. Why not Docker?
A. Docker is one of the most popular and well-known software solutions for using Linux Containers. However, contrary to Singularity, it turns out that it is hard to get Docker to play well with multi-tenant HPC environments.
Q. What’s the filename extension *.sif?
A. First of all, the filename extension is optional, and some prefer to drop them, e.g. rocker_r-base instead of rocker_r-base.sif. SIF, which is short for the Singularity Container Image Format, is a file format that can hold a Linux container environments in a single file. You might also see Singularity images named *.img and *.simg, which are legacy file formats that Singularity used in the past, where *.img indicates a writable (ext3) images whereas *.simg indicates a read-only (squashfs) image.](https://groups.google.com/a/lbl.gov/d/msg/singularity/Cq7kIbN_L68/2mOdkwx2BAAJ)