Cluster Specifications #

Overview #

Compute nodes

43 (of which 9 are communal)

Logical cores

3024 (of which 680 are communal)

RAM

32-1024 GiB/node

Local scratch

0.05-11 TiB/node

Global scratch

240 TiB

User home storage

1 TiB/user (196 TiB in total)

Summary of Compute Environment #

All nodes on the cluster runs Rocky 8 which is updated on a regular basis. The job scheduler is Slurm 21.08 (Simple Linux Utility for Resource Management) which provides partitions for both communal and lab-priority tasks.

Details #

Login Nodes #

The cluster can be accessed via SSH to one of two login nodes:

1. c4-log1.ucsf.edu
2. c4-log2.ucsf.edu

Data Transfer Nodes #

For transferring large data files, it is recommended to use one of the dedicate data transfer nodes:

1. c4-dt1.ucsf.edu
2. c4-dt2.ucsf.edu

which have 10 Gbps connection - providing a file transfer speed of up to (theoretical) 1.25 GB/s = 4.5 TB/h. Please note, as is true for all ethernet, 80% of line speed is doing pretty good. As for the login nodes, the transfer nodes can be accessed via SSH.

Comment: You can also transfer data via the login nodes, but since those only have 1 Gbps connections, you will see much lower transfer rates.

Development Nodes #

The cluster has development nodes for the purpose of validating scripts, prototyping pipelines, compiling software, and more. Development nodes can be accessed from the login nodes.

Comment: Please use the GPU development node only if you need to build or prototype GPU software. The CPU x86-64 level is the x86-64 microarchitecture levels supported by the nodes CPU.

Compute Nodes #

The majority of the compute nodes have Intel processors, while a few have AMD processes. Each compute node has a local /scratch drive (see below for size), which is either a hard disk drive (HDD), a solid state drive (SSD), or a Non-Volatile Memory Express (NVMe) drive. Each node has a tiny /tmp drive (8-8 GiB).

The compute nodes can only be utilized by submitting jobs via the scheduler.

File System #

Scratch Storage #

The C4 cluster provides two types of scratch storage:

• Local /scratch/ - 0.05-11 TiB/node storage unique to each compute node (can only be accessed from the specific compute node).
• Global /c4/scratch/ - 240 TiB storage accessible from everywhere.

There are no per-user quotas in these scratch spaces. Files not added or modified during the last two weeks will be automatically deleted on a nightly basis. Note, files with old timestamps that were “added” to the scratch place during this period will not be deleted, which covers the use case where files with old timestamps are extracted from tar.gz file. (Details: tmpwatch --ctime --dirmtime --all --force is used for the cleanup.)

User and Lab Storage #

• /c4/home: 196 TiB storage space

Each user may use up to 1 TiB disk space in the home directory. It is not possible to expand user’s home directory. Many labs have purchased their own storage servers which were then mounted on the cluster. If you’d like more information please contact Adam Olshen or Harry Putnam.

Network #

The compute nodes are connected to the local network with 1 Gbps and 10 Gbps network cards.