Sans titre :: Documentation technique (Beta version)

Quickstart

Warning: The documentation is in progress and not yet stabilised.

Getting Started With GliCID

Welcome to GliCID Quickstart. This section will guide you through the initial steps of logging into the GLiCID cluster, transferring files, loading software, and submitting your first job.

It is imperative that you read what Yoda has to say to you : French version / English version

How to access GLiCID Cluster

In order to request access to GLiCID you have to follow these steps:

Create an account on https://clam.glicid.fr (school account, or CRU account for external users).
The account will be validated by an administrator.
User has to generate and upload the pulblic SSH key to CLAM portal (in profile’s SSH Access tab).
Edit the .ssh/config file and add the defined configuration.
Finally, you can log in using SSH from terminal (on Linux and macOS) and PowerShell (on Windows).

Login using Secure Shell Protocol (SSH)

At GLiCID, we prohibit accessing our systems solely through a password. Instead, we employ key-based authentication using the Secure Shell Protocol (SSH). SSH, an open protocol, enables users to log into a remote server system from a local client system. In this section, we demonstrate how to configure this authentication on your system using both OpenSSH (compatible with GNU/Linux, macOS, Windows(PowerShell) etc.).

OpenSSH

OpenSSH stands as a widely used and freely accessible SSH client designed for UNIX-like operating systems like GNU/Linux and macOS. In macOS, OpenSSH is pre-installed. Moreover, it is available in the package repository of numerous GNU/Linux distributions, such as "openssh-client" on Debian and Ubuntu, or "openssh-clients" on Fedora, CentOS, and similar systems. Additionally, the source code for OpenSSH can be obtained from the OpenSSH website at http://www.openssh.com/portable.html.

OpenSSH Key Generation

As we rely on key-based authentication. It is important to generate secure key pairs. The current safest key type is ed25519. To generate a key, use the following command

ssh-keygen -t ed25519

Note: Should the file ~/.ssh/id_ed25519 already exist, ssh-keygen will prompt for confirmation before overwriting it. It is advisable not to overwrite the file, especially if it serves as credentials for another system. Instead, opt for a distinct file name, such as ~/.ssh/id_glicid, and ensure consistency by using the same file name in all subsequent instructions provided in this document.

Following this, ssh-keygen will prompt you to create a passphrase. While it’s optional to enter a passphrase—simply pressing Enter allows you to proceed without one—it is advisable to provide a robust passphrase. In the future, you’ll need to enter this passphrase to unlock your private key. Using a password manager is recommended for securely storing your key and facilitating the use of complex passphrases.

Ensure the safety and confidentiality of the private key, located at ~/.ssh/id_ed25519, on your local host. Simultaneously, the generated public key, found at ~/.ssh/id_ed25519.pub, must be uploaded to the CLAM user portal at https://clam.glicid.fr.

OpenSSH Configuration

The SSH configuration on the user side serves the purpose of establishing shortcuts to targets/hosts and configuring connections. These shortcuts and options not only impact the behavior of SSH but also influence any program utilizing SSH as an underlying mechanism, including git, scp, and rsync.

For a comprehensive view of all available options, refer to "man ssh_config." Recognizing that this may be overwhelming, here are a few highlighted options and configurations.

To set-up, edit the .ssh/config file and add the following:

Host Bastion(1)
    Hostname bastion.glicid.fr
    User the-login-name(2)
    IdentityFile ~/.ssh/id_ed25519(4)
    ForwardAgent yes

Host Glicid(1)
    Hostname login-001.glicid.fr
    User the-login-name(2)
    ProxyJump Bastion(3)
    IdentityFile ~/.ssh/id_ed25519(4)

Host Nautilus(1)
    Hostname nautilus-devel-001.nautilus.intra.glicid.fr
    User the-login-name(2)
    ProxyJump Bastion(3)
    IdentityFile ~/.ssh/id_ed25519(4)

1	Please note that `Nautilus` (with a capital "N") is an alias. You must now use `ssh Nautilus` This is just an example, the alias can take any name.
2	To be replaced by the correct login assigned to you by clam, obviously. For example : doe-j@univ-nantes.fr : for Nantes University or doe-j@ec-nantes.fr for Nantes Central school or doe-j@univ-angers.fr for Angers University etc.
3	Requires OpenSSH client >= 7.3. For previous versions, `ProxyCommand ssh le-nom-de-login@bastion.glicid.fr -W %h:%p` can be used instead.
4	Be careful of the confusion, it is the private key that must be referenced here.

The above entries establish shortcuts, enabling convenient logins to individual nodes such as glicid, nautilus, and philias (soon).

Subsequently, you can simply connect to the GLiCID Cluster login nodes from the terminal using the following commands:

ssh Glicid

ssh Nautilus

After a successful SSH connection, a new prompt or window will appear, reflecting the environment of the connected system. For instance, if you logged into Nautilus, you should observe the corresponding command-line interface associated with Nautilus.

<username>:~$ ssh Nautilus
#################################################################
#   This service is restricted to authorized users only. All    #
#            activities on this system are logged.              #
#  Unauthorized access will be fully investigated and reported  #
#        to the appropriate law enforcement agencies.           #
#################################################################
Last login: Tue Nov 21 11:06:05 2023 from 194.167.60.11
_   _             _   _ _                    lxkkdc
| \ | |           | | (_) |                   kWNOdc
|  \| | __ _ _   _| |_ _| |_   _ ___          kW0c
| . ` |/ _` | | | | __| | | | | / __|         kW0c
| |\  | (_| | |_| | |_| | | |_| \__ \         kW0c
\_| \_/\__,_|\__,_|\__|_|_|\__,_|___/        cOWKl
                                         cx0KXWMWXK0xc
                                ccllllloxXWMMMMMMMMMWKo
         coooolc       codxkO0KXXNNNWWWWWMMMMMMMMMMMMWOl
        c0WWWWWO   oxOKNWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNKOxoc
        lKMMMMMKxOXWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWXOdc
        lKMMMMMWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMW0xdx0NMMWKx
        lKMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWO     kWMMMWOc
        lKMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWOc    kWMMMMNd
        lKMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWKkxkKWMMMMMNd
        lKMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMXx
        lKMMMMMNNWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWXkl
        lKMMMMM0ookKNWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNKOdc
        ckKKKKKx    ldkOKXNWMMMMMMMMMMMMMMMMMMMMMMMWNXK0kdl
                         clodxkkO000KKKKKKK000OOkxdolc

-----------------------------------------------------------------------------
   Welcome to GLiCID HPC cluster Nautilus

=== Computing Nodes =========================================== #RAM/n = #C =
cnode[301-340] 40 BullSequana X440 (2 AMD EPYC 9474F@3.6GHz 48c)  384   3840
cnode[701-708]  8 BullSequana X440 (2 AMD EPYC 9474F@3.6GHz 48c)  768    768
visu[1-4]       4 BullSequana X450 (2 AMD EPYC 9474F@3.6GHz 48c)  768    384
                 with Nvidia A40 (48G) 2 GPUs per node
gnode[1-4]      4 BullSequana X410 (2 AMD EPYC 9474F@3.6GHz 48c)  768    384
                 with Nvidia A100 (80G) 4 GPUs per node
-----------------------------------------------------------------------------
Fast interconnect using InfiniBand 100 Gb/s technology
Shared Storage (scratch) : 427 TB (IBM/Spectrum Scale - GPFS)
Remote Visualization Apps through XCS portal @https://xcs.glicid.fr/xcs/
-----------------------------------------------------------------------------
User storage :
- user directory ......... /home/<username>
- project directory ...... /LAB-DATA/GLiCID/projects/<projectname>
- scratch directory  ..... /scratch/users/<username>
- scratch SSD	.......... /scratch-shared
- scratch Liger .......... /scratchliger/<old_liger_username> (temporary, ro)
- softwares directory .... /opt/software
-----------------------------------------------------------------------------
Softwares :
- use modules ......... module avail
- use GUIX ............ guix install <software> (documentation for details)
			module load guix/latest first (Nautilus only)
-----------------------------------------------------------------------------
Useful Links :
- User DOC ........ https://doc.glicid.fr
- Support ......... https://help.glicid.fr or help@glicid.fr
- Chat ............ bottom right corner on CLAM when admins are available
- Admins .......... tech@glicid.fr
- Forum ........... coming soon
- Status page ..... https://ckc.glicid.fr
[<username>@nautilus-devel-001 ~]$

To close the SSH connection, simply type

exit

Note: If you are a Windows user and find the terminal interface less familiar, you have the option to use MobaXterm.

Some particular cases

Coming from a computer running Mac OSX

In some cases, you get a "permission denied" and the connection is not established. For some people, the following command solves the problem

ssh-add --apple-use-keychain ~/.ssh/<name_of_the_key>

Software modules

HPC cluster systems typically have a large number of software packages installed. On GLiCID, we use the modules package to manage the user environment for the installed packages. Software modules simplify the utilization of pre-installed software packages and libraries by configuring the necessary environment variables. This makes it simple to use different packages or switch between versions of the same package without conflicts.

To see all available modules, run:

$ module avail
------------------------------------------------------------------------ /usr/share/Modules/modulefiles/applications -------------------------------------------------------------------------
castem/2021  castem/2023  gaussian/g16-revA01  hyperworks/2022.2  lammps/15Jun2023  openfoam/11-openmpi  orca/5.3  turbomole/7.3  turbomole/7.5  turbomole/7.41

-------------------------------------------------------------------------- /usr/share/Modules/modulefiles/libraries --------------------------------------------------------------------------
aocl-blis/4.0                            fftw/omp/3.3.10_intel_omp     intel/dnnl-cpu-gomp/latest    intel/intel_ipp_ia32/2021.8.0       intel/mkl/latest      netcdf/f-4.6.1_gnu
boost/1.82.0_gnu                         gmsh/4.11.1_gnu               intel/dnnl-cpu-iomp/2023.1.0  intel/intel_ipp_ia32/latest         intel/mkl32/2023.1.0  pnetcdf/1.12.3_gnu
cuda/12.2.0_535.54.03                    hdf5/1.14.1-2_gnu             intel/dnnl-cpu-iomp/latest    intel/intel_ipp_intel64/2021.8.0    intel/mkl32/latest    pnetcdf/1.12.3_itl
fftw/3.3.10_gnu_serial                   hdf5/1.14.1-2_gnu_openmpi     intel/dnnl-cpu-tbb/2023.1.0   intel/intel_ipp_intel64/latest      intel/tbb/2021.9.0    rdma/46.0_gnu
fftw/3.3.10_intel_serial                 hdf5/1.14.1-2_intel           intel/dnnl-cpu-tbb/latest     intel/intel_ippcp_ia32/2021.7.0     intel/tbb/latest      ucx/1.14.1_gnu
fftw/3.3.10_intel_serial_sp              hdf5/1.14.1-2_intel_intelmpi  intel/dnnl/2023.1.0           intel/intel_ippcp_ia32/latest       intel/tbb32/2021.9.0
fftw/mpi/3.3.10_gnu_13.1.0_openmpi       intel/ccl/2021.9.0            intel/dnnl/latest             intel/intel_ippcp_intel64/2021.7.0  intel/tbb32/latest
fftw/mpi/3.3.10_intel_2023.1.0_intelmpi  intel/ccl/latest              intel/dpl/2022.1.0            intel/intel_ippcp_intel64/latest    libtool/2.4.6_gnu
fftw/omp/3.3.10_gnu_omp                  intel/dnnl-cpu-gomp/2023.1.0  intel/dpl/latest              intel/mkl/2023.1.0                  netcdf/c-4.9.2_gnu

-------------------------------------------------------------------------- /usr/share/Modules/modulefiles/compilers --------------------------------------------------------------------------
amd/4.0.0     intel/compiler-rt/2023.1.0    intel/compiler-rt32/latest  intel/compiler32/2023.1.0  intel/icc/latest      julia/1.9.4  R-project/4.3.1_gnu_mkl
cmake/3.26.4  intel/compiler-rt/latest      intel/compiler/2023.1.0     intel/compiler32/latest    intel/icc32/2023.1.0  nvhpc/22.7
gcc/13.1.0    intel/compiler-rt32/2023.1.0  intel/compiler/latest       intel/icc/2023.1.0         intel/icc32/latest    nvhpc/23.9

---------------------------------------------------------------------------- /usr/share/Modules/modulefiles/tools ----------------------------------------------------------------------------
apptainer/1.1.6         intel/clck/2021.7.3  intel/debugger/2023.1.0       intel/dpct/2023.1.0         intel/inspector/2023.1.0  intel/oclfpga/2023.1.0  ompp/0.8.5_gnu
guix/latest             intel/clck/latest    intel/debugger/latest         intel/dpct/latest           intel/inspector/latest    intel/oclfpga/latest    ompp/0.8.5_itl
intel/advisor/2023.1.0  intel/dal/2023.1.0   intel/dev-utilities/2021.9.0  intel/init_opencl/2023.1.0  intel/itac/2021.9.0       intel/vtune/2023.1.0    valgrind/3.21.0
intel/advisor/latest    intel/dal/latest     intel/dev-utilities/latest    intel/init_opencl/latest    intel/itac/latest         intel/vtune/latest

-------------------------------------------------------------------------- /usr/share/Modules/modulefiles/parallel ---------------------------------------------------------------------------
intel/mpi/2021.9.0  intel/mpi/latest  openmpi/ucx/4.1.5_gcc_8.5.0_ucx_1.14.1_rdma_46.0

This list is not automatically updated so …

To get the list of all installed modules, run :

$ module avail <module_name>

To load a module, use:

module load <module_name>

For example, to load Apptainer, run:

module load Apptainer

To check the list of loaded modules, run:

module list

For example

$ module list
No Modulefiles Currently Loaded.
$ module load gcc/13.1.0 openmpi/ucx/4.1.5_gcc_8.5.0_ucx_1.14.1_rdma_46.0
$ module list
Currently Loaded Modulefiles:
 1) gcc/13.1.0   2) rdma/46.0_gnu   3) ucx/1.14.1_gnu   4) openmpi/ucx/4.1.5_gcc_8.5.0_ucx_1.14.1_rdma_46.0

Some modules may be loaded by others because of dependences

To unload a specific module, run:

module unload <module_name>

To switch between two versions of the same module, run:

module switch <old_module> <new_module>

For example

$ module list
No Modulefiles Currently Loaded.
$ module load nvhpc/22.7
$ module list
Currently Loaded Modulefiles:
 1) nvhpc/22.7
$ module switch nvhpc/22.7 nvhpc/23.9
$ module list
Currently Loaded Modulefiles:
 1) nvhpc/23.9

To unload all the loaded modules, run:

module purge

GUIX

GNU Guix is a package management tool and distribution of the GNU system. Guix makes it easy for unprivileged users to install, upgrade, or remove software packages, to roll back to a previous package set, to build packages from source, and generally assists with the creation and maintenance of software environments.

Guix on GLiCID

To use Guix on the GLiCID cluster, run 'guix pull' command at least once after logging into the Glicid. This updates the package definitions and the Guix source code to the latest version available in the distribution currently available on your local machine. It downloads the latest Guix source code and package descriptions, and deploys it.

guix pull

Note: To use Guix on Nautilus, you must load the Guix module using

module load guix

To search a new package

guix search <package_name>

To install a package

guix install <package_name>

To remove a package

guix remove <package_name>

To check the list of installed packages

guix package -l

For more information, please visit [Guix Official Docs](https://guix.gnu.org/manual/en/guix.html)

File Transfer

For transferring files to or from the cluster, we recommend using 'scp' command. To transfer a file from your system to the cluster run:

scp file_name login-name:/path/to/destination

For example, to transfer a file from local to the /scratch on Nautilus, run:

scp file_name nautilus:/scratch/users/username/path_to_folder

Also, to transfer a file from the cluster to local, run:

scp nautilus:/scratch/users/username/path_to_file /local_path_to_folder

Slurm Workload Manager

GLiCID can be accessed through a specialized set of login nodes. The workload manager regulates access to the system’s compute nodes. On GLiCID, the Slurm (Simple Linux Utility for Resource Management) Workload Manager is utilized. Slurm, a contemporary and adaptable batch system, is a freely available open-source resource manager commonly deployed globally on clusters of diverse scales. It is a best-in-class, highly-scalable scheduler for HPC clusters. It allocates resources, provides a framework for executing tasks, and arbitrates contention for resources by managing queues of pending work. To tell Slurm what resources you need, you will have to create a Slurm job script. A Slurm job script is a small text file containing information about what resources a job requires, including time, number of nodes, and memory. The Slurm script also contains the commands needed to begin executing the desired computation.

The Slurm script generally follows this format:

#!/bin/bash
# Declaring Slurm Configuration Options

# Loading Software/Libraries

# Running Code

Sample Slurm Script

For example, let’s create a sample Slurm job script and submit the job to the cluster. First, create an empty file using vim editor(or your favourite editor) and insert the following script and save it using .slurm or .sh extension (for example, myjob.slurm or myjob.sh):

#!/bin/bash

#SBATCH --job-name=myjob        # Name for your job
#SBATCH --comment="Run My Job"  # Comment for your job

#SBATCH --output=%x_%j.out      # Output file
#SBATCH --error=%x_%j.err       # Error file

#SBATCH --time=0-00:05:00       # Time limit
#SBATCH --nodes=1               # How many nodes to run on
#SBATCH --ntasks=2              # How many tasks per node
#SBATCH --cpus-per-task=2       # Number of CPUs per task
#SBATCH --mem-per-cpu=10g       # Memory per CPU
#SBATCH --qos=short             # priority/quality of service

# Command to run
hostname                        # Run the command hostname

In this example, we run the bash command hostname.

To submit this job, run:

sbatch myjob.slurm

This will submit your job to the Slurm for execution and a message with Job_ID will be displayed.

You can check the status of your jobs using

squeue -u $USER

To obtain complete information about a job (allocated resources and execution status), run:

scontrol show job $Job_ID

To cancel a job, run:

scancel $Job_ID

For more information, please check the official documentation of Slurm.

Always launch your simulations in your scratch folder, not in your home !