2. Storage Policy
There are distinct disk spaces available for each user: /home,/scratch,/micromamba or by project.
Each space has specific characteristics suitable to its usage which are described below.
To know how much the disk spaces are occupied for your project, you can see on your profile CLAM page.
Space Information in CLAM profile page :
By user :
By project :
| The command 'du' returns information in real time but its execution can take a long time depending on the size of the directory concerned. |
2.1. The User Directory
2.1.1. Home directory
$HOME : This is the home directory during an interactive connection login by SSH. This space is intended for frequently-used small-sized files such as the shell environment files, the tools, and potentially the sources and libraries if they have a reasonable size. The size of this space is limited (in space and in number of files).
The HOME characteristics are:
-
It is a private space.
-
It is mounted automatically at each connection
-
It is not backed up.
-
Not suitable for high IO data transfert
-
Intended to receive small-sized files.
-
Submitted to quota per user (Actually 3 To for betatester but will be reduce to 30 Go nearly) and not extendable.
-
It is the home directory during an interactive connection.
-
Accessible in interactive or in a batch job via the $HOME variable :
$ cd $HOME
2.1.2. The scratch Directory
/scratch/clustername/users/loginname This is an automount work and storage space which is usable in batch; You must not keep the data on this space for more than 90 days. It’s for :
-
The large-sized files used during batch executions
-
result files
-
computation restarts.
Once the post-processing has been done to reduce the data volume, you must remember to copy the significant files into your LABDATA dir so that they are not lost, or transfert to outside storage as a duplicated backup.
The characteristics of the SCRATCH are:
-
It is a private space and not shareable to others.
-
It is not backed up.
-
It is intended to receive large-sized files.
-
It is not submitted to a quota policy (but can be applied per projet if used abusely)
-
It is accessible in interactive or in a batch job.
cd /scratch/clustername<2>/users/login-name/(1)| 1 | :To be replaced by the correct login assigned to you by clam, obviously. For example :doe-j@univ-nantes.fr : for Nantes University or doe-j@ec-nantes.fr for Nantes Central school or doe-j@univ-angers.fr for Angers University etc.. |
| 2 | {CLUSTER_NAME} |
2.1.3. The micromamba Directory
/micromamba/ The micromamba directory is made to host your personal micromamba environments. You must use this space with micromamba.
-
It is a private space and not shareable to others.
-
It is not backed up.
-
It is intended to receive micromamba envirnoment.
-
It is not submitted to a quota policy (but can be applied per projet if used abusely)
-
It is accessible in interactive or in a batch job.
cd /micromamba/login-name/(1)| 1 | :To be replaced by the correct login assigned to you by clam, obviously. For example :doe-j@univ-nantes.fr : for Nantes University or doe-j@ec-nantes.fr for Nantes Central school or doe-j@univ-angers.fr for Angers University etc.. |
2.2. The LABDATA directory
it is a permanent work and storage space that can be used in batches. Large files that are used during batch executions are generally stored there: large source files and libraries, data files, executables, result files, submission scripts.
| in CLAM, you must submit your project and request disk space which can be subject to pricing |
The characteristics of storage /LAB-DATA/GLiCID/users/ or /LAB-DATA/GLiCID/projects/ are:
-
It is a permanent
-
Not suitable for high IO data transfert
-
It is not backed up.
-
Intended to receive large-sized files with a certain limit
-
Submitted to quotas per user (3 To) and per project (It depends your demand on CLAM)
-
Accessible in interactive or in a batch job
There are 2 possibilities:
-
for each user
$ cd /LAB-DATA/GLiCID/users/(1)
| 1 | :To be replaced by the correct login assigned to you by clam, obviously. For example :doe-j@univ-nantes.fr : for Nantes University or doe-j@ec-nantes.fr for Nantes Central school or doe-j@univ-angers.fr for Angers University etc..
|
| here are the files shared for the project |
$ cd /LAB-DATA/GLiCID/projects/<projectname>2.3. Summary table main disks
Space |
Default Capacity |
Features |
$HOME |
30 Go per user(*) |
Home directory at connection, Space not backed up actually |
/micromamba/login-name/ |
No quota actually, 500 GB shared by all users |
Only for microconda env, Space not backed up, use only for software not available elsewhere ! |
/scratch/nautilus/login-name |
No quota, 450 TB shared by all users |
GPFS Low Latency Storage (12 GB/s read/write operations) No keep data on this space , Space not backed up |
/LAB-DATA/GLiCID/users/<username> |
3 To per user |
Limited space, Space no backup up. |
/LAB-DATA/GLiCID/projects/<projectname> |
It depends on what is asked in CLAM |
Limited space, Space no backup up. |