Skip to content
Snippets Groups Projects
Commit 7042012e authored by Gitlab Bot's avatar Gitlab Bot
Browse files

Merge branch 'preview' into merge-preview-in-main

parents 16ff20c8 50a49b9b
No related branches found
No related tags found
5 merge requests!1116Main,!1114Foo,!1113Main,!1110Foobar,!1098Automated merge from preview to main
# Lustre
## Large Files in /scratch
The data containers in [Lustre](https://www.lustre.org) are called **object storage targets (OST)**.
The capacity of one OST is about 21 TB. All files are striped over a certain number of these OSTs.
For small and medium files, the default number is 2. As soon as a file grows above ~1 TB it makes
sense to spread it over a higher number of OSTs, e.g. 16. Once the filesystem is used >75%, the
average space per OST is only 5 GB. So, it is essential to split your larger files so that the
chunks can be saved!
Lets assume you have a directory where you tar your results, e.g. `/scratch/marie/tar`. Now, simply
set the stripe count to a higher number in this directory with:
```console
marie@login$ lfs setstripe -c 20 /scratch/ws/marie-stripe20/tar
```
!!! note
This does not affect existing files. But all files that **will be created** in this
directory will be distributed over 20 OSTs.
## Good Practices
!!! hint "Avoid accessing metadata information"
......@@ -119,18 +98,15 @@ setstripe getstripe setdirstripe getdirstripe
mkdir rm_entry pool_list find
check osts mdts df
[...]
marie@login lfs help setstripe
```
To get more information on a specific option, enter `help` followed by the option of interest:
```console
marie@login lfs help setstripe
setstripe: To create a file with specified striping/composite layout, or
create/replace the default layout on an existing directory:
usage: setstripe [--component-end|-E <comp_end>]
[...]
marie@login lfs help df
df: report filesystem disk space usage or inodes usage of each MDS and all OSDs or a batch belonging to a specific pool.
Usage: df [--inodes|-i] [--human-readable|-h] [--lazy|-l]
[--pool|-p <fsname>[.<pool>]] [path]
```
More comprehensive documentation can be found in the man pages of lfs (`man lfs`).
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment