diff --git a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md index 17394c63f91dc6536cc6351a91d52f6a972cb278..6d5abe132222e0315b0c12d54bce02dcf6c9aff1 100644 --- a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md +++ b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md @@ -1,26 +1,5 @@ # Lustre -## Large Files in /scratch - -The data containers in [Lustre](https://www.lustre.org) are called **object storage targets (OST)**. -The capacity of one OST is about 21 TB. All files are striped over a certain number of these OSTs. -For small and medium files, the default number is 2. As soon as a file grows above ~1 TB it makes -sense to spread it over a higher number of OSTs, e.g. 16. Once the filesystem is used >75%, the -average space per OST is only 5 GB. So, it is essential to split your larger files so that the -chunks can be saved! - -Lets assume you have a directory where you tar your results, e.g. `/scratch/marie/tar`. Now, simply -set the stripe count to a higher number in this directory with: - -```console -marie@login$ lfs setstripe -c 20 /scratch/ws/marie-stripe20/tar -``` - -!!! note - - This does not affect existing files. But all files that **will be created** in this - directory will be distributed over 20 OSTs. - ## Good Practices !!! hint "Avoid accessing metadata information" @@ -119,18 +98,15 @@ setstripe getstripe setdirstripe getdirstripe mkdir rm_entry pool_list find check osts mdts df [...] -marie@login lfs help setstripe - ``` To get more information on a specific option, enter `help` followed by the option of interest: ```console -marie@login lfs help setstripe -setstripe: To create a file with specified striping/composite layout, or -create/replace the default layout on an existing directory: -usage: setstripe [--component-end|-E <comp_end>] -[...] +marie@login lfs help df +df: report filesystem disk space usage or inodes usage of each MDS and all OSDs or a batch belonging to a specific pool. +Usage: df [--inodes|-i] [--human-readable|-h] [--lazy|-l] + [--pool|-p <fsname>[.<pool>]] [path] ``` More comprehensive documentation can be found in the man pages of lfs (`man lfs`).