From 817b1707ff4df96082d4d19f386ca3511e1f83b6 Mon Sep 17 00:00:00 2001 From: Martin Schroschk <martin.schroschk@tu-dresden.de> Date: Thu, 5 Aug 2021 08:18:52 +0200 Subject: [PATCH] Apply rules for prompts and code blocks --- .../docs/data_lifecycle/lustre.md | 55 ++++++++++--------- 1 file changed, 29 insertions(+), 26 deletions(-) diff --git a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md index f96ceb0c2..891808543 100644 --- a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md +++ b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md @@ -2,21 +2,24 @@ ## Large Files in /scratch -The data containers in Lustre are called object storage targets (OST). The capacity of one OST is -about 21 TB. All files are striped over a certain number of these OSTs. For small and medium files, -the default number is 2. As soon as a file grows above \~1 TB it makes sense to spread it over a -higher number of OSTs, e.g. 16. Once the file system is used \> 75%, the average space per OST is -only 5 GB. So, it is essential to split your larger files so that the chunks can be saved! - -Lets assume you have a directory where you tar your results, e.g. `/scratch/mark/tar`. Now, simply +The data containers in [Lustre](https://www.lustre.org) are called object storage targets (OST). The +capacity of one OST is about 21 TB. All files are striped over a certain number of these OSTs. For +small and medium files, the default number is 2. As soon as a file grows above ~1 TB it makes sense +to spread it over a higher number of OSTs, e.g. 16. Once the file system is used >75%, the average +space per OST is only 5 GB. So, it is essential to split your larger files so that the chunks can be +saved! + +Lets assume you have a directory where you tar your results, e.g. `/scratch/marie/tar`. Now, simply set the stripe count to a higher number in this directory with: -```Bash -lfs setstripe -c 20 /scratch/ws/mark-stripe20/tar +```console +marie@login$ lfs setstripe -c 20 /scratch/ws/marie-stripe20/tar ``` -**Note:** This does not affect existing files. But all files that **will be created** in this -directory will be distributed over 20 OSTs. +!!! note + + This does not affect existing files. But all files that **will be created** in this + directory will be distributed over 20 OSTs. ## Useful Commands for Lustre @@ -24,35 +27,35 @@ These commands work for `/scratch` and `/ssd`. ### Listing Disk Usages per OST and MDT -```Bash -lfs quota -h -u username /path/to/my/data +```console +marie@login$ lfs quota -h -u username /path/to/my/data ``` -It is possible to display the usage on each OST by adding the "-v"-parameter. +It is possible to display the usage on each OST by adding the argument `-v`. -### Listing space usage per OST and MDT +### Listing Space Usage per OST and MDT -```Bash -lfs df -h /path/to/my/data +```console +marie@login$ lfs df -h /path/to/my/data ``` ### Listing inode usage for an specific path -```Bash -lfs df -i /path/to/my/data +```console +marie@login$ lfs df -i /path/to/my/data ``` ### Listing OSTs -```Bash -lfs osts /path/to/my/data +```console +marie@login$ lfs osts /path/to/my/data ``` -### View striping information +### View Striping Information -```Bash -lfs getstripe myfile -lfs getstripe -d mydirectory +```console +marie@login$ lfs getstripe myfile +marie@login$ lfs getstripe -d mydirectory ``` -The `-d`-parameter will also display striping for all files in the directory. +The argument `-d` will also display striping for all files in the directory. -- GitLab