Skip to content
Snippets Groups Projects
Commit 817b1707 authored by Martin Schroschk's avatar Martin Schroschk
Browse files

Apply rules for prompts and code blocks

parent 98e6bb5b
No related branches found
No related tags found
3 merge requests!322Merge preview into main,!319Merge preview into main,!225Data lifecycle mf
...@@ -2,21 +2,24 @@ ...@@ -2,21 +2,24 @@
## Large Files in /scratch ## Large Files in /scratch
The data containers in Lustre are called object storage targets (OST). The capacity of one OST is The data containers in [Lustre](https://www.lustre.org) are called object storage targets (OST). The
about 21 TB. All files are striped over a certain number of these OSTs. For small and medium files, capacity of one OST is about 21 TB. All files are striped over a certain number of these OSTs. For
the default number is 2. As soon as a file grows above \~1 TB it makes sense to spread it over a small and medium files, the default number is 2. As soon as a file grows above ~1 TB it makes sense
higher number of OSTs, e.g. 16. Once the file system is used \> 75%, the average space per OST is to spread it over a higher number of OSTs, e.g. 16. Once the file system is used >75%, the average
only 5 GB. So, it is essential to split your larger files so that the chunks can be saved! space per OST is only 5 GB. So, it is essential to split your larger files so that the chunks can be
saved!
Lets assume you have a directory where you tar your results, e.g. `/scratch/mark/tar`. Now, simply
Lets assume you have a directory where you tar your results, e.g. `/scratch/marie/tar`. Now, simply
set the stripe count to a higher number in this directory with: set the stripe count to a higher number in this directory with:
```Bash ```console
lfs setstripe -c 20 /scratch/ws/mark-stripe20/tar marie@login$ lfs setstripe -c 20 /scratch/ws/marie-stripe20/tar
``` ```
**Note:** This does not affect existing files. But all files that **will be created** in this !!! note
directory will be distributed over 20 OSTs.
This does not affect existing files. But all files that **will be created** in this
directory will be distributed over 20 OSTs.
## Useful Commands for Lustre ## Useful Commands for Lustre
...@@ -24,35 +27,35 @@ These commands work for `/scratch` and `/ssd`. ...@@ -24,35 +27,35 @@ These commands work for `/scratch` and `/ssd`.
### Listing Disk Usages per OST and MDT ### Listing Disk Usages per OST and MDT
```Bash ```console
lfs quota -h -u username /path/to/my/data marie@login$ lfs quota -h -u username /path/to/my/data
``` ```
It is possible to display the usage on each OST by adding the "-v"-parameter. It is possible to display the usage on each OST by adding the argument `-v`.
### Listing space usage per OST and MDT ### Listing Space Usage per OST and MDT
```Bash ```console
lfs df -h /path/to/my/data marie@login$ lfs df -h /path/to/my/data
``` ```
### Listing inode usage for an specific path ### Listing inode usage for an specific path
```Bash ```console
lfs df -i /path/to/my/data marie@login$ lfs df -i /path/to/my/data
``` ```
### Listing OSTs ### Listing OSTs
```Bash ```console
lfs osts /path/to/my/data marie@login$ lfs osts /path/to/my/data
``` ```
### View striping information ### View Striping Information
```Bash ```console
lfs getstripe myfile marie@login$ lfs getstripe myfile
lfs getstripe -d mydirectory marie@login$ lfs getstripe -d mydirectory
``` ```
The `-d`-parameter will also display striping for all files in the directory. The argument `-d` will also display striping for all files in the directory.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment