diff --git a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md index 17394c63f91dc6536cc6351a91d52f6a972cb278..4a980612cce41bd9e617fee06f4da6344fc3fb53 100644 --- a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md +++ b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md @@ -1,26 +1,5 @@ # Lustre -## Large Files in /scratch - -The data containers in [Lustre](https://www.lustre.org) are called **object storage targets (OST)**. -The capacity of one OST is about 21 TB. All files are striped over a certain number of these OSTs. -For small and medium files, the default number is 2. As soon as a file grows above ~1 TB it makes -sense to spread it over a higher number of OSTs, e.g. 16. Once the filesystem is used >75%, the -average space per OST is only 5 GB. So, it is essential to split your larger files so that the -chunks can be saved! - -Lets assume you have a directory where you tar your results, e.g. `/scratch/marie/tar`. Now, simply -set the stripe count to a higher number in this directory with: - -```console -marie@login$ lfs setstripe -c 20 /scratch/ws/marie-stripe20/tar -``` - -!!! note - - This does not affect existing files. But all files that **will be created** in this - directory will be distributed over 20 OSTs. - ## Good Practices !!! hint "Avoid accessing metadata information"