From 613aeb5319acb49ed5112a42af2603c80e07c00b Mon Sep 17 00:00:00 2001 From: Danny Marc Rotscher <danny.rotscher@tu-dresden.de> Date: Fri, 28 Jun 2024 11:24:21 +0200 Subject: [PATCH] Update lustre.md --- .../docs/data_lifecycle/lustre.md | 21 ------------------- 1 file changed, 21 deletions(-) diff --git a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md index 17394c63f..4a980612c 100644 --- a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md +++ b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md @@ -1,26 +1,5 @@ # Lustre -## Large Files in /scratch - -The data containers in [Lustre](https://www.lustre.org) are called **object storage targets (OST)**. -The capacity of one OST is about 21 TB. All files are striped over a certain number of these OSTs. -For small and medium files, the default number is 2. As soon as a file grows above ~1 TB it makes -sense to spread it over a higher number of OSTs, e.g. 16. Once the filesystem is used >75%, the -average space per OST is only 5 GB. So, it is essential to split your larger files so that the -chunks can be saved! - -Lets assume you have a directory where you tar your results, e.g. `/scratch/marie/tar`. Now, simply -set the stripe count to a higher number in this directory with: - -```console -marie@login$ lfs setstripe -c 20 /scratch/ws/marie-stripe20/tar -``` - -!!! note - - This does not affect existing files. But all files that **will be created** in this - directory will be distributed over 20 OSTs. - ## Good Practices !!! hint "Avoid accessing metadata information" -- GitLab