From 8e8b401e6f997386491d05549055d489a51f00e3 Mon Sep 17 00:00:00 2001 From: Ulf Markwardt <ulf.markwardt@tu-dresden.de> Date: Mon, 6 Nov 2023 15:13:04 +0100 Subject: [PATCH] updated fs --- .../jobs_and_resources/architecture_2023.md | 31 +++++++++++-------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/doc.zih.tu-dresden.de/docs/jobs_and_resources/architecture_2023.md b/doc.zih.tu-dresden.de/docs/jobs_and_resources/architecture_2023.md index 8c2c5078e..392686171 100644 --- a/doc.zih.tu-dresden.de/docs/jobs_and_resources/architecture_2023.md +++ b/doc.zih.tu-dresden.de/docs/jobs_and_resources/architecture_2023.md @@ -28,26 +28,26 @@ computations, please use interactive jobs. ## Storage Systems +For an easier grasp on the major categories (size, speed), the +work filesystems now come with the names of animals. + ### Permanent Filesystems -We now have `/home`, `/projects` and `/software` in a Lustre filesystem. Snapshots -and tape backup are configured. For convenience, we will make the old home available -read-only as `/home_old` on the data mover nodes for the data migration period. +We now have `/home` and `/software` in a Lustre filesystem. Snapshots +and tape backup are configured. (`/projects` remains the same until a recabling.) -`/warm_archive` is mounted on the data movers, only. +The Lustre filesystem `/data/walrus` is meant for larger data with a slow +access. It is installed to replace `/warm_archive`. ### Work Filesystems -With new players with new software in the filesystem market it is getting more and more -complicated to identify the best suited filesystem for temporary data. In many cases, -only tests can provide the right answer, for a short time. - -For an easier grasp on the major categories (size, speed), the work filesystems now come -with the names of animals: +In the filesystem market with new players it is getting more and more +complicated to identify the best suited filesystem for a specific use case. Often, +only tests can find the best setup for a specific workload. * `/data/horse` - 20 PB - high bandwidth (Lustre) -* `/data/octopus` - 0.5 PB - for interactive usage (Lustre) -* `/data/weasel` - 1 PB - for high IOPS (WEKA) - coming soon +* `/data/octopus` - 0.5 PB - for interactive usage (Lustre) - to be mounted on AlphaCentauri +* `/data/weasel` - 1 PB - for high IOPS (WEKA) - coming 2024. ### Difference Between "Work" And "Permanent" @@ -55,8 +55,11 @@ A large number of changing files is a challenge for any backup system. To protec our snapshots and backup from work data, `/projects` cannot be used for temporary data on the compute nodes - it is mounted read-only. +For `/home`, we create snapshots and tape backups. That's why working there, +with a high frequency of changing files is a bad idea. + Please use our data mover mechanisms to transfer worthy data to permanent -storages. +storages or long-term archives. ## Migration Phase @@ -64,8 +67,10 @@ For about one month, the new cluster Barnard, and the old cluster Taurus will run side-by-side - both with their respective filesystems. We provide a comprehensive [description of the migration to Barnard](migration_to_barnard.md). +<! -- The follwing figure provides a graphical overview of the overall process (red: user action required):  {: align=center} +--> \ No newline at end of file -- GitLab