Skip to content
Snippets Groups Projects
Commit 5f681e6e authored by Martin Schroschk's avatar Martin Schroschk
Browse files

Cont: Restruture for better guidance

parent c91ca110
No related branches found
No related tags found
2 merge requests!920Automated merge from preview to main,!918Barnard cleanup
...@@ -109,7 +109,7 @@ migrated to the new filesystems on Barnard. This migration comprises ...@@ -109,7 +109,7 @@ migrated to the new filesystems on Barnard. This migration comprises
Please **do not copy your entire data** from the old to the new filesystems, but consider this Please **do not copy your entire data** from the old to the new filesystems, but consider this
opportunity for **cleaning up your data**. E.g., it might make sense to delete outdated scripts, opportunity for **cleaning up your data**. E.g., it might make sense to delete outdated scripts,
old log files, etc., and move other files to an archive filesystem. old log files, etc., and move other files, e.g., results, to the `/data/walrus` filesystem.
!!! hint "Generic login" !!! hint "Generic login"
...@@ -122,19 +122,40 @@ of the old Taurus and new Barnard system. Do not use the datamovers from Taurus, ...@@ -122,19 +122,40 @@ of the old Taurus and new Barnard system. Do not use the datamovers from Taurus,
transfer need to be invoked from Barnard! Thus, the very first step is to transfer need to be invoked from Barnard! Thus, the very first step is to
[login to Barnard](#login-to-barnard). [login to Barnard](#login-to-barnard).
The command `dtinfo` will provide you the mountpoints of the old filesystems
```console
marie@barnard$ dtinfo
[...]
directory on datamover mounting clusters directory on cluster
/data/old/home Taurus /home
/data/old/lustre/scratch2 Taurus /scratch
/data/old/lustre/ssd Taurus /lustre/ssd
[...]
```
In terms of working data, make a plan which directories and files need to be transferred to
`/data/horse` and which to `/data/walrus`. Therefor, the following instructions and examples are
divided by the
In the following, we will provide instructions with comprehensive examples for the data transfer of
your data to the new `/home` filesystem, as well as the working filesystems `/data/horse` and
`/data/walrus`. With respect to the later, the instructions and examples are divided by the target
not the source filesystem.
??? "Migration from Home Directory" ??? "Migration from Home Directory"
Your personal (old) home directory at Taurus will not be automatically transferred to the new Your personal (old) home directory at Taurus will not be automatically transferred to the new
Barnard system. **You are responsible for this task.** Please do not copy your entire home, but Barnard system. Please do not copy your entire home, but clean up your data. E.g., it might
consider this opportunity for cleaning up your data. E.g., it might make sense to delete make sense to delete outdated scripts, old log files, etc., and move other files to an archive
outdated scripts, old log files, etc., and move other files to an archive filesystem. Thus, filesystem. Thus, please transfer only selected directories and files that you need on the new
please transfer only selected directories and files that you need on the new system. system.
The well-known [datamover tools](../../data_transfer/datamover/) are available to run such The steps are as follows:
transfer jobs under Slurm. The steps are as follows:
1. Login to Barnard, i.e., `ssh login[1-4].barnard.tu-dresden.de` 1. Login to Barnard, i.e., `ssh login[1-4].barnard.tu-dresden.de`
1. The command `dtinfo` will provide you the mountpoints 1. The command `dtinfo` will provide you the mountpoint
```console ```console
marie@barnard$ dtinfo marie@barnard$ dtinfo
...@@ -142,8 +163,6 @@ transfer need to be invoked from Barnard! Thus, the very first step is to ...@@ -142,8 +163,6 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
directory on datamover mounting clusters directory on cluster directory on datamover mounting clusters directory on cluster
/data/old/home Taurus /home /data/old/home Taurus /home
/data/old/lustre/scratch2 Taurus /scratch
/data/old/lustre/ssd Taurus /lustre/ssd
[...] [...]
``` ```
...@@ -154,7 +173,7 @@ transfer need to be invoked from Barnard! Thus, the very first step is to ...@@ -154,7 +173,7 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
[...] [...]
``` ```
1. Use `dtcp` command to invoke a transfer job, e.g., 1. Use the `dtcp` command to invoke a transfer job, e.g.,
```console ```console
marie@barnard$ dtcp --recursive /data/old/home/marie/<useful data> /home/marie/ marie@barnard$ dtcp --recursive /data/old/home/marie/<useful data> /home/marie/
...@@ -165,11 +184,8 @@ transfer need to be invoked from Barnard! Thus, the very first step is to ...@@ -165,11 +184,8 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
!!! warning !!! warning
Please be aware that there is **no synchronisation process** between your home directories at Please be aware that there is **no synchronisation process** between your home directories
Taurus and Barnard. Thus, after the very first transfer, they will become divergent. at Taurus and Barnard. Thus, after the very first transfer, they will become divergent.
We recommand to **take some minutes for planing the transfer process**. Do not act with
precipitation.
#### Migration of Working Data from Old Filesystems `ssd`, `beegfs`, and `scratch` #### Migration of Working Data from Old Filesystems `ssd`, `beegfs`, and `scratch`
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment