Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
hpc-compendium
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
ZIH
hpcsupport
hpc-compendium
Commits
5f681e6e
Commit
5f681e6e
authored
1 year ago
by
Martin Schroschk
Browse files
Options
Downloads
Patches
Plain Diff
Cont: Restruture for better guidance
parent
c91ca110
No related branches found
Branches containing commit
No related tags found
2 merge requests
!920
Automated merge from preview to main
,
!918
Barnard cleanup
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
doc.zih.tu-dresden.de/docs/jobs_and_resources/barnard_test.md
+32
-16
32 additions, 16 deletions
...zih.tu-dresden.de/docs/jobs_and_resources/barnard_test.md
with
32 additions
and
16 deletions
doc.zih.tu-dresden.de/docs/jobs_and_resources/barnard_test.md
+
32
−
16
View file @
5f681e6e
...
@@ -109,7 +109,7 @@ migrated to the new filesystems on Barnard. This migration comprises
...
@@ -109,7 +109,7 @@ migrated to the new filesystems on Barnard. This migration comprises
Please **do not copy your entire data** from the old to the new filesystems, but consider this
Please **do not copy your entire data** from the old to the new filesystems, but consider this
opportunity for **cleaning up your data**. E.g., it might make sense to delete outdated scripts,
opportunity for **cleaning up your data**. E.g., it might make sense to delete outdated scripts,
old log files, etc., and move other files
to an archive
filesystem.
old log files, etc., and move other files
, e.g., results, to the `/data/walrus`
filesystem.
!!! hint "Generic login"
!!! hint "Generic login"
...
@@ -122,19 +122,40 @@ of the old Taurus and new Barnard system. Do not use the datamovers from Taurus,
...
@@ -122,19 +122,40 @@ of the old Taurus and new Barnard system. Do not use the datamovers from Taurus,
transfer need to be invoked from Barnard! Thus, the very first step is to
transfer need to be invoked from Barnard! Thus, the very first step is to
[
login to Barnard
](
#login-to-barnard
)
.
[
login to Barnard
](
#login-to-barnard
)
.
The command
`dtinfo`
will provide you the mountpoints of the old filesystems
```
console
marie@barnard$
dtinfo
[...]
directory on datamover mounting clusters directory on cluster
/data/old/home Taurus /home
/data/old/lustre/scratch2 Taurus /scratch
/data/old/lustre/ssd Taurus /lustre/ssd
[...]
```
In terms of working data, make a plan which directories and files need to be transferred to
`/data/horse`
and which to
`/data/walrus`
. Therefor, the following instructions and examples are
divided by the
In the following, we will provide instructions with comprehensive examples for the data transfer of
your data to the new
`/home`
filesystem, as well as the working filesystems
`/data/horse`
and
`/data/walrus`
. With respect to the later, the instructions and examples are divided by the target
not the source filesystem.
??? "Migration from Home Directory"
??? "Migration from Home Directory"
Your personal (old) home directory at Taurus will not be automatically transferred to the new
Your personal (old) home directory at Taurus will not be automatically transferred to the new
Barnard system.
**You are responsible for this task.**
Please do not copy your entire home, but
Barnard system. Please do not copy your entire home, but
clean up your data. E.g., it might
consider this opportunity for cleaning up your data. E.g., it might make sense to delet
e
make sense to delete outdated scripts, old log files, etc., and move other files to an archiv
e
outdated scripts, old log files, etc., and move other files to an archive filesystem. Thus,
filesystem. Thus, please transfer only selected directories and files that you need on the new
please transfer only selected directories and files that you need on the new
system.
system.
The well-known [datamover tools](../../data_transfer/datamover/) are available to run such
The steps are as follows:
transfer jobs under Slurm. The steps are as follows:
1. Login to Barnard, i.e., `ssh login[1-4].barnard.tu-dresden.de`
1. Login to Barnard, i.e., `ssh login[1-4].barnard.tu-dresden.de`
1. The command `dtinfo` will provide you the mountpoint
s
1. The command `dtinfo` will provide you the mountpoint
```console
```console
marie@barnard$ dtinfo
marie@barnard$ dtinfo
...
@@ -142,8 +163,6 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
...
@@ -142,8 +163,6 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
directory on datamover mounting clusters directory on cluster
directory on datamover mounting clusters directory on cluster
/data/old/home Taurus /home
/data/old/home Taurus /home
/data/old/lustre/scratch2 Taurus /scratch
/data/old/lustre/ssd Taurus /lustre/ssd
[...]
[...]
```
```
...
@@ -154,7 +173,7 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
...
@@ -154,7 +173,7 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
[...]
[...]
```
```
1. Use `dtcp` command to invoke a transfer job, e.g.,
1. Use
the
`dtcp` command to invoke a transfer job, e.g.,
```console
```console
marie@barnard$ dtcp --recursive /data/old/home/marie/<useful data> /home/marie/
marie@barnard$ dtcp --recursive /data/old/home/marie/<useful data> /home/marie/
...
@@ -165,11 +184,8 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
...
@@ -165,11 +184,8 @@ transfer need to be invoked from Barnard! Thus, the very first step is to
!!! warning
!!! warning
Please be aware that there is **no synchronisation process** between your home directories at
Please be aware that there is **no synchronisation process** between your home directories
Taurus and Barnard. Thus, after the very first transfer, they will become divergent.
at Taurus and Barnard. Thus, after the very first transfer, they will become divergent.
We recommand to **take some minutes for planing the transfer process**. Do not act with
precipitation.
#### Migration of Working Data from Old Filesystems `ssd`, `beegfs`, and `scratch`
#### Migration of Working Data from Old Filesystems `ssd`, `beegfs`, and `scratch`
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment