Skip to content
Snippets Groups Projects
Commit 7bebda88 authored by Jan Frenzel's avatar Jan Frenzel
Browse files

Merge branch 'merge-preview-in-main' into 'main'

Automated merge from preview to main

See merge request !1090
parents 8a8d5de9 3f6846f2
No related branches found
No related tags found
5 merge requests!1116Main,!1114Foo,!1113Main,!1110Foobar,!1090Automated merge from preview to main
...@@ -7,8 +7,8 @@ another, you have to use the following commands after logging in to any of the Z ...@@ -7,8 +7,8 @@ another, you have to use the following commands after logging in to any of the Z
- `dtcp`, `dtls`, `dtmv`, `dtrm`, `dtrsync`, `dttar`, and `dtwget` - `dtcp`, `dtls`, `dtmv`, `dtrm`, `dtrsync`, `dttar`, and `dtwget`
These special commands submit a [batch job](../jobs_and_resources/slurm.md) to the data transfer machines These special commands submit a [batch job](../jobs_and_resources/slurm.md) to the data transfer
performing the selected command. Their syntax and behavior is the very same as the machines performing the selected command. Their syntax and behavior is the very same as the
well-known shell commands without the prefix *`dt`*, except for the following options. well-known shell commands without the prefix *`dt`*, except for the following options.
| Additional Option | Description | | Additional Option | Description |
...@@ -48,38 +48,40 @@ To identify the mount points of the different filesystems on the data transfer m ...@@ -48,38 +48,40 @@ To identify the mount points of the different filesystems on the data transfer m
!!! example "Copying data from `/beegfs/.global0` to `/projects` filesystem." !!! example "Copying data from `/beegfs/.global0` to `/projects` filesystem."
``` console ```console
marie@login$ dtcp -r /data/old/beegfs/.global0/ws/marie-workdata/results /projects/p_number_crunch/. marie@login$ dtcp -r /data/old/beegfs/.global0/ws/marie-workdata/results /projects/p_number_crunch/.
``` ```
!!! example "Archive data from `/beegfs/.global0` to `/archiv` filesystem." !!! example "Archive data from `/beegfs/.global0` to `/archiv` filesystem."
``` console ```console
marie@login$ dttar -czf /data/archiv/p_number_crunch/results.tgz /data/old/beegfs/global0/ws/marie-workdata/results marie@login$ dttar -czf /data/archiv/p_number_crunch/results.tgz /data/old/beegfs/global0/ws/marie-workdata/results
``` ```
!!! example "Copy data from `/data/horse` to `/projects` filesystem." !!! example "Copy data from `/data/horse` to `/projects` filesystem."
``` console ```console
marie@login$ dtcp -r /data/horse/ws/marie-workdata/results /projects/p_number_crunch/. marie@login$ dtcp -r /data/horse/ws/marie-workdata/results /projects/p_number_crunch/.
``` ```
!!! example "Move data from `/data/horse` to `/data/walrus` filesystem." !!! example "Move data from `/data/horse` to `/data/walrus` filesystem."
``` console ```console
marie@login$ dtmv /data/horse/ws/marie-workdata/results /data/walrus/ws/marie-archive/. marie@login$ dtmv /data/horse/ws/marie-workdata/results /data/walrus/ws/marie-archive/.
``` ```
!!! example "Archive data from `/data/walrus` to `/archiv` filesystem." !!! example "Archive data from `/data/walrus` to `/archiv` filesystem."
``` console ```console
marie@login$ dttar -czf /archiv/p_number_crunch/results.tgz /data/walrus/ws/marie-workdata/results marie@login$ dttar -czf /archiv/p_number_crunch/results.tgz /data/walrus/ws/marie-workdata/results
``` ```
!!! warning !!! warning
Do not generate files in the `/archiv` filesystem much larger that 500 GB! Do not generate files in the `/archiv` filesystem much larger that 500 GB!
!!! note !!! note
The [warm archive](../data_lifecycle/warm_archive.md) and the `projects` filesystem are not The [warm archive](../data_lifecycle/warm_archive.md) and the `projects` filesystem are not
writable from within batch jobs. writable from within batch jobs.
However, you can store the data in the [`walrus` filesystem](../data_lifecycle/working.md) However, you can store the data in the [`walrus` filesystem](../data_lifecycle/working.md)
...@@ -90,7 +92,7 @@ To identify the mount points of the different filesystems on the data transfer m ...@@ -90,7 +92,7 @@ To identify the mount points of the different filesystems on the data transfer m
In order to let the datamover have access to your group drive, copy your public SSH key from ZIH In order to let the datamover have access to your group drive, copy your public SSH key from ZIH
system to `login1.zih.tu-dresden.de`, first. system to `login1.zih.tu-dresden.de`, first.
``` console ```console
marie@login$ ssh-copy-id -i ~/.ssh/id_rsa.pub login1.zih.tu-dresden.de marie@login$ ssh-copy-id -i ~/.ssh/id_rsa.pub login1.zih.tu-dresden.de
# Export the name of your group drive for reuse of example commands # Export the name of your group drive for reuse of example commands
marie@login$ export GROUP_DRIVE_NAME=<my-drive-name> marie@login$ export GROUP_DRIVE_NAME=<my-drive-name>
...@@ -98,12 +100,12 @@ system to `login1.zih.tu-dresden.de`, first. ...@@ -98,12 +100,12 @@ system to `login1.zih.tu-dresden.de`, first.
!!! example "Copy data from your group drive to `/data/horse` filesystem." !!! example "Copy data from your group drive to `/data/horse` filesystem."
``` console ```console
marie@login$ dtrsync -av dgw.zih.tu-dresden.de:/glw/${GROUP_DRIVE_NAME}/inputfile /data/horse/ws/marie-workdata/. marie@login$ dtrsync -av dgw.zih.tu-dresden.de:/glw/${GROUP_DRIVE_NAME}/inputfile /data/horse/ws/marie-workdata/.
``` ```
!!! example "Copy data from `/data/horse` filesystem to your group drive." !!! example "Copy data from `/data/horse` filesystem to your group drive."
``` console ```console
marie@login$ dtrsync -av /data/horse/ws/marie-workdata/resultfile dgw.zih.tu-dresden.de:/glw/${GROUP_DRIVE_NAME}/. marie@login$ dtrsync -av /data/horse/ws/marie-workdata/resultfile dgw.zih.tu-dresden.de:/glw/${GROUP_DRIVE_NAME}/.
``` ```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment