Skip to content
Snippets Groups Projects
Commit 8ed7228b authored by Ulf Markwardt's avatar Ulf Markwardt
Browse files

update after feedbacks

parent 656ebaea
No related branches found
No related tags found
3 merge requests!899Preview,!897Draft: Preview,!894Barnard update
Hide whitespace changes
Inline Side-by-side
doc.zih.tu-dresden.de/docs/jobs_and_resources/barnard_test.md
+ 106
− 88
View file @ 8ed7228b
...@@ -6,140 +6,158 @@ and workflows for production there. For general hints please refer to these site ...@@ -6,140 +6,158 @@ and workflows for production there. For general hints please refer to these site
* [Details on architecture](/jobs_and_resources/architecture_2023), * [Details on architecture](/jobs_and_resources/architecture_2023),
* [Description of the migration](migration_2023.md). * [Description of the migration](migration_2023.md).
!!! "Feedback welcome" We value your feedback. Please provide it directly via our ticket system. For better processing,
Please provide your feedback directly via our ticket system. For better processing, please add "Barnard:" as a prefix to the subject of the [support ticket](../support/support).
please add "Barnard:" as a prefix to the subject of the [support ticket](../support/support).
Here, you can find few hints which might help you with the first steps. Here, you can find few hints which might help you with the first steps.
## Login to Barnard ## Login to Barnard
* All users and projects from Taurus now can work on Barnard. All users and projects from Taurus now can work on Barnard.
* They can use `login[2-4].barnard.hpc.tu-dresden.de` to access the system
They can use `login[2-4].barnard.hpc.tu-dresden.de` to access the system
from campus (or VPN). [Fingerprints](/access/key_fingerprints/#barnard) from campus (or VPN). [Fingerprints](/access/key_fingerprints/#barnard)
* All users have **new HOME** file systems, this means you have to do two things: All users have **new empty HOME** file systems, this means you have first have to...
1. ??? Install your public ssh key on the system ??? "... install your public ssh key on the system"
- Please create a new SSH keypair with ed25519 encryption, secured with - Please create a new SSH keypair with ed25519 encryption, secured with
a passphrase. Please refer to this a passphrase. Please refer to this
[page for instructions](../../access/ssh_login#before-your-first-connection). [page for instructions](../../access/ssh_login#before-your-first-connection).
- After login, add the public key to your `.ssh/authorized_keys` file - After login, add the public key to your `.ssh/authorized_keys` file
on Barnard. on Barnard.
1. ??? "Transfer Data to New Home Directory"
Your personal (old) home directory at Taurus will not be automatically transferred to the new Barnard ## Data Management
system. **You are responsible for this task.** Please do not copy your entire home, but consider
this opportunity for cleaning up you data. E.g., it might make sense to delete outdated scripts, old * The `/project` filesystem is the same on Taurus and Barnard
log files, etc., and move other files to an archive filesystem. Thus, please transfer only selected (mounted read-only on the compute nodes).
directories and files that you need on the new system. * The new work filesystem is `/data/horse`.
* The slower `/data/walrus` can be considered as a substitute for the old
`/warm_archive`- mounted **read-only** on the compute nodes.
It can be used to store e.g. results.
These `/data/horse` and `/data/walrus` can be accesed via workspaces. Please refer to the
[workspace page](../../data_lifecycle/workspaces/), if you are not familiar with workspaces.
??? "Tips on workspaces"
* To list all available workspace filessystem, invoke the command `ws_list -l`."
* Please use the command `dtinfo` to get the current mount points:
```
marie@login1> dtinfo
[...]
directory on datamover mounting clusters directory on cluster
/data/old/home Taurus /home
/data/old/lustre/scratch2 Taurus /scratch
/data/old/lustre/ssd Taurus /lustre/ssd
[...]
```
!!! Warning
All old filesystems fill be shutdown by the end of 2023.
To work with your data from Taurus you might have to move/copy them to the new storages.
The well-known [datamover tools](../../data_transfer/datamover/) are available to run such transfer For this, we have four new [datamover nodes](/data_transfer/datamover) that have mounted all storages
jobs under Slurm. The steps are as follows: of the old and new system. (Do not use the datamovers from Taurus!)
1. Login to Barnard: `ssh login[1-4].barnard.tu-dresden.de` ??? "Migration from Home Directory"
1. The command `dtinfo` will provide you the mountpoints
```console Your personal (old) home directory at Taurus will not be automatically transferred to the new Barnard
marie@barnard$ dtinfo system. **You are responsible for this task.** Please do not copy your entire home, but consider
[...] this opportunity for cleaning up you data. E.g., it might make sense to delete outdated scripts, old
directory on datamover mounting clusters directory on cluster log files, etc., and move other files to an archive filesystem. Thus, please transfer only selected
directories and files that you need on the new system.
/data/old/home Taurus /home The well-known [datamover tools](../../data_transfer/datamover/) are available to run such transfer
/data/old/lustre/scratch2 Taurus /scratch jobs under Slurm. The steps are as follows:
/data/old/lustre/ssd Taurus /lustre/ssd
[...]
```
1. Use the `dtls` command to list your files on the old home directory: `marie@barnard$ dtls 1. Login to Barnard: `ssh login[1-4].barnard.tu-dresden.de`
/data/old/home/marie` 1. The command `dtinfo` will provide you the mountpoints
1. Use `dtcp` command to invoke a transfer job, e.g.,
```console ```console
marie@barnard$ dtcp --recursive /data/old/home/marie/<useful data> /home/marie/ marie@barnard$ dtinfo
[...]
directory on datamover mounting clusters directory on cluster
/data/old/home Taurus /home
/data/old/lustre/scratch2 Taurus /scratch
/data/old/lustre/ssd Taurus /lustre/ssd
[...]
``` ```
**Note**, please adopt the source and target paths to your needs. All available options can be 1. Use the `dtls` command to list your files on the old home directory: `marie@barnard$ dtls
queried via `dtinfo --help`. /data/old/home/marie`
1. Use `dtcp` command to invoke a transfer job, e.g.,
!!! warning ```console
marie@barnard$ dtcp --recursive /data/old/home/marie/<useful data> /home/marie/
```
Please be aware that there is **no synchronisation process** between your home directories at **Note**, please adopt the source and target paths to your needs. All available options can be
Taurus and Barnard. Thus, with the very first transfer, they will become divergent. queried via `dtinfo --help`.
We recommand to **take some minutes for planing the transfer process**. Do not act with !!! warning
precipitation.
## Data Management Please be aware that there is **no synchronisation process** between your home directories at
Taurus and Barnard. Thus, after the very first transfer, they will become divergent.
* The `/project` filesystem is the same on Taurus and Barnard We recommand to **take some minutes for planing the transfer process**. Do not act with
(mounted read-only on the compute nodes). precipitation.
* The **new work filesystem** is `/data/horse`.
* The slower `/data/walrus` shall substitute the old `/warm_archive` - mounted **read-only** on
the compute nodes. It can be used to store e.g. results.
These two (horse + walrus) can be accesed via workspaces. Please refer to the ??? "Migration from `/lustre/ssd` or `/beegfs`"
[workspace page](../../data_lifecycle/workspaces/), if you are not familiar with workspaces. To list
all available workspace filessystem, invoke the command `ws_list -l`.
!!! Note **You** are entirely responsible for the transfer of these data to the new location.
Start the dtrsync process as soon as possible. (And maybe repeat it at a later time.)
**To work with your data from Taurus you might have to move/copy them to the new storages.** ??? "Migration from `/lustre/scratch2` aka `/scratch`"
For this, we have four new [datamover nodes](/data_transfer/datamover) that have mounted all storages We are synchronizing this (**last: October 18**) to `/data/horse/lustre/scratch2/`.
of the old and new system. (Do not use the datamovers from Taurus!)
Please use the command `dtinfo` to get the current mount points: Please do **NOT** copy those data yourself. Instead check if it is already sychronized
to `/data/walrus/warm_archive/ws`.
``` In case you need to update this (Gigabytes, not Terabytes!) please run `dtrsync` like in
marie@login1> dtinfo `dtrsync -a /data/old/lustre/scratch2/ws/0/my-workspace/newest/ /data/horse/lustre/scratch2/ws/0/my-workspace/newest/`
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST
datamover* up infinite 1 down* service5
datamover* up infinite 3 idle service[1-4]
--------------------------------------------------------------------------------
directory on datamover mounting clusters directory on cluster
/data/old/home Taurus /home ??? "Migration from `/warm_archive`"
/data/old/lustre/scratch2 Taurus /scratch
/data/old/lustre/ssd Taurus /lustre/ssd
/data/old/beegfs Taurus /beegfs
/data/old/warm_archive Taurus /warm_archive
/data/horse Barnard /data/horse
/data/walrus Barnard /data/walrus
--------------------------------------------------------------------------------
```
* In May we have copied all workspaces from `/scratch/ws` data to We are preparing another sync from `/warm_archive` to `The process of syncing data from `/warm_archive` to `/data/walrus/warm_archive` is still ongoing.
` /data/horse/lustre/scratch2/ws`. This replication took a **few weeks**. Ideally you
can now just **move** the content to a newly created workspace.
A second synchronization has started on **October, 18** and is nearly done.
* Please manually copy your needed data from your `beegfs` or `ssd` workspaces. These Please do **NOT** copy those data yourself. Instead check if it is already sychronized
old storages will be purged, probably by the end of November. to `/data/walrus/warm_archive/ws`.
The process of syncing data from `/warm_archive` to `/data/walrus` is still ongoing. In case you need to update this (Gigabytes, not Terabytes!) please run `dtrsync` like in
`dtrsync -a /data/old/warm_archive/ws/my-workspace/newest/ /data/walrus/warm_archive/ws/my-workspace/newest/`
When the last compute system will have been migrated the old file systems will be
set write-protected and we start a final synchronization (sratch+walrus).
The target directories for synchronization `/data/horse/lustre/scratch2/ws` and
`/data/walrus/warm_archive/ws/` will not be deleted automatically in the mean time.
## Software ## Software
Please use `module spider` to identify the software modules you need to load. Like Please use `module spider` to identify the software modules you need to load.Like
on Taurus. on Taurus.
The default release version is 23.10.
## Slurm ## Slurm
* We are running the most recent Slurm version. * We are running the most recent Slurm version.
* You must not use the old partition names. * You must not use the old partition names.
* Not all things are tested. * Not all things are tested.
## Updates after your feedback ## Updates after your feedback (state: October 19)
* A **second synchronization** from `/scratch` has started on **October, 18** and is nearly done. * A **second synchronization** from `/scratch` has started on **October, 18**, and is
* The **data tranfer tools** now work fine. now nearly done.
* A first, and incomplete synchronization from `/warm_archive` has been done (see above).
With support from NEC we are transferring the rest in the next weeks.
* The **data transfer tools** now work fine.
* After fixing too tight security restrictions, **all users can login** now. * After fixing too tight security restrictions, **all users can login** now.
* **ANSYS** now starts: please check if your specific use case works. * **ANSYS** now starts: please check if your specific use case works.
* **login1** is under construction, do not use it at the moment. Workspace creation does * **login1** is under construction, do not use it at the moment. Workspace creation does
not work there. not work there.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
Nutzungsbedingungen | Datenschutzerklärung | Barrierefreiheitserklärung