Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
hpc-compendium
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
ZIH
hpcsupport
hpc-compendium
Commits
c6763b60
Commit
c6763b60
authored
10 months ago
by
Martin Schroschk
Browse files
Options
Downloads
Patches
Plain Diff
Apply generic name for workspace
parent
07625cb1
No related branches found
Branches containing commit
No related tags found
2 merge requests
!1086
Automated merge from preview to main
,
!1085
Apply generic name for workspace
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
doc.zih.tu-dresden.de/docs/quickstart/getting_started.md
+14
-13
14 additions, 13 deletions
doc.zih.tu-dresden.de/docs/quickstart/getting_started.md
with
14 additions
and
13 deletions
doc.zih.tu-dresden.de/docs/quickstart/getting_started.md
+
14
−
13
View file @
c6763b60
...
@@ -148,9 +148,9 @@ To start we recommend the Lustre filesystem **horse**.
...
@@ -148,9 +148,9 @@ To start we recommend the Lustre filesystem **horse**.
The following command creates a workspace
The following command creates a workspace
```console
```console
marie@login$ ws_allocate -F horse -r 7 -m marie@tu-dresden.de -n
test-workspace
-d 90
marie@login$ ws_allocate -F horse -r 7 -m marie@tu-dresden.de -n
number_crunch
-d 90
Info: creating workspace.
Info: creating workspace.
/data/horse/ws/marie-
test-workspace
/data/horse/ws/marie-
number_crunch
remaining extensions : 10
remaining extensions : 10
remaining time in days: 90
remaining time in days: 90
```
```
...
@@ -160,10 +160,10 @@ To start we recommend the Lustre filesystem **horse**.
...
@@ -160,10 +160,10 @@ To start we recommend the Lustre filesystem **horse**.
- `ws_allocate` - command to allocate
- `ws_allocate` - command to allocate
- `-F horse` - on the horse filesystem
- `-F horse` - on the horse filesystem
- `-r 7 -m marie@tu-dresden.de` - send a reminder to `marie@tu-dresden.de` 7 days before expiration
- `-r 7 -m marie@tu-dresden.de` - send a reminder to `marie@tu-dresden.de` 7 days before expiration
- `-n
test-workspace
` - workspace name
- `-n
number_crunch
` - workspace name
- `-d 90` - a life time of 90 days
- `-d 90` - a life time of 90 days
The path to this workspace is `/data/horse/ws/marie-
test-workspace
`. You will need it when
The path to this workspace is `/data/horse/ws/marie-
number_crunch
`. You will need it when
transferring data or running jobs.
transferring data or running jobs.
Find more
[
information on workspaces in the compendium
](
../data_lifecycle/workspaces.md
)
.
Find more
[
information on workspaces in the compendium
](
../data_lifecycle/workspaces.md
)
.
...
@@ -177,7 +177,7 @@ The approach depends on the data volume: up to 100 MB or above.
...
@@ -177,7 +177,7 @@ The approach depends on the data volume: up to 100 MB or above.
Use the command `cp` to copy the file `example.R` from your ZIH home directory to a workspace:
Use the command `cp` to copy the file `example.R` from your ZIH home directory to a workspace:
```console
```console
marie@login$ cp /home/marie/example.R /data/horse/ws/marie-
test-workspace
marie@login$ cp /home/marie/example.R /data/horse/ws/marie-
number_crunch
```
```
Analogously use command `mv` to move a file.
Analogously use command `mv` to move a file.
...
@@ -191,7 +191,7 @@ The approach depends on the data volume: up to 100 MB or above.
...
@@ -191,7 +191,7 @@ The approach depends on the data volume: up to 100 MB or above.
filesystem location to another:
filesystem location to another:
```console
```console
marie@login$ dtcp -r /walrus/ws/large-dataset /data/horse/ws/marie-
test-workspace
/data
marie@login$ dtcp -r /walrus/ws/large-dataset /data/horse/ws/marie-
number_crunch
/data
```
```
Analogously use the command `dtmv` to move a file or folder.
Analogously use the command `dtmv` to move a file or folder.
...
@@ -199,31 +199,32 @@ The approach depends on the data volume: up to 100 MB or above.
...
@@ -199,31 +199,32 @@ The approach depends on the data volume: up to 100 MB or above.
transfer section.
transfer section.
### Transferring Data *To/From* ZIH HPC Systems
### Transferring Data *To/From* ZIH HPC Systems
<!-- [NT] currently not available
???+ example "
`scp`
for transferring data to ZIH HPC systems"
???+ example "
`scp`
for transferring data to ZIH HPC systems"
Copy the file `example.R` from your local machine to a workspace on the ZIH systems:
Copy the file `example.R` from your local machine to a workspace on the ZIH systems:
```console
```console
marie@local$ scp /home/marie/Documents/example.R marie@dataport1.hpc.tu-dresden.de:/data/horse/ws/
your_workspace
/
marie@local$ scp /home/marie/Documents/example.R marie@dataport1.hpc.tu-dresden.de:/data/horse/ws/
marie-number_crunch
/
Password:
Password:
example.R 100% 312 32.2KB/s 00:00
``
example.R 100% 312 32.2KB/s 00:00
```
```
Note, the target path contains `dataport1.hpc.tu-dresden.de`, which is one of the
Note, the target path contains `dataport1.hpc.tu-dresden.de`, which is one of the
so called [dataport nodes](../data_transfer/dataport_nodes.md) that allows for data transfer from/to the outside.
so called [dataport nodes](../data_transfer/dataport_nodes.md) that allows for data transfer
from/to the outside.
???+ example "
`scp`
to transfer data from ZIH HPC systems to local machine"
???+ example "
`scp`
to transfer data from ZIH HPC systems to local machine"
Copy the file `results.csv` from a workspace on the ZIH HPC systems to your local machine:
Copy the file `results.csv` from a workspace on the ZIH HPC systems to your local machine:
```console
```console
marie@local$ scp marie@dataport1.hpc.tu-dresden.de:/data/horse/ws/marie-
test-workspace
/results.csv /home/marie/Documents/
marie@local$ scp marie@dataport1.hpc.tu-dresden.de:/data/horse/ws/marie-
number_crunch
/results.csv /home/marie/Documents/
```
```
Feel free to explore further [examples](http://bropages.org/scp) of the `scp` command
Feel free to explore further [examples](http://bropages.org/scp) of the `scp` command
and possibilities of the [dataport nodes](../data_transfer/dataport_nodes.md).
and possibilities of the [dataport nodes](../data_transfer/dataport_nodes.md).
-->
!!! caution "Terabytes of data"
!!! caution "Terabytes of data"
If you are planning to move terabytes or even more from an outside machine into ZIH systems,
If you are planning to move terabytes or even more from an outside machine into ZIH systems,
...
@@ -423,7 +424,7 @@ See [Slurm documentation](../jobs_and_resources/slurm.md#interactive-jobs) for m
...
@@ -423,7 +424,7 @@ See [Slurm documentation](../jobs_and_resources/slurm.md#interactive-jobs) for m
```
console
```
console
marie@login$
srun
--ntasks
=
1
--cpus-per-task
=
4
--time
=
1:00:00
--mem-per-cpu
=
1700
--pty
bash
-l
#allocate 4 cores for the interactive job
marie@login$
srun
--ntasks
=
1
--cpus-per-task
=
4
--time
=
1:00:00
--mem-per-cpu
=
1700
--pty
bash
-l
#allocate 4 cores for the interactive job
marie@compute$
module load Python
#load necessary packages
marie@compute$
module load Python
#load necessary packages
marie@compute$
cd
/data/horse/ws/marie-
test-workspace
/
#go to your created workspace
marie@compute$
cd
/data/horse/ws/marie-
number_crunch
/
#go to your created workspace
marie@compute$
python test.py
#execute your file
marie@compute$
python test.py
#execute your file
Hello, World!
Hello, World!
```
```
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment