Skip to content
Snippets Groups Projects

Added further Slurm example with job dependency to a Datamover job

Merged Danny Marc Rotscher requested to merge 89-docs-jobs_and_resources-slurm_examples-md into preview
All threads resolved!
1 file
+ 19
1
Compare changes
  • Side-by-side
  • Inline
@@ -355,4 +355,22 @@ file) that will be executed one after each other with different CPU numbers:
@@ -355,4 +355,22 @@ file) that will be executed one after each other with different CPU numbers:
## Array-Job with Afterok-Dependency and Datamover Usage
## Array-Job with Afterok-Dependency and Datamover Usage
This part is under construction.
In this example scenario, imagine you need to move data, before starting the main job.
 
For this you may use a data transfer job and tell Slurm to start the main job immediately after
 
data transfer job successfully finish.
 
 
First you have to start your data transfer job, which for example transfers your input data from one
 
workspace to another.
 
 
```console
 
marie@login: export DATAMOVER_JOB=$(dtcp /scratch/ws/1/marie-source/input.txt /beegfs/ws/1/marie-target/. | awk '{print $4}')
 
```
 
 
Now you can refer to the job id of the Datamover jobs from your work load jobs.
 
 
```console
 
marie@login: srun --dependency afterok:${DATAMOVER_JOB} ls /beegfs/ws/1/marie-target
 
srun: job 23872871 queued and waiting for resources
 
srun: job 23872871 has been allocated resources
 
input.txt
 
```
Loading