From f9f12648a59688d157e4e5f015ce08f0549d00d8 Mon Sep 17 00:00:00 2001 From: Alexander Grund <alexander.grund@tu-dresden.de> Date: Wed, 28 Feb 2024 16:11:12 +0100 Subject: [PATCH] Replace references to "ml" partition and "modenv" --- .../docs/application/request_for_resources.md | 9 +- .../docs/archive/beegfs_on_demand.md | 6 +- .../docs/archive/install_jupyter.md | 4 +- .../docs/archive/scs5_software.md | 39 +----- .../docs/jobs_and_resources/slurm.md | 2 +- .../docs/jobs_and_resources/slurm_examples.md | 10 +- .../jobs_and_resources/slurm_generator.md | 15 +-- .../docs/quickstart/getting_started.md | 7 +- .../docs/software/big_data_frameworks.md | 5 +- doc.zih.tu-dresden.de/docs/software/cicd.md | 10 +- .../docs/software/containers.md | 11 +- .../software/custom_easy_build_environment.md | 15 +-- .../software/data_analytics_with_python.md | 4 - .../docs/software/data_analytics_with_r.md | 8 -- .../docs/software/distributed_training.md | 7 +- .../docs/software/gpu_programming.md | 10 +- .../software/hyperparameter_optimization.md | 8 +- .../docs/software/machine_learning.md | 12 +- .../docs/software/modules.md | 123 ++++-------------- .../docs/software/nanoscale_simulations.md | 7 - .../docs/software/power_ai.md | 4 +- .../docs/software/pytorch.md | 6 +- .../docs/software/singularity_power9.md | 2 +- doc.zih.tu-dresden.de/docs/software/spec.md | 22 ++-- .../docs/software/tensorflow.md | 11 -- .../docs/software/virtual_machines.md | 2 +- .../docs/software/visualization.md | 4 - 27 files changed, 91 insertions(+), 272 deletions(-) diff --git a/doc.zih.tu-dresden.de/docs/application/request_for_resources.md b/doc.zih.tu-dresden.de/docs/application/request_for_resources.md index 09c516114..51da94672 100644 --- a/doc.zih.tu-dresden.de/docs/application/request_for_resources.md +++ b/doc.zih.tu-dresden.de/docs/application/request_for_resources.md @@ -14,9 +14,6 @@ Think in advance about the parallelization strategy for your project and how to ## Available Software Pre-installed software on our HPC systems is managed via [modules](../software/modules.md). -You can see the -[list of software that's already installed and accessible via modules](https://gauss-allianz.de/de/application?organizations%5B0%5D=1200). -However, there are many -different variants of these modules available. We have divided these into two different software -environments: `scs5` (for regular partitions) and `ml` (for the Machine Learning partition). Within -each environment there are further dependencies and variants. +There are many variants of these modules available. +We have divided these into two different software environments such as `release/23.04`. +Within each environment there are further dependencies and variants. diff --git a/doc.zih.tu-dresden.de/docs/archive/beegfs_on_demand.md b/doc.zih.tu-dresden.de/docs/archive/beegfs_on_demand.md index d44116c0b..ebbbb9162 100644 --- a/doc.zih.tu-dresden.de/docs/archive/beegfs_on_demand.md +++ b/doc.zih.tu-dresden.de/docs/archive/beegfs_on_demand.md @@ -66,16 +66,16 @@ Check the status of the job with `squeue -u \<username>`. ## Mount BeeGFS Filesystem -You can mount BeeGFS filesystem on the partition ml (PowerPC architecture) or on the +You can mount BeeGFS filesystem on the partition power9 (PowerPC architecture) or on the partition haswell (x86_64 architecture). -### Mount BeeGFS Filesystem on the Partition `ml` +### Mount BeeGFS Filesystem on the Partition `power9` Job submission can be done with the command (use job id (n.1) from batch job used for creating BeeGFS system): ```console -srun -p ml --beegfs-mount=yes --beegfs-jobid=11047414 --pty bash #Job submission on ml nodes +srun -p power9 --beegfs-mount=yes --beegfs-jobid=11047414 --pty bash #Job submission on power9 nodes ```console Example output: diff --git a/doc.zih.tu-dresden.de/docs/archive/install_jupyter.md b/doc.zih.tu-dresden.de/docs/archive/install_jupyter.md index b5687bc08..859aa52c4 100644 --- a/doc.zih.tu-dresden.de/docs/archive/install_jupyter.md +++ b/doc.zih.tu-dresden.de/docs/archive/install_jupyter.md @@ -50,11 +50,11 @@ one is to download Anaconda in your home directory. 1. Load Anaconda module (recommended): ```console -marie@compute$ module load modenv/scs5 +marie@compute$ module load release/23.04 marie@compute$ module load Anaconda3 ``` -1. Download latest Anaconda release (see example below) and change the rights to make it an +1. Download the latest Anaconda release (see example below) and change the rights to make it an executable script and run the installation script: ```console diff --git a/doc.zih.tu-dresden.de/docs/archive/scs5_software.md b/doc.zih.tu-dresden.de/docs/archive/scs5_software.md index 79c7ba16d..2f2b99d82 100644 --- a/doc.zih.tu-dresden.de/docs/archive/scs5_software.md +++ b/doc.zih.tu-dresden.de/docs/archive/scs5_software.md @@ -42,44 +42,13 @@ module available ml av ``` -There is a special module that is always loaded (sticky) called -**modenv**. It determines the module environment you can see. +There is a special module that is always loaded (sticky) called **release**. +It determines the modules you can use. | Module Environment | Description | Status | |--------------------|---------------------------------------------|---------| -| `modenv/scs5` | SCS5 software | default | -| `modenv/ml` | Software for data analytics (partition ml) | | -| `modenv/classic` | Manually built pre-SCS5 (AE4.0) software | hidden | - -The old modules (pre-SCS5) are still available after loading the -corresponding **modenv** version (**classic**), however, due to changes -in the libraries of the operating system, it is not guaranteed that they -still work under SCS5. That's why those modenv versions are hidden. - -Example: - -```Bash -marie@compute$ ml modenv/classic ansys/19.0 - -The following have been reloaded with a version change: - 1) modenv/scs5 => modenv/classic - -Module ansys/19.0 loaded. -``` - -**modenv/scs5** will be loaded by default and contains all the software -that was built especially for SCS5. - -### Which modules should I use? - -If possible, please use the modules from **modenv/scs5**. In case there is a certain software -missing, you can write an [email to hpcsupport](mailto:hpc-support@tu-dresden.de) and we will try -to install the latest version of this particular software for you. - -However, if you still need *older* versions of some software, you have to resort to using the -modules in the old module environment (**modenv/classic** most probably). We won't keep those around -forever though, so in the long-term, it is advisable to migrate your workflow to up-to-date versions -of the software used. +| `release/23.04` | Release of April 2023 | default | +| `release/23.10` | Release of October 2023 | | ### Compilers, MPI-Libraries and Toolchains diff --git a/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm.md b/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm.md index eea01a7ce..f5c722791 100644 --- a/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm.md +++ b/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm.md @@ -98,7 +98,7 @@ can find it via `squeue --me`. The job ID allows you to !!! warning "srun vs. mpirun" On ZIH systems, `srun` is used to run your parallel application. The use of `mpirun` is provenly - broken on partitions `ml` and `alpha` for jobs requiring more than one node. Especially when + broken on partitions `power9` and `alpha` for jobs requiring more than one node. Especially when using code from github projects, double-check its configuration by looking for a line like 'submit command mpirun -n $ranks ./app' and replace it with 'srun ./app'. diff --git a/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_examples.md b/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_examples.md index 252b2fab5..e1feca90f 100644 --- a/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_examples.md +++ b/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_examples.md @@ -111,7 +111,7 @@ But, do you need to request tasks or CPUs from Slurm in order to provide resourc Slurm will allocate one or many GPUs for your job if requested. Please note that GPUs are only available in the GPU clusters, like -[Alpha Centauri](hardware_overview.md#alpha-centauri) or +[Alpha Centauri](hardware_overview.md#alpha-centauri) and [Power9](hardware_overview.md#power9). The option for `sbatch/srun` in this case is `--gres=gpu:[NUM_PER_NODE]`, where `NUM_PER_NODE` is the number of GPUs **per node** that will be used for the job. @@ -175,7 +175,7 @@ srun: error: Unable to allocate resources: Job violates accounting/QOS policy (j ### Running Multiple GPU Applications Simultaneously in a Batch Job Our starting point is a (serial) program that needs a single GPU and four CPU cores to perform its -task (e.g. TensorFlow). The following batch script shows how to run such a job on the partition `ml`. +task (e.g. TensorFlow). The following batch script shows how to run such a job on the partition `power9`. !!! example @@ -187,7 +187,7 @@ task (e.g. TensorFlow). The following batch script shows how to run such a job o #SBATCH --gpus-per-task=1 #SBATCH --time=01:00:00 #SBATCH --mem-per-cpu=1443 - #SBATCH --partition=ml + #SBATCH --partition=power9 srun some-gpu-application ``` @@ -196,7 +196,7 @@ When `srun` is used within a submission script, it inherits parameters from `sba `--ntasks=1`, `--cpus-per-task=4`, etc. So we actually implicitly run the following ```bash -srun --ntasks=1 --cpus-per-task=4 [...] --partition=ml some-gpu-application +srun --ntasks=1 --cpus-per-task=4 [...] --partition=power9 some-gpu-application ``` Now, our goal is to run four instances of this program concurrently in a single batch script. Of @@ -230,7 +230,7 @@ three things: #SBATCH --gpus-per-task=1 #SBATCH --time=01:00:00 #SBATCH --mem-per-cpu=1443 - #SBATCH --partition=ml + #SBATCH --partition=power9 srun --exclusive --gres=gpu:1 --ntasks=1 --cpus-per-task=4 --gpus-per-task=1 --mem-per-cpu=1443 some-gpu-application & srun --exclusive --gres=gpu:1 --ntasks=1 --cpus-per-task=4 --gpus-per-task=1 --mem-per-cpu=1443 some-gpu-application & diff --git a/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_generator.md b/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_generator.md index e608b823a..963139b6d 100644 --- a/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_generator.md +++ b/doc.zih.tu-dresden.de/docs/jobs_and_resources/slurm_generator.md @@ -420,7 +420,7 @@ along with sgen. If not, see <http://www.gnu.org/licenses/>. 'MemoryPerNode': 95000, 'MemoryPerCore': 7916 }, - 'ml' : ml = { + 'power9' : power9 = { 'MaxTime': 'INFINITE', 'DefaultTime': 60, 'Sockets': 2, @@ -433,19 +433,6 @@ along with sgen. If not, see <http://www.gnu.org/licenses/>. 'MemoryPerNode': 254000, 'MemoryPerCore': 1443 }, - 'ml-interactive': { - 'MaxTime': 480, - 'DefaultTime': 10, - 'Sockets': 2, - 'CoresPerSocket': 22, - 'ThreadsPerCore': 4, - 'nodes': 2, - 'GPU': 6, - 'HTCores': 176, - 'Cores': 44, - 'MemoryPerNode': 254000, - 'MemoryPerCore': 1443 - }, 'romeo' : romeo = { 'MaxTime': 'INFINITE', 'DefaultTime': 480, diff --git a/doc.zih.tu-dresden.de/docs/quickstart/getting_started.md b/doc.zih.tu-dresden.de/docs/quickstart/getting_started.md index b7d853ab1..39e3c68f9 100644 --- a/doc.zih.tu-dresden.de/docs/quickstart/getting_started.md +++ b/doc.zih.tu-dresden.de/docs/quickstart/getting_started.md @@ -357,15 +357,12 @@ marie@login$ module spider Python/3.9.5 ``` In some cases it is required to load additional modules before loading the desired software. -In the example above, these are `modenv/hiera` and `GCCcore/10.3.0`. +In the example above, it is `GCCcore/11.3.0`. - Load prerequisites and the desired software: ```console -marie@login$ module load modenv/hiera GCCcore/10.3.0 # load prerequisites - -The following have been reloaded with a version change: - 1) modenv/scs5 => modenv/hiera +marie@login$ module load GCCcore/11.3.0 # load prerequisites Module GCCcore/10.3.0 loaded. diff --git a/doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md b/doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md index cfc5c6d82..171b41629 100644 --- a/doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md +++ b/doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md @@ -2,8 +2,9 @@ [Apache Spark](https://spark.apache.org/), [Apache Flink](https://flink.apache.org/) and [Apache Hadoop](https://hadoop.apache.org/) are frameworks for processing and integrating -Big Data. These frameworks are also offered as software [modules](modules.md) in both `ml` and -`scs5` software environments. You can check module versions and availability with the command +Big Data. +These frameworks are also offered as software [modules](modules.md). +You can check module versions and availability with the command === "Spark" ```console diff --git a/doc.zih.tu-dresden.de/docs/software/cicd.md b/doc.zih.tu-dresden.de/docs/software/cicd.md index 7294622a2..f5eb3b42c 100644 --- a/doc.zih.tu-dresden.de/docs/software/cicd.md +++ b/doc.zih.tu-dresden.de/docs/software/cicd.md @@ -74,10 +74,10 @@ Use the variable `SCHEDULER_PARAMETERS` and define the same parameters you would !!! example The following YAML file defines a configuration section `.test-job`, and two jobs, - `test-job-haswell` and `test-job-ml`, extending from that. The two job share the + `test-job-haswell` and `test-job-power9`, extending from that. The two job share the `before_script`, `script`, and `after_script` configuration, but differ in the - `SCHEDULER_PARAMETERS`. The `test-job-haswell` and `test-job-ml` are scheduled on the partition - `haswell` and partition `ml`, respectively. + `SCHEDULER_PARAMETERS`. The `test-job-haswell` and `test-job-power9` are scheduled on the partition + `haswell` and partition `power9`, respectively. ``` yaml .test-job: @@ -100,10 +100,10 @@ Use the variable `SCHEDULER_PARAMETERS` and define the same parameters you would SCHEDULER_PARAMETERS: -p haswell - test-job-ml: + test-job-power9: extends: .test-job variables: - SCHEDULER_PARAMETERS: -p ml + SCHEDULER_PARAMETERS: -p power9 ``` ## Current limitations diff --git a/doc.zih.tu-dresden.de/docs/software/containers.md b/doc.zih.tu-dresden.de/docs/software/containers.md index 402dedb5e..5046de4b8 100644 --- a/doc.zih.tu-dresden.de/docs/software/containers.md +++ b/doc.zih.tu-dresden.de/docs/software/containers.md @@ -9,11 +9,12 @@ opposed to Docker (the most famous container solution), Singularity is much more used in an HPC environment and more efficient in many cases. Docker images can easily be used in Singularity. Information about the use of Singularity on ZIH systems can be found on this page. -In some cases using Singularity requires a Linux machine with root privileges (e.g. using the -partition `ml`), the same architecture and a compatible kernel. For many reasons, users on ZIH -systems cannot be granted root permissions. A solution is a Virtual Machine (VM) on the partition -`ml` which allows users to gain root permissions in an isolated environment. There are two main -options on how to work with Virtual Machines on ZIH systems: +In some cases using Singularity requires a Linux machine with root privileges +(e.g. using the partition `power9`), the same architecture and a compatible kernel. +For many reasons, users on ZIH systems cannot be granted root permissions. +A solution is a Virtual Machine (VM) on the partition `power9` which allows users to gain +root permissions in an isolated environment. +There are two main options on how to work with Virtual Machines on ZIH systems: 1. [VM tools](singularity_power9.md): Automated algorithms for using virtual machines; 1. [Manual method](virtual_machines.md): It requires more operations but gives you more flexibility diff --git a/doc.zih.tu-dresden.de/docs/software/custom_easy_build_environment.md b/doc.zih.tu-dresden.de/docs/software/custom_easy_build_environment.md index 913665cc3..c009e532c 100644 --- a/doc.zih.tu-dresden.de/docs/software/custom_easy_build_environment.md +++ b/doc.zih.tu-dresden.de/docs/software/custom_easy_build_environment.md @@ -76,24 +76,15 @@ marie@login$ srun --nodes=1 --cpus-per-task=4 --time=08:00:00 --pty /bin/bash -l **Step 3:** Specify the workspace. The rest of the guide is based on it. Please create an environment variable called `WORKSPACE` with the path to your workspace: -_The module environments /hiera, /scs5, /classic and /ml originated from the taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - ```console marie@compute$ export WORKSPACE=/data/horse/ws/marie-EasyBuild #see output of ws_list above ``` -**Step 4:** Load the correct module environment `modenv` according to your current or target -architecture: +**Step 4:** Load the correct module environment `release` according to your needs: -=== "x86 (default, e. g. partition `haswell`)" - ```console - marie@compute$ module load modenv/scs5 - ``` -=== "Power9 (partition `ml`)" +=== "23.04" ```console - marie@ml$ module load modenv/ml + marie@compute$ module load release/23.04 ``` **Step 5:** Load module `EasyBuild` diff --git a/doc.zih.tu-dresden.de/docs/software/data_analytics_with_python.md b/doc.zih.tu-dresden.de/docs/software/data_analytics_with_python.md index b319fc3ee..90f82449f 100644 --- a/doc.zih.tu-dresden.de/docs/software/data_analytics_with_python.md +++ b/doc.zih.tu-dresden.de/docs/software/data_analytics_with_python.md @@ -434,10 +434,6 @@ comm = MPI.COMM_WORLD print("%d of %d" % (comm.Get_rank(), comm.Get_size())) ``` -_The module environments /hiera, /scs5, /classic and /ml originated from the taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - For the multi-node case, use a script similar to this: ```bash diff --git a/doc.zih.tu-dresden.de/docs/software/data_analytics_with_r.md b/doc.zih.tu-dresden.de/docs/software/data_analytics_with_r.md index 266f145ca..bd994d6fa 100644 --- a/doc.zih.tu-dresden.de/docs/software/data_analytics_with_r.md +++ b/doc.zih.tu-dresden.de/docs/software/data_analytics_with_r.md @@ -14,10 +14,6 @@ see our [hardware documentation](../jobs_and_resources/hardware_overview.md). In the following example, the `srun` command is used to start an interactive job, so that the output is visible to the user. Please check the [Slurm page](../jobs_and_resources/slurm.md) for details. -_The module environments /hiera, /scs5, /classic and /ml originated from the taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - ```console marie@login.barnard$ srun --ntasks=1 --nodes=1 --cpus-per-task=4 --mem-per-cpu=2541 --time=01:00:00 --pty bash [marie@barnard ]$ module load release/23.10 GCC/11.3.0 OpenMPI/4.1.4 R/4.2.1 @@ -265,10 +261,6 @@ Submitting a multicore R job to Slurm is very similar to submitting an [OpenMP Job](../jobs_and_resources/binding_and_distribution_of_tasks.md), since both are running multicore jobs on a **single** node. Below is an example: -_The module environments /hiera, /scs5, /classic and /ml originated from the taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - ```Bash #!/bin/bash #SBATCH --nodes=1 diff --git a/doc.zih.tu-dresden.de/docs/software/distributed_training.md b/doc.zih.tu-dresden.de/docs/software/distributed_training.md index be8a71e02..445c326f3 100644 --- a/doc.zih.tu-dresden.de/docs/software/distributed_training.md +++ b/doc.zih.tu-dresden.de/docs/software/distributed_training.md @@ -123,8 +123,7 @@ Each worker runs the training loop independently. IP_1=$(dig +short ${NODE_1}.alpha.hpc.tu-dresden.de) IP_2=$(dig +short ${NODE_2}.alpha.hpc.tu-dresden.de) - module load modenv/hiera - module load modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 TensorFlow/2.4.1 + module load release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 TensorFlow/2.4.1 # On the first node TF_CONFIG='{"cluster": {"worker": ["'"${NODE_1}"':33562", "'"${NODE_2}"':33561"]}, "task": {"index": 0, "type": "worker"}}' srun --nodelist=${NODE_1} --nodes=1 --ntasks=1 --gres=gpu:1 python main_ddl.py & @@ -260,7 +259,7 @@ Or if you want to use Horovod on the cluster `alpha`, you can load it with the d ```console marie@alpha$ module spider Horovod #Check available modules -marie@alpha$ module load modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 Horovod/0.21.1-TensorFlow-2.4.1 +marie@alpha$ module load release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 Horovod/0.21.1-TensorFlow-2.4.1 ``` #### Horovod Installation @@ -335,7 +334,7 @@ Hello from: 0 #SBATCH --time=01:00:00 #SBATCH --output=run_horovod.out - module load modenv/ml + module load release/23.04 module load Horovod/0.19.5-fosscuda-2019b-TensorFlow-2.2.0-Python-3.7.4 srun python your_program.py diff --git a/doc.zih.tu-dresden.de/docs/software/gpu_programming.md b/doc.zih.tu-dresden.de/docs/software/gpu_programming.md index 686fe37d7..5de6a12d9 100644 --- a/doc.zih.tu-dresden.de/docs/software/gpu_programming.md +++ b/doc.zih.tu-dresden.de/docs/software/gpu_programming.md @@ -6,11 +6,6 @@ The full hardware specifications of the GPU-compute nodes may be found in the [HPC Resources](../jobs_and_resources/hardware_overview.md#hpc-resources) page. Each node uses a different modules(modules.md#module-environments): -* [NVIDIA A100 nodes](../jobs_and_resources/hardware_overview.md#amd-rome-cpus-nvidia-a100) -(cluster `alpha`): use the `hiera` module environment (`module switch modenv/hiera`) -* [NVIDIA Tesla V100 nodes](../jobs_and_resources/hardware_overview.md#ibm-power9-nodes-for-machine-learning) -(cluster `power9`): use the `module spider <module name>` - ## Using GPUs with Slurm For general information on how to use Slurm, read the respective [page in this compendium](../jobs_and_resources/slurm.md). @@ -104,7 +99,7 @@ Furthermore, some compilers, such as GCC, have basic support for target offloadi enable these features by default and/or achieve poor performance. On the ZIH system, compilers with OpenMP target offloading support are provided on the clusters -`ml` and `alpha`. Two compilers with good performance can be used: the NVIDIA HPC compiler and the +`power9` and `alpha`. Two compilers with good performance can be used: the NVIDIA HPC compiler and the IBM XL compiler. #### Using OpenMP target offloading with NVIDIA HPC compilers @@ -121,8 +116,7 @@ available for OpenMP, including the `-gpu=ccXY` flag as mentioned above. #### Using OpenMP target offloading with the IBM XL compilers The IBM XL compilers (`xlc` for C, `xlc++` for C++ and `xlf` for Fortran (with sub-version for -different versions of Fortran)) are only available on the cluster `ml` with NVIDIA Tesla V100 GPUs. -They are available by default when switching to `modenv/ml`. +different versions of Fortran)) are only available on the cluster `power9` with NVIDIA Tesla V100 GPUs. * The `-qsmp -qoffload` combination of flags enables OpenMP target offloading support * Optimizations specific to the V100 GPUs can be enabled by using the diff --git a/doc.zih.tu-dresden.de/docs/software/hyperparameter_optimization.md b/doc.zih.tu-dresden.de/docs/software/hyperparameter_optimization.md index 5b79aaa47..1af996cb5 100644 --- a/doc.zih.tu-dresden.de/docs/software/hyperparameter_optimization.md +++ b/doc.zih.tu-dresden.de/docs/software/hyperparameter_optimization.md @@ -202,7 +202,7 @@ There are the following script preparation steps for OmniOpt: [workspace](../data_lifecycle/workspaces.md). ```console - marie@login$ module load modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 PyTorch/1.9.0 + marie@login$ module load release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 PyTorch/1.9.0 marie@login$ mkdir </path/to/workspace/python-environments> #create folder marie@login$ virtualenv --system-site-packages </path/to/workspace/python-environments/torchvision_env> marie@login$ source </path/to/workspace/python-environments/torchvision_env>/bin/activate #activate virtual environment @@ -212,11 +212,9 @@ There are the following script preparation steps for OmniOpt: ```console # Job submission on alpha nodes with 1 GPU on 1 node with 800 MB per CPU marie@login$ srun --gres=gpu:1 -n 1 -c 7 --pty --mem-per-cpu=800 bash - marie@alpha$ module load modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 PyTorch/1.9.0 + marie@alpha$ module load release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 PyTorch/1.9.0 # Activate virtual environment marie@alpha$ source </path/to/workspace/python-environments/torchvision_env>/bin/activate - The following have been reloaded with a version change: - 1) modenv/scs5 => modenv/hiera Module GCC/10.2.0, CUDA/11.1.1, OpenMPI/4.0.5, PyTorch/1.9.0 and 54 dependencies loaded. marie@alpha$ python </path/to/your/script/mnistFashion.py> --out-layer1=200 --batchsize=10 --epochs=3 @@ -252,7 +250,7 @@ environment. The recommended way is to wrap the necessary calls in a shell scrip # srun bash -l run.sh # Load modules your program needs, always specify versions! - module load modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 PyTorch/1.7.1 + module load release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 PyTorch/1.7.1 source </path/to/workspace/python-environments/torchvision_env>/bin/activate #activate virtual environment # Load your script. $@ is all the parameters that are given to this shell file. diff --git a/doc.zih.tu-dresden.de/docs/software/machine_learning.md b/doc.zih.tu-dresden.de/docs/software/machine_learning.md index 4dea3aa30..37b326705 100644 --- a/doc.zih.tu-dresden.de/docs/software/machine_learning.md +++ b/doc.zih.tu-dresden.de/docs/software/machine_learning.md @@ -18,17 +18,7 @@ cluster `power` has 6x Tesla V-100 GPUs. You can find a detailed specification o !!! note The cluster `power` is based on the Power9 architecture, which means that the software built - for x86_64 will not work on this cluster. Also, users need to use the modules which are - specially build for this architecture (from `modenv/ml`). - -### Modules - -On the cluster `power` load the module environment: - -```console -marie@power$ module load modenv/ml -The following have been reloaded with a version change: 1) modenv/scs5 => modenv/ml -``` + for x86_64 will not work on this cluster. ### Power AI diff --git a/doc.zih.tu-dresden.de/docs/software/modules.md b/doc.zih.tu-dresden.de/docs/software/modules.md index aa3c21c08..fc16562f1 100644 --- a/doc.zih.tu-dresden.de/docs/software/modules.md +++ b/doc.zih.tu-dresden.de/docs/software/modules.md @@ -114,7 +114,7 @@ certain module, you can use `module avail softwarename` and it will display the Die folgenden Module wurden nicht entladen: (Benutzen Sie "module --force purge" um alle Module zu entladen): - 1) modenv/scs5 + 1) release/23.04 Module Python/3.8.6-GCCcore-10.2.0 and 11 dependencies unloaded. ``` @@ -168,7 +168,7 @@ There is a front end for the module command, which helps you to type less. It is marie@compute$ ml Derzeit geladene Module: - 1) modenv/scs5 (S) 5) bzip2/1.0.8-GCCcore-10.2.0 9) SQLite/3.33.0-GCCcore-10.2.0 13) Python/3.8.6-GCCcore-10.2.0 + 1) release/23.04 (S) 5) bzip2/1.0.8-GCCcore-10.2.0 9) SQLite/3.33.0-GCCcore-10.2.0 13) Python/3.8.6-GCCcore-10.2.0 2) GCCcore/10.2.0 6) ncurses/6.2-GCCcore-10.2.0 10) XZ/5.2.5-GCCcore-10.2.0 3) zlib/1.2.11-GCCcore-10.2.0 7) libreadline/8.0-GCCcore-10.2.0 11) GMP/6.2.0-GCCcore-10.2.0 4) binutils/2.35-GCCcore-10.2.0 8) Tcl/8.6.10-GCCcore-10.2.0 12) libffi/3.3-GCCcore-10.2.0 @@ -184,41 +184,20 @@ There is a front end for the module command, which helps you to type less. It is ## Module Environments On ZIH systems, there exist different **module environments**, each containing a set of software -modules. They are activated via the meta module `modenv` which has different versions, one of which -is loaded by default. You can switch between them by simply loading the desired modenv-version, e.g. +modules. +They are activated via the meta module `release` which has different versions, +one of which is loaded by default. +You can switch between them by simply loading the desired version, e.g. ```console -marie@compute$ module load modenv/ml +marie@compute$ module load release/23.10 ``` -### modenv/scs5 (default) - -* SCS5 software -* usually optimized for Intel processors (Cluster `Barnard`, `Julia`) - -### modenv/ml - -* data analytics software (for use on the Cluster `ml`) -* necessary to run most software on the cluster `ml` -(The instruction set [Power ISA](https://en.wikipedia.org/wiki/Power_ISA#Power_ISA_v.3.0) -is different from the usual x86 instruction set. -Thus the 'machine code' of other modenvs breaks). - -### modenv/hiera - -* uses a hierarchical module load scheme -* optimized software for AMD processors (Cluster `Romeo` and `Alpha`) - -### modenv/classic - -* deprecated, old software. Is not being curated. -* may break due to library inconsistencies with the operating system. -* please don't use software from that modenv - ### Searching for Software -The command `module spider <modname>` allows searching for a specific software across all modenv -environments. It will also display information on how to load a particular module when giving a +The command `module spider <modname>` allows searching for a specific software across all module +environments. +It will also display information on how to load a particular module when giving a precise module (with version) as the parameter. ??? example "Spider command" @@ -270,9 +249,7 @@ In some cases a desired software is available as an extension of a module. -------------------------------------------------------------------------------------------------------------------------------- This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy. - TensorFlow/2.4.1 (modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5) - TensorFlow/2.4.1-fosscuda-2019b-Python-3.7.4 (modenv/ml) - TensorFlow/2.4.1-foss-2020b (modenv/scs5) + TensorFlow/2.4.1 (release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5) Names marked by a trailing (E) are extensions provided by another module. ``` @@ -280,10 +257,7 @@ In some cases a desired software is available as an extension of a module. Finaly, you can load the dependencies and `tensorboard/2.4.1` and check the version. ```console - marie@login$ module load modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 - - The following have been reloaded with a version change: - 1) modenv/scs5 => modenv/hiera + marie@login$ module load release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 Module GCC/10.2.0, CUDA/11.1.1, OpenMPI/4.0.5 and 15 dependencies loaded. marie@login$ module load TensorFlow/2.4.1 @@ -295,7 +269,7 @@ In some cases a desired software is available as an extension of a module. ## Toolchains A program or library may break in various ways (e.g. not starting, crashing or producing wrong -results) when it is used with a software of a different version than it expects. So each module +results) when it is used with a software of a different version than it expects.So each module specifies the exact other modules it depends on. They get loaded automatically when the dependent module is loaded. @@ -308,14 +282,9 @@ means they now have a wrong dependency (version) which can be a problem (see abo To avoid this there are (versioned) toolchains and for each toolchain there is (usually) at most one version of each software. A "toolchain" is a set of modules used to build the software for other modules. -The most common one is the `foss`-toolchain comprising of `GCC`, `OpenMPI`, `OpenBLAS` & `FFTW`. - -!!! info - - Modules are named like `<Softwarename>/<Version>-<Toolchain>` so `Python/3.6.6-foss-2019a` - uses the `foss-2019a` toolchain. +The most common one is the `foss`-toolchain consisting of `GCC`, `OpenMPI`, `OpenBLAS` & `FFTW`. -This toolchain can be broken down into a sub-toolchain called `gompi` comprising of only +This toolchain can be broken down into a sub-toolchain called `gompi` consisting of only `GCC` & `OpenMPI`, or further to `GCC` (the compiler and linker) and even further to `GCCcore` which is only the runtime libraries required to run programs built with the GCC standard library. @@ -338,7 +307,7 @@ Examples: | `iimpi` | `intel-compilers` `impi` | | `intel-compilers` | `GCCcore` `binutils` | -As you can see `GCC` and `intel-compilers` are on the same level, as are `gompi` and `iimpi` +As you can see `GCC` and `intel-compilers` are on the same level, as are `gompi` and `iimpi`, although they are one level higher than the former. You can load and use modules from a lower toolchain with modules from @@ -352,58 +321,23 @@ However `LLVM/7.0.1-GCCcore-8.2.0` can be used with either `QuantumESPRESSO/6.5-intel-2019a` or `Python/3.6.6-foss-2019a` because `GCCcore-8.2.0` is a sub-toolchain of `intel-2019a` and `foss-2019a`. -For [modenv/hiera](#modenvhiera) it is much easier to avoid loading incompatible -modules as modules from other toolchains cannot be directly loaded -and don't show up in `module av`. +With the hierarchical module scheme we use at ZIH modules from other toolchains cannot be directly +loaded and don't show up in `module av` which avoids loading incompatible modules. So the concept if this hierarchical toolchains is already built into this module environment. -In the other module environments it is up to you to make sure the modules you load are compatible. - -So watch the output when you load another module as a message will be shown when loading a module -causes other modules to be loaded in a different version: - -??? example "Module reload" - - ```console - marie@login$ ml OpenFOAM/8-foss-2020a - Module OpenFOAM/8-foss-2020a and 72 dependencies loaded. - - marie@login$ ml Biopython/1.78-foss-2020b - The following have been reloaded with a version change: - 1) FFTW/3.3.8-gompi-2020a => FFTW/3.3.8-gompi-2020b 15) binutils/2.34-GCCcore-9.3.0 => binutils/2.35-GCCcore-10.2.0 - 2) GCC/9.3.0 => GCC/10.2.0 16) bzip2/1.0.8-GCCcore-9.3.0 => bzip2/1.0.8-GCCcore-10.2.0 - 3) GCCcore/9.3.0 => GCCcore/10.2.0 17) foss/2020a => foss/2020b - [...] - ``` !!! info - The higher toolchains have a year and letter as their version corresponding to their release. + The toolchains usually have a year and letter as their version corresponding to their release. So `2019a` and `2020b` refer to the first half of 2019 and the 2nd half of 2020 respectively. ## Per-Architecture Builds -Since we have a heterogeneous cluster, we do individual builds of some of the software for each +Since we have a heterogeneous cluster, we do individual builds of the software for each architecture present. This ensures that, no matter what partition the software runs on, a build optimized for the host architecture is used automatically. -For that purpose we have created symbolic links on the compute nodes, -at the system path `/sw/installed`. - -However, not every module will be available for each node type or partition. Especially when -introducing new hardware to the cluster, we do not want to rebuild all of the older module versions -and in some cases cannot fall-back to a more generic build either. That's why we provide the script: -`ml_arch_avail` that displays the availability of modules for the different node architectures. - -### Example Invocation of ml_arch_avail - -```console -marie@compute$ ml_arch_avail TensorFlow/2.4.1 -TensorFlow/2.4.1: haswell, rome -TensorFlow/2.4.1: haswell, rome -``` -The command shows all modules that match on `TensorFlow/2.4.1`, and their respective availability. -Note that this will not work for meta-modules that do not have an installation directory -(like some tool chain modules). +However, not every module will be available for each node type or partition. +Use `ml av` or `ml spider` to search for modules available on the sub-cluster you are on. ## Advanced Usage @@ -414,8 +348,7 @@ For writing your own module files please have a look at the ### When I log in, the wrong modules are loaded by default -Reset your currently loaded modules with `module purge` -(or `module purge --force` if you also want to unload your basic `modenv` module). +Reset your currently loaded modules with `module purge`. Then run `module save` to overwrite the list of modules you load by default when logging in. @@ -446,11 +379,11 @@ before the TensorFlow module can be loaded. Sie müssen alle Module in einer der nachfolgenden Zeilen laden bevor Sie das Modul "TensorFlow/2.4.1" laden können. - modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 + release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy. - TensorFlow/2.4.1 (modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5) + TensorFlow/2.4.1 (release/23.04 GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5) This module provides the following extensions: @@ -484,18 +417,18 @@ before the TensorFlow module can be loaded. - marie@compute$ ml +modenv/hiera +GCC/10.2.0 +CUDA/11.1.1 +OpenMPI/4.0.5 +TensorFlow/2.4.1 + marie@compute$ ml +GCC/10.2.0 +CUDA/11.1.1 +OpenMPI/4.0.5 +TensorFlow/2.4.1 Die folgenden Module wurden in einer anderen Version erneut geladen: 1) GCC/7.3.0-2.30 => GCC/10.2.0 3) binutils/2.30-GCCcore-7.3.0 => binutils/2.35 - 2) GCCcore/7.3.0 => GCCcore/10.2.0 4) modenv/scs5 => modenv/hiera + 2) GCCcore/7.3.0 => GCCcore/10.2.0 Module GCCcore/7.3.0, binutils/2.30-GCCcore-7.3.0, GCC/7.3.0-2.30, GCC/7.3.0-2.30 and 3 dependencies unloaded. Module GCCcore/7.3.0, GCC/7.3.0-2.30, GCC/10.2.0, CUDA/11.1.1, OpenMPI/4.0.5, TensorFlow/2.4.1 and 50 dependencies loaded. marie@compute$ module list Derzeit geladene Module: - 1) modenv/hiera (S) 28) Tcl/8.6.10 + 1) release/23.04 (S) 28) Tcl/8.6.10 2) GCCcore/10.2.0 29) SQLite/3.33.0 3) zlib/1.2.11 30) GMP/6.2.0 4) binutils/2.35 31) libffi/3.3 diff --git a/doc.zih.tu-dresden.de/docs/software/nanoscale_simulations.md b/doc.zih.tu-dresden.de/docs/software/nanoscale_simulations.md index aac64bfc2..bf60df228 100644 --- a/doc.zih.tu-dresden.de/docs/software/nanoscale_simulations.md +++ b/doc.zih.tu-dresden.de/docs/software/nanoscale_simulations.md @@ -64,13 +64,7 @@ please look at the [GAMESS home page](https://www.msg.chem.iastate.edu/gamess/in GAMESS is available as [modules](modules.md) within the classic environment. Available packages can be listed and loaded with the following commands: -_The module environments /hiera, /scs5, /classic and /ml originated from the taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - ```console -marie@login$ module load modenv/classic -[...] marie@login$:~> module avail gamess ----------------------- /sw/modules/taurus/applications ------------------------ gamess/2013 @@ -91,7 +85,6 @@ For runs with [Slurm](../jobs_and_resources/slurm.md), please use a script like ## you have to make sure that an even number of tasks runs on each node !! #SBATCH --mem-per-cpu=1900 -module load modenv/classic module load gamess rungms.slurm cTT_M_025.inp /data/horse/ws/marie-gamess # the third parameter is the location of your horse directory diff --git a/doc.zih.tu-dresden.de/docs/software/power_ai.md b/doc.zih.tu-dresden.de/docs/software/power_ai.md index 1488d85f9..e4578ba69 100644 --- a/doc.zih.tu-dresden.de/docs/software/power_ai.md +++ b/doc.zih.tu-dresden.de/docs/software/power_ai.md @@ -5,7 +5,7 @@ the PowerAI Framework for Machine Learning. In the following the links are valid for PowerAI version 1.5.4. !!! warning - The information provided here is available from IBM and can be used on partition `ml` only! + The information provided here is available from IBM and can be used on partition `power9` only! ## General Overview @@ -47,7 +47,7 @@ are valid for PowerAI version 1.5.4. (Open Neural Network Exchange) provides support for moving models between those frameworks. - [Distributed Deep Learning](https://www.ibm.com/support/knowledgecenter/SS5SF7_1.5.4/navigation/pai_getstarted_ddl.html?view=kc) - Distributed Deep Learning (DDL). Works on up to 4 nodes on partition `ml`. + Distributed Deep Learning (DDL). Works on up to 4 nodes on partition `power9`. ## PowerAI Container diff --git a/doc.zih.tu-dresden.de/docs/software/pytorch.md b/doc.zih.tu-dresden.de/docs/software/pytorch.md index 574df2cab..4f6d0d78b 100644 --- a/doc.zih.tu-dresden.de/docs/software/pytorch.md +++ b/doc.zih.tu-dresden.de/docs/software/pytorch.md @@ -20,10 +20,6 @@ and the PyTorch library. You can find detailed hardware specification in our [hardware documentation](../jobs_and_resources/hardware_overview.md). -_The module environments /hiera, /scs5, /classic and /ml originated from the taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - ## PyTorch Console On the cluster `alpha`, load the module environment: @@ -69,7 +65,7 @@ marie@login.power$ module spider pytorch we know that we can load PyTorch (including torchvision) with ```console -marie@power$ module load modenv/ml torchvision/0.7.0-fossCUDA-2019b-Python-3.7.4-PyTorch-1.6.0 +marie@power$ module load torchvision/0.7.0-fossCUDA-2019b-Python-3.7.4-PyTorch-1.6.0 Module torchvision/0.7.0-fossCUDA-2019b-Python-3.7.4-PyTorch-1.6.0 and 55 dependencies loaded. ``` diff --git a/doc.zih.tu-dresden.de/docs/software/singularity_power9.md b/doc.zih.tu-dresden.de/docs/software/singularity_power9.md index 5abb5c501..609e52801 100644 --- a/doc.zih.tu-dresden.de/docs/software/singularity_power9.md +++ b/doc.zih.tu-dresden.de/docs/software/singularity_power9.md @@ -67,7 +67,7 @@ have reasonable defaults. The most important ones are: * Various Singularity options are passed through. E.g. `--notest, --force, --update`. See, e.g., `singularity --help` for details. -For **advanced users**, it is also possible to manually request a job with a VM (`srun -p ml +For **advanced users**, it is also possible to manually request a job with a VM (`srun -p power9 --cloud=kvm ...`) and then use this script to build a Singularity container from within the job. In this case, the `--arch` and other Slurm related parameters are not required. The advantage of using this script is that it automates the waiting for the VM and mounting of host directories into it diff --git a/doc.zih.tu-dresden.de/docs/software/spec.md b/doc.zih.tu-dresden.de/docs/software/spec.md index f567f8de6..34af0bac2 100644 --- a/doc.zih.tu-dresden.de/docs/software/spec.md +++ b/doc.zih.tu-dresden.de/docs/software/spec.md @@ -32,9 +32,9 @@ Once the target partition is determined, follow SPEC's [Installation Guide](https://www.spec.org/hpg/hpc2021/Docs/install-guide-linux.html). It is straight-forward and easy to use. -???+ tip "Building for partition `ml`" +???+ tip "Building for partition `power9`" - The partition `ml` is a Power9 architecture. Thus, you need to provide the `-e ppc64le` switch + The partition `power9` is a Power9 architecture. Thus, you need to provide the `-e ppc64le` switch when installing. ???+ tip "Building with NVHPC for partition `alpha`" @@ -52,8 +52,8 @@ listed there. The behavior in terms of how to build, run and report the benchmark in a particular environment is controlled by a configuration file. There are a few examples included in the source code. Here you can apply compiler tuning and porting, specify the runtime environment and describe the -system under test. SPEChpc 2021 has been deployed on the partitions `haswell`, `ml` and -`alpha`. Configurations are available, respectively: +system under test. +Configurations are available, respectively: - [gnu-taurus.cfg](misc/spec_gnu-taurus.cfg) - [nvhpc-ppc.cfg](misc/spec_nvhpc-ppc.cfg) @@ -89,7 +89,7 @@ configuration and controls it's runtime behavior. For all options, see SPEC's do First, execute `source shrc` in your SPEC installation directory. Then use a job script to submit a job with the benchmark or parts of it. -In the following there are job scripts shown for partitions `haswell`, `ml` and `alpha`, +In the following there are job scripts shown for partitions `haswell`, `power9` and `alpha`, respectively. You can use them as a template in order to reproduce results or to transfer the execution to a different partition. @@ -128,7 +128,7 @@ execution to a different partition. ```bash linenums="1" #!/bin/bash #SBATCH --account=<p_number_crunch> - #SBATCH --partition=ml + #SBATCH --partition=power9 #SBATCH --exclusive #SBATCH --nodes=1 #SBATCH --ntasks=6 @@ -141,7 +141,7 @@ execution to a different partition. #SBATCH --hint=nomultithread module --force purge - module load modenv/ml NVHPC OpenMPI/4.0.5-NVHPC-21.2-CUDA-11.2.1 + module load NVHPC OpenMPI/4.0.5-NVHPC-21.2-CUDA-11.2.1 ws=</scratch/ws/spec/installation> cd ${ws} @@ -178,7 +178,7 @@ execution to a different partition. #SBATCH --hint=nomultithread module --force purge - module load modenv/hiera NVHPC OpenMPI + module load NVHPC OpenMPI ws=</scratch/ws/spec/installation> cd ${ws} @@ -314,12 +314,12 @@ execution to a different partition. For OpenACC, NVHPC was in the process of adding OpenMP array reduction support which is needed for the `pot3d` benchmark. An Nvidia driver version of 450.80.00 or higher is required. Since - the driver version on partiton `ml` is 440.64.00, it is not supported and not possible to run + the driver version on partiton `power9` is 440.64.00, it is not supported and not possible to run the `pot3d` benchmark in OpenACC mode here. !!! note "Workaround" - As for the partition `ml`, you can only wait until the OS update to CentOS 8 is carried out, + As for the partition `power9`, you can only wait until the OS update to CentOS 8 is carried out, as no driver update will be done beforehand. As a workaround, you can do one of the following: - Exclude the `pot3d` benchmark. @@ -329,7 +329,7 @@ execution to a different partition. !!! warning "Wrong resource distribution" - When working with multiple nodes on partition `ml` or `alpha`, the Slurm parameter + When working with multiple nodes on partition `power9` or `alpha`, the Slurm parameter `$SLURM_NTASKS_PER_NODE` does not work as intended when used in conjunction with `mpirun`. !!! note "Explanation" diff --git a/doc.zih.tu-dresden.de/docs/software/tensorflow.md b/doc.zih.tu-dresden.de/docs/software/tensorflow.md index f95b93687..eb6442663 100644 --- a/doc.zih.tu-dresden.de/docs/software/tensorflow.md +++ b/doc.zih.tu-dresden.de/docs/software/tensorflow.md @@ -23,10 +23,6 @@ and the TensorFlow library. You can find detailed hardware specification in our ## TensorFlow Console -_The module environments /hiera, /scs5, /classic and /ml originated from the old taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - On the cluster `alpha`, load the module environment: ```console @@ -53,13 +49,6 @@ Module TensorFlow/2.9.1 and 35 dependencies loaded. >Module: Recommended toolchain version, load to access other modules that depend on it ``` -On the cluster `power` load the module environment: - -```console -marie@power$ module load modenv/ml -The following have been reloaded with a version change: 1) modenv/scs5 => modenv/ml -``` - This example shows how to install and start working with TensorFlow using the modules system. ```console diff --git a/doc.zih.tu-dresden.de/docs/software/virtual_machines.md b/doc.zih.tu-dresden.de/docs/software/virtual_machines.md index 0738f4fb4..dcd240778 100644 --- a/doc.zih.tu-dresden.de/docs/software/virtual_machines.md +++ b/doc.zih.tu-dresden.de/docs/software/virtual_machines.md @@ -47,7 +47,7 @@ times till it succeeds. bash-4.2$ cat /tmp/marie_2759627/activate #!/bin/bash -if ! grep -q -- "Key for the VM on the partition ml" "/home/marie/.ssh/authorized_keys" > /dev/null; then +if ! grep -q -- "Key for the VM on the partition power9" "/home/marie/.ssh/authorized_keys" > /dev/null; then cat "/tmp/marie_2759627/kvm.pub" >> "/home/marie/.ssh/authorized_keys" else sed -i "s|.*Key for the VM on the cluster power.*|ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC3siZfQ6vQ6PtXPG0RPZwtJXYYFY73TwGYgM6mhKoWHvg+ZzclbBWVU0OoU42B3Ddofld7TFE8sqkHM6M+9jh8u+pYH4rPZte0irw5/27yM73M93q1FyQLQ8Rbi2hurYl5gihCEqomda7NQVQUjdUNVc6fDAvF72giaoOxNYfvqAkw8lFyStpqTHSpcOIL7pm6f76Jx+DJg98sXAXkuf9QK8MurezYVj1qFMho570tY+83ukA04qQSMEY5QeZ+MJDhF0gh8NXjX/6+YQrdh8TklPgOCmcIOI8lwnPTUUieK109ndLsUFB5H0vKL27dA2LZ3ZK+XRCENdUbpdoG2Czz Key for the VM on the cluster power|" "/home/marie/.ssh/authorized_keys" diff --git a/doc.zih.tu-dresden.de/docs/software/visualization.md b/doc.zih.tu-dresden.de/docs/software/visualization.md index 00a49f263..427bf7468 100644 --- a/doc.zih.tu-dresden.de/docs/software/visualization.md +++ b/doc.zih.tu-dresden.de/docs/software/visualization.md @@ -11,10 +11,6 @@ batch and in-situ workflows. ParaView is available on ZIH systems from the [modules system](modules.md#module-environments). The following command lists the available versions -_The module environments /hiera, /scs5, /classic and /ml originated from the taurus system are -momentarily under construction. The script will be updated after completion of the redesign -accordingly_ - ```console marie@login$ module avail ParaView -- GitLab