Merge branch 'preview' into merge-preview-in-main

doc.zih.tu-dresden.de/docs/jobs_and_resources/overview.md

@@ -56,7 +56,7 @@ The following questions may help to decide which cluster to use
 |**Barnard**<br>_2023_| CPU|`n[node].barnard.hpc.tu-dresden.de` |n[1001-1630] | 630 |104| 2 |515,000 |2,475 | 0 |
 |**Alpha**<br>_2021_| GPU |`i[node].alpha.hpc.tu-dresden.de`|taurusi[8001-8034] |  34 | 48 | 2 | 990,000 | 10,312|  8  |
 |**Romeo**<br>_2020_| CPU |`i[node].romeo.hpc.tu-dresden.de`|taurusi[7001-7192] | 192|128 | 2 | 505,000| 1,972 |  0  |
-|**Julia**<br>_2021_| single SMP system |`julia.hpc.tu-dresden.de`|taurusa[3-16] | 14 | 12 | 1 | 95,000 | 7,916 | 3 |
+|**Julia**<br>_2021_| single SMP system |`julia.hpc.tu-dresden.de`| julia | 1 | 896 | 1 | 48,390,000 | 54,006 | - |
 |**Power**<br>_2018_|IBM Power/GPU system |`ml[node].power9.hpc.tu-dresden.de`|taurusml[3-32] | 30 | 44 | 4 | 254,000 | 1,443 | 6 |
 {: summary="cluster overview table" align="bottom"}
 

doc.zih.tu-dresden.de/docs/software/custom_easy_build_environment.md

@@ -98,15 +98,12 @@ marie@compute$ module load EasyBuild
 This can be either done via environment variables:
 
 ```console
-marie@compute$ export EASYBUILD_ALLOW_LOADED_MODULES=EasyBuild \
+marie@compute$ export EASYBUILD_CONFIGFILES=/software/util/etc/easybuild.d/barnard.cfg \
 export EASYBUILD_DETECT_LOADED_MODULES=unload \
-export EASYBUILD_BUILDPATH="/tmp/${USER}-EasyBuild${SLURM_JOB_ID:-}" \
-export EASYBUILD_SOURCEPATH="${WORKSPACE}/sources" \
-export EASYBUILD_INSTALLPATH="${WORKSPACE}/easybuild" \
-export EASYBUILD_SUBDIR_MODULES="modules" \
-export EASYBUILD_MODULE_NAMING_SCHEME="HierarchicalMNS" \
-export EASYBUILD_MODULE_DEPENDS_ON=1 \
-export EASYBUILD_HOOKS="/software/util/easybuild/ebhooks.py"
+export EASYBUILD_SUBDIR_USER_MODULES= \
+export EASYBUILD_BUILDPATH="/dev/shm/${USER}-EasyBuild${SLURM_JOB_ID:-}" \
+export EASYBUILD_SOURCEPATH="${WORKSPACE}/sources:/software/util/sources" \
+export EASYBUILD_INSTALLPATH="${WORKSPACE}/easybuild"
 ```
 
 Or you can do that via the configuration file at `$HOME/.config/easybuild/config.cfg`.
@@ -119,7 +116,7 @@ marie@compute$ eb --confighelp > ~/.config/easybuild/config.cfg
 Edit this file by uncommenting the above settings and specifying the respective values.
 Note the difference in naming as each setting in the environment has the `EASYBUILD_` prefix
 and is uppercase, while it is lowercase in the config.
-For example `$EASYBUILD_MODULE_NAMING_SCHEME` above corresponds to `module-naming-scheme`
+For example `$EASYBUILD_DETECT_LOADED_MODULES` above corresponds to `detect-loaded-modules`
 in the config file.
 
 Note that you cannot use environment variables (like `$WORKSPACE` or `$USER`) in the config file.
@@ -128,6 +125,12 @@ use of EasyBuild and could be forgotten.
 
 You can also combine those approaches setting some in the config and some in the environment,
 the latter will take precedence.
+The first variable `$EASYBUILD_CONFIGFILES` makes sure the settings used for installing all other modules
+on the cluster are used.
+I.e. that config file is read before the custom one in your `$HOME`.
+By that most of the configuration is already set up.\
+But of course e.g. the installation path needs to be set by you.
+
 The configuration used can be shown via:
 
 ```console
@@ -136,7 +139,7 @@ marie@compute$ eb --show-config
 
 This shows all changed/non-default options while the parameter `--show-full-config` shows all options.
 
-The hierarchical module naming scheme (used on Barnard) affects e.g. location and naming of modules.
+The hierarchical module naming scheme (used on our systems) affects e.g. location and naming of modules.
 In order for EasyBuild to use the existing modules,
 you need to use the "all" modules folder of the main tree.
 But likely only the "Core" subdirectory is set in `$MODULEPATH`.
@@ -150,6 +153,23 @@ marie@compute$ echo $MODULEPATH
 /software/modules/rapids/r23.10/all:/software/modules/rapids/r23.10/all/Core:/software/modules/releases/rapids
 ```
 
+Take care to adjust the path to the release you use.
+I.e. in the above example the module `release/23.10` was loaded resulting in
+`/software/modules/rapids/r23.10/all/Core` on this cluster.
+For the `module use` command you take that (from `$MODULEPATH`) and only strip of the `/Core`.
+
+Or you can use this one-line command to do it automatically:
+
+```console
+marie@compute$ ml use $(echo "$MODULEPATH" | grep -oE '(^|:)[^:]+/Core:' | sed 's|/Core:||')
+```
+
+Finally, you need to tell LMod about your modules:
+
+```console
+marie@compute$ export ZIH_USER_MODULES=$EASYBUILD_INSTALLPATH
+```
+
 **Step 7:** Now search for an existing EasyConfig:
 
 ```console
@@ -177,6 +197,7 @@ Using the variable from step 6:
 
 ```console
 marie@compute$ module use "${EASYBUILD_INSTALLPATH}/modules/all"
+marie@compute$ export ZIH_USER_MODULES=$EASYBUILD_INSTALLPATH
 marie@compute$ export LMOD_IGNORE_CACHE=1
 ```
 
@@ -184,6 +205,7 @@ marie@compute$ export LMOD_IGNORE_CACHE=1
 
 ```console
 marie@compute$ module use "/data/horse/ws/marie-EasyBuild/easybuild/modules/all"
+marie@compute$ export ZIH_USER_MODULES=/data/horse/ws/marie-EasyBuild/easybuild
 marie@compute$ export LMOD_IGNORE_CACHE=1
 ```
 
@@ -217,3 +239,27 @@ It might also be helpful to use
 ```console
 marie@compute$ export LMOD_IGNORE_CACHE=0
 ```
+
+(Especially) when you want to use additional features of EasyBuild, such as the
+[GitHub integration](https://docs.easybuild.io/integration-with-github/),
+ you might need to set a specific Python version to use by EasyBuild.
+
+That is unrelated to any Python module you might wish to use or install!
+Furthermore, when using EasyBuild you should **not** have any other modules loaded,
+not even `Python`.
+
+Which Python executable is used by EasyBuild can be shown by executing:
+
+```console
+marie@compute$ EB_VERBOSE=1 eb --version
+```
+
+You can change it by setting `EB_PYTHON`, e.g.:
+
+```console
+marie@compute$ export EB_PYTHON=python3.8
+```
+
+In case you are using a virtualenv for use with EasyBuild
+then using `python` instead of `python3.8` or similar is enough
+as there will be a `python` binary available inside your virtualenv.

doc.zih.tu-dresden.de/docs/software/machine_learning.md

@@ -142,8 +142,8 @@ still need to download some datasets use [Datamover](../data_transfer/datamover.
 The ImageNet project is a large visual database designed for use in visual object recognition
 software research. In order to save space in the filesystem by avoiding to have multiple duplicates
 of this lying around, we have put a copy of the ImageNet database (ILSVRC2012 and ILSVR2017) under
-`/data/horse/imagenet` which you can use without having to download it again. For the future, the
-ImageNet dataset will be available in
+`/data/horse/shared/imagenet` which you can use without having to download it again. For the future,
+the ImageNet dataset will be available in
 [Warm Archive](../data_lifecycle/workspaces.md#mid-term-storage). ILSVR2017 also includes a dataset
 for recognition objects from a video. Please respect the corresponding
 [Terms of Use](https://image-net.org/download.php).

doc.zih.tu-dresden.de/wordlist.aspell

@@ -209,6 +209,7 @@ LINPACK
 linter
 Linter
 lmod
+LMod
 LoadLeveler
 localhost
 lsf