Skip to content
Snippets Groups Projects
Commit 1cc590a5 authored by Alexander Grund's avatar Alexander Grund
Browse files

Update instructions for custom EB environments

parent d891cb89
No related branches found
No related tags found
2 merge requests!1021Automated merge from preview to main,!1016Update instructions for custom EB environments
......@@ -98,15 +98,12 @@ marie@compute$ module load EasyBuild
This can be either done via environment variables:
```console
marie@compute$ export EASYBUILD_ALLOW_LOADED_MODULES=EasyBuild \
marie@compute$ export EASYBUILD_CONFIGFILES=/software/util/etc/easybuild.d/barnard.cfg \
export EASYBUILD_DETECT_LOADED_MODULES=unload \
export EASYBUILD_BUILDPATH="/tmp/${USER}-EasyBuild${SLURM_JOB_ID:-}" \
export EASYBUILD_SOURCEPATH="${WORKSPACE}/sources" \
export EASYBUILD_INSTALLPATH="${WORKSPACE}/easybuild" \
export EASYBUILD_SUBDIR_MODULES="modules" \
export EASYBUILD_MODULE_NAMING_SCHEME="HierarchicalMNS" \
export EASYBUILD_MODULE_DEPENDS_ON=1 \
export EASYBUILD_HOOKS="/software/util/easybuild/ebhooks.py"
export EASYBUILD_SUBDIR_USER_MODULES= \
export EASYBUILD_BUILDPATH="/dev/shm/${USER}-EasyBuild${SLURM_JOB_ID:-}" \
export EASYBUILD_SOURCEPATH="${WORKSPACE}/sources:/software/util/sources" \
export EASYBUILD_INSTALLPATH="${WORKSPACE}/easybuild"
```
Or you can do that via the configuration file at `$HOME/.config/easybuild/config.cfg`.
......@@ -119,7 +116,7 @@ marie@compute$ eb --confighelp > ~/.config/easybuild/config.cfg
Edit this file by uncommenting the above settings and specifying the respective values.
Note the difference in naming as each setting in the environment has the `EASYBUILD_` prefix
and is uppercase, while it is lowercase in the config.
For example `$EASYBUILD_MODULE_NAMING_SCHEME` above corresponds to `module-naming-scheme`
For example `$EASYBUILD_DETECT_LOADED_MODULES` above corresponds to `detect-loaded-modules`
in the config file.
Note that you cannot use environment variables (like `$WORKSPACE` or `$USER`) in the config file.
......@@ -128,6 +125,12 @@ use of EasyBuild and could be forgotten.
You can also combine those approaches setting some in the config and some in the environment,
the latter will take precedence.
The first variable `$EASYBUILD_CONFIGFILES` makes sure the settings used for installing all other modules
on the cluster are used.
I.e. that config file is read before the custom one in your `$HOME`.
By that most of the configuration is already set up.\
But of course e.g. the installation path needs to be set by you.
The configuration used can be shown via:
```console
......@@ -136,7 +139,7 @@ marie@compute$ eb --show-config
This shows all changed/non-default options while the parameter `--show-full-config` shows all options.
The hierarchical module naming scheme (used on Barnard) affects e.g. location and naming of modules.
The hierarchical module naming scheme (used on our systems) affects e.g. location and naming of modules.
In order for EasyBuild to use the existing modules,
you need to use the "all" modules folder of the main tree.
But likely only the "Core" subdirectory is set in `$MODULEPATH`.
......@@ -150,6 +153,23 @@ marie@compute$ echo $MODULEPATH
/software/modules/rapids/r23.10/all:/software/modules/rapids/r23.10/all/Core:/software/modules/releases/rapids
```
Take care to adjust the path to the release you use.
I.e. in the above example the module `release/23.10` was loaded resulting in
`/software/modules/rapids/r23.10/all/Core` on this cluster.
For the `module use` command you take that (from `$MODULEPATH`) and only strip of the `/Core`.
Or you can use this one-line command to do it automatically:
```console
marie@compute$ ml use $(echo "$MODULEPATH" | grep -oE '(^|:)[^:]+/Core:' | sed 's|/Core:||')
```
Finally, you need to tell LMod about your modules:
```console
marie@compute$ export ZIH_USER_MODULES=$EASYBUILD_INSTALLPATH
```
**Step 7:** Now search for an existing EasyConfig:
```console
......@@ -177,6 +197,7 @@ Using the variable from step 6:
```console
marie@compute$ module use "${EASYBUILD_INSTALLPATH}/modules/all"
marie@compute$ export ZIH_USER_MODULES=$EASYBUILD_INSTALLPATH
marie@compute$ export LMOD_IGNORE_CACHE=1
```
......@@ -184,6 +205,7 @@ marie@compute$ export LMOD_IGNORE_CACHE=1
```console
marie@compute$ module use "/data/horse/ws/marie-EasyBuild/easybuild/modules/all"
marie@compute$ export ZIH_USER_MODULES=/data/horse/ws/marie-EasyBuild/easybuild
marie@compute$ export LMOD_IGNORE_CACHE=1
```
......
......@@ -209,6 +209,7 @@ LINPACK
linter
Linter
lmod
LMod
LoadLeveler
localhost
lsf
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment