diff --git a/doc.zih.tu-dresden.de/docs/software/private_modules.md b/doc.zih.tu-dresden.de/docs/software/private_modules.md index 56ad32a45d7bbb4ec5d0fa7098f553567ce84a56..92c7ce8b25b20b34b5d02170a436ab1c3220096f 100644 --- a/doc.zih.tu-dresden.de/docs/software/private_modules.md +++ b/doc.zih.tu-dresden.de/docs/software/private_modules.md @@ -2,104 +2,134 @@ Private module files allow you to load your own installed software packages into your environment and to handle different versions without getting into conflicts. Private modules can be setup for a -single user as well as all users of project group. The workflow and settings for user private module -files is described in the following. The [settings for project private -modules](#project-private-modules) differ only in details. +**single user** as well as **all users of a project group**. The workflow and settings for user +private as well as project private module files are described in the following. -In order to use your own module files please use the command -`module use <path_to_module_files>`. It will add the path to the list of module directories -that are searched by lmod (i.e. the `module` command). You may use a directory `privatemodules` -within your home or project directory to setup your own module files. +## Setup -Please see the [Environment Modules open source project's web page](http://modules.sourceforge.net/) -for further information on writing module files. +### 0. Build and Install Software -## 1. Create Directories +Obviously the first step is to build and install the software you'd like to use. Please follow the +instructions and tips provided on the page [building_software](building_software.md). +For consistency, we use the placeholder variable `<sw_name>` in this documentation. When following this +instructions, please substitute it with the actual software name within the commands. + +### 1. Create Directory + +Now, create the directory `privatemodules` to store all your private module files and the directory +`sw_name` therein. All module files for different versions or build options of `sw_name` should be +located in this directory. ```console marie@compute$ cd $HOME -marie@compute$ mkdir --verbose --parents privatemodules/testsoftware -marie@compute$ cd privatemodules/testsoftware +marie@compute$ mkdir --verbose --parents privatemodules/<sw_name> +marie@compute$ cd privatemodules/<sw_name> ``` -(create a directory in your home directory) - -## 2. Notify lmod +Project private module files for software that can be used by all members of your group should be +located in your global projects directory, e.g., `/projects/p_marie/privatemodules`. Thus, create +this directory: ```console -marie@compute$ module use $HOME/privatemodules +marie@compute$ mkdir --verbose --parents /projects/p_marie/privatemodules/<sw_name> +marie@compute$ cd /projects/p_marie/privatemodules/<sw_name> ``` -(add the directory in the list of module directories) +!!! note -## 3. Create Modulefile + Make sure, that the directory is group-readable. -Create a file with the name `1.0` with a -test software in the `testsoftware` directory you created earlier -(using your favorite editor) and paste the following text into it: +### 2. Create Modulefile -``` -#%Module###################################################################### -## -## testsoftware modulefile -## -proc ModulesHelp { } { - puts stderr "Loads testsoftware" -} - -set version 1.0 -set arch x86_64 -set path /home/<user>/opt/testsoftware/$version/$arch/ - -prepend-path PATH $path/bin -prepend-path LD_LIBRARY_PATH $path/lib - -if [ module-info mode load ] { - puts stderr "Load testsoftware version $version" -} -``` +Within the directory `<sw_name>` create the module file. The file can either be a TCL or a Lua. We +recommend to use Lua. The module file name should reflect the particular version of the software, +e.g., `1.4.lua`. -## 4. Check lmod +!!! note -Check the availability of the module with `ml av`, the output should look like this: + If you create a group private module file, make sure it is group-readable. -``` ---------------------- /home/masterman/privatemodules --------------------- - testsoftware/1.0 -``` +A template module file is: -## 5. Load Module +```lua linenums="1" +help([[ -Load the test module with `module load testsoftware`, the output should look like this: +Description +=========== +<sw_name> is ... -```console -Load testsoftware version 1.0 -Module testsoftware/1.0 loaded. +More Information +================ +For detailed instructions, go to: + https://... + +]]) + +whatis("Version: 1.4") +whatis("Keywords: [System, Utility, ...]") +whatis("URL: <...>") +whatis("Description: <...>") + +conflict("<sw_name>") + +local root = "</path/to/installation>" +prepend_path( "PATH", pathJoin(root, "bin")) +prepend_path( "LD_LIBRARY_PATH", pathJoin(root, "lib")) +prepend_path( "LIBRARY_PATH", pathJoin(root, "lib")) + +setenv( "<SOME_ENV>", "<value>") ``` -## Project Private Modules +There most important functions to adjust the environment are listed and described in the following +table. + +| Function | Description | +|----------|-------------| +| `help([[ help string ]]) ` | Message when the help command is called. | +| `conflict(“name1â€, “name2â€)` | The current modulefile will only load if all listed modules are NOT loaded. | +| `depends_on(“pkgAâ€, “pkgBâ€, “pkgCâ€)` | Loads all modules. When unloading only dependent modules are unloaded. | +| `load(“pkgAâ€, “pkgBâ€, “pkgCâ€)` | Load all modules. Report error if unable to load. +| `prepend_path(â€PATHâ€, “/path/to/pkg/binâ€)` | Prepend the value to a path-like variable. | +| `setenv(“NAMEâ€, “valueâ€):` | Assign the value to the environment variable `NAME`. | + +Please refer to the official documentation of Lmod on +[writing modules](https://lmod.readthedocs.io/en/latest/015_writing_modules.html) and +[Lua ModulefileFunctions](https://lmod.readthedocs.io/en/latest/050_lua_modulefiles.html) +for detailed information. +You can also have a look at present module files at the system. -Private module files allow you to load project- or group-wide installed software into your -environment and to handle different versions without getting into conflicts. +## Usage -The module files have to be stored in your global projects directory -`/projects/p_projectname/privatemodules`. An example of a module file can be found in the section -above. To use a project-wide module file you have to add the path to the module file to the module -environment with the command +In order to use private module files and the corresponding software, you need to expand the module +search path. This is done by invoking the command ```console -marie@compute$ module use /projects/p_projectname/privatemodules +marie@login$ module use $HOME/privatemodules ``` -After that, the modules are available in your module environment and you can load the modules with -the `module load` command. +for your private module files and + +```console +marie@login$ module use /projects/p_marie/privatemodules +``` -## Using Private Modules and Programs in the $HOME Directory +for group private module files, respectively. -An automated backup system provides security for the HOME-directories on the cluster on a daily +Afterwards, you can use the [module commands](modules.md) to, e.g., load and unload your private modules +as usual. + +## Caveats + +An automated backup system provides security for the `/home` directories on the cluster on a daily basis. This is the reason why we urge users to store (large) temporary data (like checkpoint files) -on the /scratch filesystem or at local scratch disks. +on the `/scratch` filesystem or at local scratch disks. + + +This also why, we have set `ulimit -c 0` as a default setting to prevent users from filling the +`/home` directories with the dump of crashed programs. In particular, `ulimit -c 0` sets the core +file size (blocks) to 0, which disables creation of core dumbs in case an application crashes. + +!!! note "Enable core files for debugging" -**Please note**: We have set `ulimit -c 0` as a default to prevent users from filling the disk with -the dump of crashed programs. `bash` users can use `ulimit -Sc unlimited` to enable the debugging -via analyzing the core file. + If you use `bash` as shell and you need these core files for analysis, set `ulimit -c + unlimited`.