Skip to content
Snippets Groups Projects
Commit 800504e4 authored by Martin Schroschk's avatar Martin Schroschk
Browse files

Rework, simplify and provide template in Lua

Resolve #283
parent 0337774d
No related branches found
No related tags found
2 merge requests!530Automated merge from preview to main,!507Rework, simplify and provide template in Lua
......@@ -2,104 +2,134 @@
Private module files allow you to load your own installed software packages into your environment
and to handle different versions without getting into conflicts. Private modules can be setup for a
single user as well as all users of project group. The workflow and settings for user private module
files is described in the following. The [settings for project private
modules](#project-private-modules) differ only in details.
**single user** as well as **all users of a project group**. The workflow and settings for user
private as well as project private module files are described in the following.
In order to use your own module files please use the command
`module use <path_to_module_files>`. It will add the path to the list of module directories
that are searched by lmod (i.e. the `module` command). You may use a directory `privatemodules`
within your home or project directory to setup your own module files.
## Setup
Please see the [Environment Modules open source project's web page](http://modules.sourceforge.net/)
for further information on writing module files.
### 0. Build and Install Software
## 1. Create Directories
Obviously the first step is to build and install the software you'd like to use. Please follow the
instructions and tips provided on the page [building_software](building_software.md).
For consistency, we use the placeholder variable `<sw_name>` in this documentation. When following this
instructions, please substitute it with the actual software name within the commands.
### 1. Create Directory
Now, create the directory `privatemodules` to store all your private module files and the directory
`sw_name` therein. All module files for different versions or build options of `sw_name` should be
located in this directory.
```console
marie@compute$ cd $HOME
marie@compute$ mkdir --verbose --parents privatemodules/testsoftware
marie@compute$ cd privatemodules/testsoftware
marie@compute$ mkdir --verbose --parents privatemodules/<sw_name>
marie@compute$ cd privatemodules/<sw_name>
```
(create a directory in your home directory)
## 2. Notify lmod
Project private module files for software that can be used by all members of your group should be
located in your global projects directory, e.g., `/projects/p_marie/privatemodules`. Thus, create
this directory:
```console
marie@compute$ module use $HOME/privatemodules
marie@compute$ mkdir --verbose --parents /projects/p_marie/privatemodules/<sw_name>
marie@compute$ cd /projects/p_marie/privatemodules/<sw_name>
```
(add the directory in the list of module directories)
!!! note
## 3. Create Modulefile
Make sure, that the directory is group-readable.
Create a file with the name `1.0` with a
test software in the `testsoftware` directory you created earlier
(using your favorite editor) and paste the following text into it:
### 2. Create Modulefile
```
#%Module######################################################################
##
## testsoftware modulefile
##
proc ModulesHelp { } {
puts stderr "Loads testsoftware"
}
set version 1.0
set arch x86_64
set path /home/<user>/opt/testsoftware/$version/$arch/
prepend-path PATH $path/bin
prepend-path LD_LIBRARY_PATH $path/lib
if [ module-info mode load ] {
puts stderr "Load testsoftware version $version"
}
```
Within the directory `<sw_name>` create the module file. The file can either be a TCL or a Lua. We
recommend to use Lua. The module file name should reflect the particular version of the software,
e.g., `1.4.lua`.
## 4. Check lmod
!!! note
Check the availability of the module with `ml av`, the output should look like this:
If you create a group private module file, make sure it is group-readable.
```
--------------------- /home/masterman/privatemodules ---------------------
testsoftware/1.0
```
A template module file is:
## 5. Load Module
```lua linenums="1"
help([[
Load the test module with `module load testsoftware`, the output should look like this:
Description
===========
<sw_name> is ...
```console
Load testsoftware version 1.0
Module testsoftware/1.0 loaded.
More Information
================
For detailed instructions, go to:
https://...
]])
whatis("Version: 1.4")
whatis("Keywords: [System, Utility, ...]")
whatis("URL: <...>")
whatis("Description: <...>")
conflict("<sw_name>")
local root = "</path/to/installation>"
prepend_path( "PATH", pathJoin(root, "bin"))
prepend_path( "LD_LIBRARY_PATH", pathJoin(root, "lib"))
prepend_path( "LIBRARY_PATH", pathJoin(root, "lib"))
setenv( "<SOME_ENV>", "<value>")
```
## Project Private Modules
There most important functions to adjust the environment are listed and described in the following
table.
| Function | Description |
|----------|-------------|
| `help([[ help string ]]) ` | Message when the help command is called. |
| `conflict(“name1”, “name2”)` | The current modulefile will only load if all listed modules are NOT loaded. |
| `depends_on(“pkgA”, “pkgB”, “pkgC”)` | Loads all modules. When unloading only dependent modules are unloaded. |
| `load(“pkgA”, “pkgB”, “pkgC”)` | Load all modules. Report error if unable to load.
| `prepend_path(”PATH”, “/path/to/pkg/bin”)` | Prepend the value to a path-like variable. |
| `setenv(“NAME”, “value”):` | Assign the value to the environment variable `NAME`. |
Please refer to the official documentation of Lmod on
[writing modules](https://lmod.readthedocs.io/en/latest/015_writing_modules.html) and
[Lua ModulefileFunctions](https://lmod.readthedocs.io/en/latest/050_lua_modulefiles.html)
for detailed information.
You can also have a look at present module files at the system.
Private module files allow you to load project- or group-wide installed software into your
environment and to handle different versions without getting into conflicts.
## Usage
The module files have to be stored in your global projects directory
`/projects/p_projectname/privatemodules`. An example of a module file can be found in the section
above. To use a project-wide module file you have to add the path to the module file to the module
environment with the command
In order to use private module files and the corresponding software, you need to expand the module
search path. This is done by invoking the command
```console
marie@compute$ module use /projects/p_projectname/privatemodules
marie@login$ module use $HOME/privatemodules
```
After that, the modules are available in your module environment and you can load the modules with
the `module load` command.
for your private module files and
```console
marie@login$ module use /projects/p_marie/privatemodules
```
## Using Private Modules and Programs in the $HOME Directory
for group private module files, respectively.
An automated backup system provides security for the HOME-directories on the cluster on a daily
Afterwards, you can use the [module commands](modules.md) to, e.g., load and unload your private modules
as usual.
## Caveats
An automated backup system provides security for the `/home` directories on the cluster on a daily
basis. This is the reason why we urge users to store (large) temporary data (like checkpoint files)
on the /scratch filesystem or at local scratch disks.
on the `/scratch` filesystem or at local scratch disks.
This also why, we have set `ulimit -c 0` as a default setting to prevent users from filling the
`/home` directories with the dump of crashed programs. In particular, `ulimit -c 0` sets the core
file size (blocks) to 0, which disables creation of core dumbs in case an application crashes.
!!! note "Enable core files for debugging"
**Please note**: We have set `ulimit -c 0` as a default to prevent users from filling the disk with
the dump of crashed programs. `bash` users can use `ulimit -Sc unlimited` to enable the debugging
via analyzing the core file.
If you use `bash` as shell and you need these core files for analysis, set `ulimit -c
unlimited`.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment