Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
hpc-compendium
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Deploy
Releases
Package registry
Container Registry
Model registry
Operate
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
ZIH
hpcsupport
hpc-compendium
Commits
800504e4
Commit
800504e4
authored
3 years ago
by
Martin Schroschk
Browse files
Options
Downloads
Patches
Plain Diff
Rework, simplify and provide template in Lua
Resolve
#283
parent
0337774d
No related branches found
No related tags found
2 merge requests
!530
Automated merge from preview to main
,
!507
Rework, simplify and provide template in Lua
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
doc.zih.tu-dresden.de/docs/software/private_modules.md
+98
-68
98 additions, 68 deletions
doc.zih.tu-dresden.de/docs/software/private_modules.md
with
98 additions
and
68 deletions
doc.zih.tu-dresden.de/docs/software/private_modules.md
+
98
−
68
View file @
800504e4
...
...
@@ -2,104 +2,134 @@
Private module files allow you to load your own installed software packages into your environment
and to handle different versions without getting into conflicts. Private modules can be setup for a
single user as well as all users of project group. The workflow and settings for user private module
files is described in the following. The
[
settings for project private
modules
](
#project-private-modules
)
differ only in details.
**single user**
as well as
**all users of a project group**
. The workflow and settings for user
private as well as project private module files are described in the following.
In order to use your own module files please use the command
`module use <path_to_module_files>`
. It will add the path to the list of module directories
that are searched by lmod (i.e. the
`module`
command). You may use a directory
`privatemodules`
within your home or project directory to setup your own module files.
## Setup
Please see the
[
Environment Modules open source project's web page
](
http://modules.sourceforge.net/
)
for further information on writing module files.
### 0. Build and Install Software
## 1. Create Directories
Obviously the first step is to build and install the software you'd like to use. Please follow the
instructions and tips provided on the page
[
building_software
](
building_software.md
)
.
For consistency, we use the placeholder variable
`<sw_name>`
in this documentation. When following this
instructions, please substitute it with the actual software name within the commands.
### 1. Create Directory
Now, create the directory
`privatemodules`
to store all your private module files and the directory
`sw_name`
therein. All module files for different versions or build options of
`sw_name`
should be
located in this directory.
```
console
marie@compute$
cd
$HOME
marie@compute$
mkdir
--verbose
--parents
privatemodules/
testsoftware
marie@compute$
cd
privatemodules/
testsoftware
marie@compute$
mkdir
--verbose
--parents
privatemodules/
<sw_name>
marie@compute$
cd
privatemodules/
<sw_name>
```
(create a directory in your home directory)
## 2. Notify lmod
Project private module files for software that can be used by all members of your group should be
located in your global projects directory, e.g.,
`/projects/p_marie/privatemodules`
. Thus, create
this directory:
```
console
marie@compute$
module use
$HOME
/privatemodules
marie@compute$
mkdir
--verbose
--parents
/projects/p_marie/privatemodules/<sw_name>
marie@compute$
cd
/projects/p_marie/privatemodules/<sw_name>
```
(add the directory in the list of module directories)
!!! note
## 3. Create Modulefi
le
Make sure, that the directory is group-readab
le
.
Create a file with the name
`1.0`
with a
test software in the
`testsoftware`
directory you created earlier
(using your favorite editor) and paste the following text into it:
### 2. Create Modulefile
```
#%Module######################################################################
##
## testsoftware modulefile
##
proc ModulesHelp { } {
puts stderr "Loads testsoftware"
}
set version 1.0
set arch x86_64
set path /home/<user>/opt/testsoftware/$version/$arch/
prepend-path PATH $path/bin
prepend-path LD_LIBRARY_PATH $path/lib
if [ module-info mode load ] {
puts stderr "Load testsoftware version $version"
}
```
Within the directory
`<sw_name>`
create the module file. The file can either be a TCL or a Lua. We
recommend to use Lua. The module file name should reflect the particular version of the software,
e.g.,
`1.4.lua`
.
## 4. Check lmod
!!! note
Check the availability of the module with
`ml av`
, the output should look like this:
If you create a group private module file, make sure it is group-readable.
```
--------------------- /home/masterman/privatemodules ---------------------
testsoftware/1.0
```
A template module file is:
## 5. Load Module
```
lua linenums="1"
help([[
Load the test module with
`module load testsoftware`
, the output should look like this:
Description
===========
<sw_name> is ...
```
console
Load testsoftware version 1.0
Module testsoftware/1.0 loaded.
More Information
================
For detailed instructions, go to:
https://...
]])
whatis("Version: 1.4")
whatis("Keywords: [System, Utility, ...]")
whatis("URL: <...>")
whatis("Description: <...>")
conflict("<sw_name>")
local root = "</path/to/installation>"
prepend_path( "PATH", pathJoin(root, "bin"))
prepend_path( "LD_LIBRARY_PATH", pathJoin(root, "lib"))
prepend_path( "LIBRARY_PATH", pathJoin(root, "lib"))
setenv( "<SOME_ENV>", "<value>")
```
## Project Private Modules
There most important functions to adjust the environment are listed and described in the following
table.
| Function | Description |
|----------|-------------|
|
`help([[ help string ]]) `
| Message when the help command is called. |
|
`conflict(“name1”, “name2”)`
| The current modulefile will only load if all listed modules are NOT loaded. |
|
`depends_on(“pkgA”, “pkgB”, “pkgC”)`
| Loads all modules. When unloading only dependent modules are unloaded. |
|
`load(“pkgA”, “pkgB”, “pkgC”)`
| Load all modules. Report error if unable to load.
|
`prepend_path(”PATH”, “/path/to/pkg/bin”)`
| Prepend the value to a path-like variable. |
|
`setenv(“NAME”, “value”):`
| Assign the value to the environment variable
`NAME`
. |
Please refer to the official documentation of Lmod on
[
writing modules
](
https://lmod.readthedocs.io/en/latest/015_writing_modules.html
)
and
[
Lua ModulefileFunctions
](
https://lmod.readthedocs.io/en/latest/050_lua_modulefiles.html
)
for detailed information.
You can also have a look at present module files at the system.
Private module files allow you to load project- or group-wide installed software into your
environment and to handle different versions without getting into conflicts.
## Usage
The module files have to be stored in your global projects directory
`/projects/p_projectname/privatemodules`
. An example of a module file can be found in the section
above. To use a project-wide module file you have to add the path to the module file to the module
environment with the command
In order to use private module files and the corresponding software, you need to expand the module
search path. This is done by invoking the command
```
console
marie@
compute
$
module use
/projects/p_projectname
/privatemodules
marie@
login
$
module use
$HOME
/privatemodules
```
After that, the modules are available in your module environment and you can load the modules with
the
`module load`
command.
for your private module files and
```
console
marie@login$
module use /projects/p_marie/privatemodules
```
## Using P
rivate
M
odule
s and Programs in the $HOME Directory
for group p
rivate
m
odule
files, respectively.
An automated backup system provides security for the HOME-directories on the cluster on a daily
Afterwards, you can use the
[
module commands
](
modules.md
)
to, e.g., load and unload your private modules
as usual.
## Caveats
An automated backup system provides security for the
`/home`
directories on the cluster on a daily
basis. This is the reason why we urge users to store (large) temporary data (like checkpoint files)
on the /scratch filesystem or at local scratch disks.
on the
`/scratch`
filesystem or at local scratch disks.
This also why, we have set
`ulimit -c 0`
as a default setting to prevent users from filling the
`/home`
directories with the dump of crashed programs. In particular,
`ulimit -c 0`
sets the core
file size (blocks) to 0, which disables creation of core dumbs in case an application crashes.
!!! note "Enable core files for debugging"
**Please note**
: We have set
`ulimit -c 0`
as a default to prevent users from filling the disk with
the dump of crashed programs.
`bash`
users can use
`ulimit -Sc unlimited`
to enable the debugging
via analyzing the core file.
If you use `bash` as shell and you need these core files for analysis, set `ulimit -c
unlimited`.
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment