diff --git a/doc.zih.tu-dresden.de/docs/software/virtual_desktops.md b/doc.zih.tu-dresden.de/docs/software/virtual_desktops.md index 123c323b2d3acc9c24863ca203179f8338da4dce..cb809c3a99022c8ec5c5d3e9a98b96b8533baa0b 100644 --- a/doc.zih.tu-dresden.de/docs/software/virtual_desktops.md +++ b/doc.zih.tu-dresden.de/docs/software/virtual_desktops.md @@ -1,89 +1,3 @@ -# Virtual desktops +# Virtual Desktops -Use WebVNC or NICE DCV to run GUI applications on HPC resources. - -<span class="twiki-macro TABLE" columnwidths="10%,45%,45%"></span> - -| | | | -|----------------|-------------------------------------------------------|-------------------------------------------------| -| | **WebVNC** | **NICE DCV** | -| **use case** | all GUI applications that do \<u>not need\</u> OpenGL | only GUI applications that \<u>need\</u> OpenGL | -| **partitions** | all\* (except partitions with GPUs (gpu2, hpdlf, ml) | dcv | - -## Launch a virtual desktop - -<span class="twiki-macro TABLE" columnwidths="10%,45%,45%"></span> \| -**step 1** \| Navigate to \<a href="<https://taurus.hrsk.tu-dresden.de>" -target="\_blank"><https://taurus.hrsk.tu-dresden.de>\</a>. There is our -[JupyterHub](../access/jupyterhub.md) instance. \|\| \| **step 2** \| -Click on the "advanced" tab and choose a preset: \|\| - -| | | | -|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------| -| ^ | **WebVNC** | **DCV** | -| **step 3** | Optional: Finetune your session with the available SLURM job parameters or assign a certain project or reservation. Then save your settings in a new preset for future use. | | -| **step 4** | Click on "Spawn". JupyterHub starts now a SLURM job for you. If everything is ready the JupyterLab interface will appear to you. | | -| **step 5"** | Click on the **button "WebVNC"** to start a virtual desktop. | Click on the \*button "NICE DCV"to start a virtual desktop. | -| ^ | The virtual desktop starts in a new tab or window. | | - -### Demonstration - -\<video controls="" width="320" style="border: 1px solid black">\<source -src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/VirtualDesktops/start-virtual-desktop-dcv.mp4>" -type="video/mp4">\<source -src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/VirtualDesktops/start-virtual-desktop-dcv.webm>" -type="video/webm">\</video> - -### Using the quickstart feature - -JupyterHub can start a job automatically if the URL contains certain -parameters. - -<span class="twiki-macro TABLE" columnwidths="10%,45%,45%"></span> - -| | | | -|----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| examples | \<a href="<https://taurus.hrsk.tu-dresden.de/jupyter/hub/spawn#/>\~(partition\~'interactive\~cpuspertask\~'2\~mempercpu\~'2583)" target="\_blank" style="font-size: 1.5em">WebVNC\</a> | \<a href="<https://taurus.hrsk.tu-dresden.de/jupyter/hub/spawn#/>\~(partition\~'dcv\~cpuspertask\~'6\~gres\~'gpu\*3a1\~mempercpu\~'2583)" target="\_blank" style="font-size: 1.5em">NICE DCV\</a> | -| details about the examples | `interactive` partition, 2 CPUs with 2583 MB RAM per core, no GPU | `dcv` partition, 6 CPUs with 2583 MB RAM per core, 1 GPU | -| link creator | Use the spawn form to set your preferred options. The browser URL will be updated with the corresponding parameters. | | - -If you close the browser tabs or windows or log out from your local -machine, you are able to open the virtual desktop later again - as long -as the session runs. But please remember that a SLURM job is running in -the background which has a certain timelimit. - -## Reconnecting to a session - -In order to reconnect to an active instance of WebVNC, simply repeat the -steps required to start a session, beginning - if required - with the -login, then clicking "My server", then by pressing the "+" sign on the -upper left corner. Provided your server is still running and you simply -closed the window or logged out without stopping your server, you will -find your WebVNC desktop the way you left it. - -## Terminate a remote session - -<span class="twiki-macro TABLE" columnwidths="10%,90%"></span> \| **step -1** \| Close the VNC viewer tab or window. \| \| **step 2** \| Click on -File \> Log Out in the JupyterLab main menu. Now you get redirected to -the JupyterLab control panel. If you don't have your JupyterLab tab or -window anymore, navigate directly to \<a -href="<https://taurus.hrsk.tu-dresden.de/jupyter/hub/home>" -target="\_blank"><https://taurus.hrsk.tu-dresden.de/jupyter/hub/home>\</a>. -\| \| **step 3** \| Click on "Stop My Server". This cancels the SLURM -job and terminates your session. \| - -### Demonstration - -\<video controls="" width="320" style="border: 1px solid black">\<source -src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/VirtualDesktops/terminate-virtual-desktop-dcv.mp4>" -type="video/mp4">\<source -src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/VirtualDesktops/terminate-virtual-desktop-dcv.webm>" -type="video/webm">\</video> - -**Remark:** This does not work if you click on the "Logout"-Btn in your -virtual desktop. Instead this will just close your DCV session or cause -a black screen in your WebVNC window without a possibility to recover a -virtual desktop in the same Jupyter session. The solution for now would -be to terminate the whole jupyter session and start a new one like -mentioned above. +coming soon diff --git a/doc.zih.tu-dresden.de/hackathon.md b/doc.zih.tu-dresden.de/hackathon.md deleted file mode 100644 index d41781c45455139c62708c708cf42e05babc3b65..0000000000000000000000000000000000000000 --- a/doc.zih.tu-dresden.de/hackathon.md +++ /dev/null @@ -1,75 +0,0 @@ -# Hackathon June 2021 - -The goals for the hackathon are: - -* Familiarize main editors (ZIH admin group and domain experts) with new workflow and system -* Bringing new compendium to life by - 1. Transferring content from old compendium into new structure and system - 1. Fixing checks - 1. Reviewing and updating transferred content - -## twiki2md - -The script `twiki2md` converts twiki source files into markdown source files using pandoc. It -outputs the markdown source files according to the old pages tree into subdirectories. The -output and **starting point for transferring** old content into the new system can be found -at branch `preview` within directory `twiki2md/root/`. - -## Steps - -### Familiarize with New Wiki System - -* Make sure your are member of the [repository](https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium). - If not, ask Danny Rotscher for adding you. -* Clone repository and checkout branch `preview` - -```Shell Session -~ git clone git@gitlab.hrz.tu-chemnitz.de:zih/hpcsupport/hpc-compendium.git -~ cd hpc-compendium -~ git checkout preview -``` - -* Open terminal and build documentation using `mkdocs` - * [using mkdocs](README.md#preview-using-mkdocs) - * [installing dependencies](README.md#install-dependencies) - -### Transferring Content - -1. Grab a markdown source file from `twiki2md/root/` directory (a topic you are comfortable with) -1. Find place in new structure according to -[Typical Project Schedule](https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/TypicalProjectSchedule) - - * Create new feature branch holding your work `~ git checkout -b <BRANCHNAME>`, whereas branch - name can be `<FILENAME>` for simplicity - * Copy reviewed markdown source file to `docs/` directory via - `~ git mv twiki2md/root/<FILENAME>.md doc.zih.tu-dresden.de/docs/<SUBDIR>/<FILENAME>.md` - * Update navigation section in `mkdocs.yaml` - -1. Commit and push to feature branch via - -```Shell Session -~ git commit docs/<SUBDIR>/<FILENAME>.md mkdocs.yaml -m "MESSAGE" -~ git push origin <BRANCHNAME> -``` - -1. Run checks locally and fix the issues. Otherwise the pipeline will fail. - * [Check links](README.md#check-links) (There might be broken links which can only be solved - with ongoing transfer of content.) - * [Check pages structure](README.md#check-pages-structure) - * [Markdown Linter](README.md#markdown-linter) -1. Create - [merge request](https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/merge_requests) - against `preview` branch - -### Review Content - -The following steps are optional in a sense, that the first goal of the hackathon is to transfer all -old pages into new structure. If this is done, the content of the files need to be reviewed: - - * Remove outdated information - * Update content - * Apply [writing style](README.md#writing-style) - * Replace or remove (leftover) html constructs in markdown source file - * Add ticks for code blocks and command if necessary - * Fix internal links (mark as todo if necessary) - * Review and update, remove outdated content diff --git a/twiki2md/root/Applications/Electrodynamics.md b/twiki2md/root/Applications/Electrodynamics.md deleted file mode 100644 index c7b99613df2abfbad8b708d27cae01925a382925..0000000000000000000000000000000000000000 --- a/twiki2md/root/Applications/Electrodynamics.md +++ /dev/null @@ -1,17 +0,0 @@ -# Electromagnetic Field Simulation - -The following applications are installed at ZIH: - -| | | | -|----------|----------|------------| -| | **Mars** | **Deimos** | -| **HFSS** | | 11.0.2 | - -## HFSS - -[HFSS](http://www.ansoft.com/products/hf/hfss/) is the industry-standard -simulation tool for 3D full-wave electromagnetic field simulation. HFSS -provides E- and H-fields, currents, S-parameters and near and far -radiated field results. - --- Main.mark - 2010-01-06 diff --git a/twiki2md/root/Applications/SoftwareModulesList.md b/twiki2md/root/Applications/SoftwareModulesList.md deleted file mode 100644 index 70feb25fc836e0cee838889ef15cc9ca284d9cfa..0000000000000000000000000000000000000000 --- a/twiki2md/root/Applications/SoftwareModulesList.md +++ /dev/null @@ -1,962 +0,0 @@ -### SCS5 Environment - -<span class="twiki-macro TABLE">headerrows= 1"</span> <span -class="twiki-macro EDITTABLE" -format="| text, 30, Software | text, 40, Kategorie | text, 30, Letzte Änderung | text, 30, SGI-UV | text, 30, Taurus | " -changerows="on"></span> - -| Software | Category | Last change | Venus | Taurus | -|:--------------------------|:----------|:------------|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| ABAQUS | cae | 2020-04-30 | \- | 2019 | -| ABINIT | chem | 2019-07-12 | \- | 8.6.3\<br />8.10.3 | -| ACE | lib | 2018-11-22 | \- | 6.5.1 | -| ACTC | lib | 2018-11-22 | \- | 1.1 | -| AFNI | bio | 2018-11-22 | \- | 20180521 | -| AMDLibM | perf | 2019-08-20 | \- | 3.4.0 | -| AMDuProf | perf | 2020-11-26 | \- | 3.3.462\<br />3.1.35 | -| ANSA | cae | 2021-04-08 | \- | 20.1.4 | -| ANSYS | tools | 2020-10-23 | \- | 2020R2\<br />19.5\<br />19.2 | -| ANTLR | tools | 2020-09-29 | \- | 2.7.7 | -| ANTs | data | 2020-10-06 | \- | 2.3.4 | -| APR | tools | 2019-11-08 | \- | 1.7.0\<br />1.6.3 | -| APR-util | tools | 2019-11-08 | \- | 1.6.1 | -| ASE | chem | 2020-09-29 | \- | 3.19.0\<br />3.18.1\<br />3.16.2 | -| ATK | vis | 2020-09-02 | \- | 2.34.1\<br />2.28.1 | -| Advisor | perf | 2018-11-22 | \- | 2018 | -| Anaconda3 | lang | 2019-10-17 | \- | 2019.03 | -| AnsysEM | phys | 2018-11-22 | \- | 19.0 | -| Arrow | data | 2020-12-01 | \- | 0.16.0\<br />0.14.1 | -| AtomPAW | chem | 2019-09-11 | \- | 4.1.0.6 | -| Autoconf | devel | 2021-02-17 | \- | 2.69 | -| Automake | devel | 2021-02-17 | \- | 1.16.2\<br />1.16.1\<br />1.15.1\<br />1.15 | -| Autotools | devel | 2021-02-17 | \- | 20200321\<br />20180311\<br />20170619\<br />20150215 | -| Bazel | devel | 2021-03-19 | \- | 3.7.2\<br />3.7.1\<br />3.4.1\<br />0.29.1\<br />0.26.1\<br />0.20.0\<br />0.16.0\<br />0.12.0 | -| BigDataFrameworkConfigure | devel | 2019-09-16 | \- | 0.0.2\<br />0.0.1 | -| Bison | lang | 2020-12-11 | \- | 3.7.1\<br />3.5.3\<br />3.3.2\<br />3.0.5\<br />3.0.4\<br />2.7 | -| Blitz++ | lib | 2019-04-09 | \- | 0.10 | -| Boost | devel | 2021-04-09 | \- | 1.74.0\<br />1.72.0\<br />1.71.0\<br />1.70.0\<br />1.69.0\<br />1.68.0\<br />1.67.0\<br />1.66.0\<br />1.61.0\<br />1.55.0 | -| Boost.Python | lib | 2019-02-26 | \- | 1.66.0 | -| CDO | data | 2020-10-06 | \- | 1.9.8\<br />1.9.4 | -| CFITSIO | lib | 2019-06-03 | \- | 3.45 | -| CGAL | numlib | 2020-08-20 | \- | 4.14.3\<br />4.14.1\<br />4.11.1 | -| CGNS | cae | 2020-10-09 | \- | 3.3.1 | -| CMake | devel | 2021-02-17 | \- | 3.9.5\<br />3.9.1\<br />3.8.0\<br />3.18.4\<br />3.16.4\<br />3.15.3\<br />3.14.5\<br />3.13.3\<br />3.12.1\<br />3.11.4\<br />3.10.2\<br />3.10.1\<br />3.10.0 | -| COMSOL | phys | 2021-02-10 | \- | 5.6\<br />5.5\<br />5.4 | -| CP2K | chem | 2019-12-12 | \- | 6.1\<br />5.1 | -| CUDA | system | 2021-02-17 | \- | 9.2.88\<br />9.1.85\<br />11.1.1\<br />11.0.2\<br />10.1.243\<br />10.0.130 | -| CUDAcore | system | 2021-02-17 | \- | 11.1.1\<br />11.0.2 | -| Check | lib | 2021-02-17 | \- | 0.15.2 | -| Clang | compiler | 2020-12-09 | \- | 9.0.1\<br />5.0.0 | -| ClustalW2 | bio | 2018-11-22 | \- | 2.1 | -| CubeGUI | perf | 2021-01-29 | \- | 4.4.4\<br />4.4 | -| CubeLib | perf | 2020-10-12 | \- | 4.4.4\<br />4.4 | -| CubeW | perf | 2018-11-22 | \- | 4.4 | -| CubeWriter | perf | 2020-10-12 | \- | 4.4.3 | -| Cython | lang | 2018-11-22 | \- | 0.28.5\<br />0.28.2 | -| DAMASK | phys | 2020-12-10 | \- | 3.0.0\<br />2.0.3-2992\<br />2.0.3 | -| DASH | lib | 2018-11-22 | \- | dash | -| DB | tools | 2021-02-17 | \- | 18.1.40 | -| DBus | devel | 2020-08-11 | \- | 1.13.8\<br />1.13.6\<br />1.13.12 | -| DFTB+ | phys | 2021-04-12 | \- | 19.1\<br />18.2\<br />18.1 | -| DMTCP | tools | 2018-12-06 | \- | 2.5.2\<br />2.5.1 | -| DOLFIN | math | 2020-12-21 | \- | 2019.1.0\<br />2018.1.0.post1\<br />2017.2.0 | -| Delft3D | geo | 2020-01-29 | \- | 6.03 | -| Devel-NYTProf | perf | 2019-08-30 | \- | 6.06 | -| Doxygen | devel | 2021-03-03 | \- | 1.8.20\<br />1.8.17\<br />1.8.16\<br />1.8.15\<br />1.8.14\<br />1.8.13 | -| Dyninst | tools | 2019-01-21 | \- | 9.3.2 | -| ELPA | math | 2021-01-05 | \- | 2019.11.001\<br />2018.11.001\<br />2016.11.001.pre | -| ELSI | math | 2020-02-27 | \- | 2.5.0 | -| EMBOSS | bio | 2019-02-11 | \- | 6.6.0 | -| ESPResSo | phys | 2019-11-19 | \- | 4.1.1 | -| ETSF_IO | lib | 2020-10-15 | \- | 1.0.4 | -| EasyBuild | tools | 2021-03-08 | \- | 4.3.3\<br />4.3.2\<br />4.3.1\<br />4.3.0\<br />4.2.2\<br />4.2.0\<br />4.1.2\<br />4.1.1\<br />4.1.0\<br />4.0.1\<br />3.9.4\<br />3.9.3\<br />3.9.2\<br />3.9.1\<br />3.8.1\<br />3.8.0\<br />3.7.1\<br />3.7.0\<br />3.6.2\<br />3.6.1 | -| Eigen | math | 2021-02-17 | \- | 3.3.8\<br />3.3.7\<br />3.3.4 | -| Emacs | tools | 2020-02-20 | \- | 25.3 | -| ErlangOTP | lang | 2019-03-13 | \- | 21.3-no | -| FFC | math | 2019-02-26 | \- | 2018.1.0 | -| FFTW | numlib | 2021-02-17 | \- | 3.3.8\<br />3.3.7\<br />3.3.4 | -| FFmpeg | vis | 2020-08-11 | \- | 4.2.2\<br />4.2.1\<br />4.1.3\<br />4.1\<br />3.4.2 | -| FIAT | math | 2019-02-26 | \- | 2018.1.0 | -| FLANN | lib | 2019-08-13 | \- | 1.8.4 | -| FLTK | vis | 2019-11-08 | \- | 1.3.5\<br />1.3.4 | -| FSL | bio | 2020-10-30 | \- | 6.0.2\<br />5.0.11 | -| Flink | devel | 2019-09-16 | \- | 1.9.0\<br />1.8.1 | -| FoX | lib | 2018-11-22 | \- | 4.1.2 | -| FreeSurfer | bio | 2020-07-22 | \- | 7.1.1\<br />6.0.0\<br />5.3.0\<br />5.1.0 | -| FriBidi | vis | 2020-08-11 | \- | 1.0.9\<br />1.0.5\<br />1.0.1 | -| GAMS | math | 2018-11-28 | \- | 25.1.3 | -| GCC | compiler | 2020-12-11 | \- | 9.3.0\<br />9.1.0-2.32\<br />8.3.0\<br />8.2.0-2.31.1\<br />8.2.0-2.30\<br />7.3.0-2.30\<br />6.4.0-2.28\<br />10.2.0 | -| GCCcore | compiler | 2020-12-11 | \- | 9.3.0\<br />9.1.0\<br />8.3.0\<br />8.2.0\<br />7.3.0\<br />6.4.0\<br />6.3.0\<br />10.2.0 | -| GCL | compiler | 2018-11-22 | \- | 2.6.12 | -| GDAL | data | 2020-10-06 | \- | 3.0.2\<br />3.0.0\<br />2.2.3 | -| GDB | debugger | 2020-08-11 | \- | 9.1\<br />8.1 | -| GDRCopy | lib | 2021-02-17 | \- | 2.1 | -| GEOS | math | 2020-10-06 | \- | 3.8.0\<br />3.7.2\<br />3.6.2 | -| GL2PS | vis | 2019-11-08 | \- | 1.4.0 | -| GLM | lib | 2018-12-11 | \- | 0.9.9.0 | -| GLPK | tools | 2020-08-28 | \- | 4.65 | -| GLib | vis | 2020-08-11 | \- | 2.64.1\<br />2.62.0\<br />2.60.1\<br />2.54.3 | -| GLibmm | vis | 2020-09-24 | \- | 2.49.7 | -| GMP | math | 2021-02-17 | \- | 6.2.0\<br />6.1.2 | -| GObject-Introspection | devel | 2020-09-02 | \- | 1.63.1\<br />1.60.1\<br />1.54.1 | -| GPAW | chem | 2020-01-15 | \- | 19.8.1 | -| GPAW-setups | chem | 2020-01-15 | \- | 0.9.20000 | -| GPI2 | base | 2019-05-29 | \- | next-27-05-19\<br />1.3.0 | -| GROMACS | bio | 2020-10-12 | \- | 2020\<br />2019.4\<br />2018.2 | -| GSL | numlib | 2020-08-11 | \- | 2.6\<br />2.5\<br />2.4 | -| GTK+ | vis | 2020-01-13 | \- | 2.24.32 | -| Gdk-Pixbuf | vis | 2020-09-02 | \- | 2.38.2\<br />2.36.12\<br />2.36.11 | -| Ghostscript | tools | 2020-08-28 | \- | 9.52\<br />9.50\<br />9.27\<br />9.23\<br />9.22 | -| GitPython | lib | 2020-04-16 | \- | 3.1.1\<br />3.0.3 | -| GlobalArrays | lib | 2020-10-12 | \- | 5.7 | -| Go | compiler | 2019-06-26 | \- | 1.12 | -| GraphicsMagick | vis | 2019-11-08 | \- | 1.3.33\<br />1.3.31\<br />1.3.28 | -| Guile | lang | 2020-08-11 | \- | 1.8.8 | -| Gurobi | math | 2020-12-01 | \- | 9.1.0\<br />9.0.1\<br />8.0.1 | -| HDF | data | 2018-11-22 | \- | 4.2.13 | -| HDF5 | data | 2021-03-17 | \- | 1.10.7\<br />1.10.6\<br />1.10.5\<br />1.10.2\<br />1.10.1 | -| HDFView | vis | 2019-02-20 | \- | 2.14 | -| Hadoop | devel | 2019-08-29 | \- | 2.7.7 | -| HarfBuzz | vis | 2020-09-02 | \- | 2.6.4\<br />2.4.0\<br />2.2.0\<br />1.7.5 | -| Horovod | tools | 2020-01-24 | \- | 0.18.2 | -| Hyperopt | lib | 2020-02-19 | \- | 0.2.2\<br />0.1.1 | -| Hypre | numlib | 2020-12-08 | \- | 2.18.2\<br />2.14.0 | -| ICU | lib | 2021-03-17 | \- | 67.1\<br />66.1\<br />64.2\<br />61.1\<br />56.1 | -| IPython | tools | 2019-11-19 | \- | 7.7.0\<br />6.4.0\<br />6.3.1 | -| ImageMagick | vis | 2020-08-28 | \- | 7.0.9-5\<br />7.0.8-46\<br />7.0.8-11\<br />7.0.7-39\<br />7.0.10-1 | -| Inspector | tools | 2019-01-22 | \- | 2019\<br />2018 | -| Ipopt | lib | 2018-11-22 | \- | 3.12.11 | -| JUnit | devel | 2019-02-13 | \- | 4.12 | -| JasPer | vis | 2020-08-11 | \- | 2.0.14\<br />1.900.1 | -| Java | lang | 2020-12-02 | \- | 14.0.2\<br />11.0.2\<br />1.8.0-162\<br />1.8.0-161 | -| JsonCpp | lib | 2021-03-17 | \- | 1.9.4\<br />1.9.3 | -| Julia | lang | 2020-07-14 | \- | 1.4.2\<br />1.1.1\<br />1.0.2 | -| Keras | math | 2020-03-20 | \- | 2.3.1\<br />2.2.4\<br />2.2.0 | -| LAME | data | 2020-08-11 | \- | 3.100 | -| LAMMPS | chem | 2020-08-11 | \- | 7Aug19\<br />3Mar2020\<br />20180316\<br />12Dec2018 | -| LLVM | compiler | 2020-09-25 | \- | 9.0.1\<br />9.0.0\<br />8.0.1\<br />7.0.1\<br />6.0.0\<br />5.0.1 | -| LMDB | lib | 2021-03-17 | \- | 0.9.24 | -| LS-DYNA | cae | 2020-12-11 | \- | DEV-81069\<br />12.0.0\<br />11.1.0\<br />11.0.0\<br />10.1.0 | -| LS-Opt | cae | 2019-08-20 | \- | 6.0.0\<br />5.2.1 | -| LS-PrePost | cae | 2019-05-03 | \- | 4.6\<br />4.5\<br />4.3 | -| LibTIFF | lib | 2021-02-17 | \- | 4.1.0\<br />4.0.9\<br />4.0.10 | -| LibUUID | lib | 2019-11-08 | \- | 1.0.3 | -| Libint | chem | 2019-10-14 | \- | 1.1.6 | -| LittleCMS | vis | 2020-08-28 | \- | 2.9 | -| M4 | devel | 2020-12-11 | \- | 1.4.18\<br />1.4.17 | -| MATIO | lib | 2018-12-11 | \- | 1.5.12 | -| MATLAB | math | 2021-03-22 | \- | 2021a\<br />2020a\<br />2019b\<br />2018b\<br />2018a\<br />2017a | -| MDAnalysis | phys | 2020-04-21 | \- | 0.20.1 | -| METIS | math | 2020-08-20 | \- | 5.1.0 | -| MPFR | math | 2020-08-20 | \- | 4.0.2\<br />4.0.1 | -| MUMPS | math | 2020-12-08 | \- | 5.2.1\<br />5.1.2 | -| MUST | perf | 2019-01-25 | \- | 1.6.0-rc3 | -| Mako | devel | 2020-08-11 | \- | 1.1.2\<br />1.1.0\<br />1.0.8\<br />1.0.7 | -| Mathematica | math | 2018-11-22 | \- | 11.3.0\<br />11.2.0 | -| Maven | devel | 2020-04-29 | \- | 3.6.3 | -| Maxima | math | 2018-11-22 | \- | 5.42.1 | -| Mercurial | tools | 2018-11-22 | \- | 4.6.1 | -| Mesa | vis | 2020-08-11 | \- | 20.0.2\<br />19.1.7\<br />19.0.1\<br />18.1.1\<br />17.3.6 | -| Meson | tools | 2021-02-17 | \- | 0.55.3\<br />0.53.2\<br />0.51.2\<br />0.50.0 | -| Mesquite | math | 2020-02-12 | \- | 2.3.0 | -| Miniconda2 | lang | 2018-11-22 | \- | 4.5.11 | -| Miniconda3 | lang | 2019-10-17 | \- | 4.5.4 | -| MongoDB | data | 2019-07-29 | \- | 4.0.3 | -| NAMD | chem | 2018-11-22 | \- | 2.12 | -| NASM | lang | 2021-02-17 | \- | 2.15.05\<br />2.14.02\<br />2.13.03\<br />2.13.01 | -| NCCL | lib | 2021-03-17 | \- | 2.8.3\<br />2.4.8\<br />2.4.2\<br />2.3.7 | -| NCO | tools | 2020-09-29 | \- | 4.9.3 | -| NFFT | lib | 2019-08-09 | \- | 3.5.1 | -| NLTK | data | 2018-12-05 | \- | 3.4 | -| NLopt | numlib | 2020-08-28 | \- | 2.6.1\<br />2.4.2 | -| NSPR | lib | 2020-08-11 | \- | 4.25\<br />4.21\<br />4.20 | -| NSS | lib | 2020-08-11 | \- | 3.51\<br />3.45\<br />3.42.1\<br />3.39 | -| NWChem | chem | 2018-11-22 | \- | 6.8.revision47\<br />6.6.revision27746 | -| Nektar++ | math | 2020-02-19 | \- | 5.0.0 | -| NetLogo | math | 2018-11-22 | \- | 6.0.4-64 | -| Ninja | tools | 2021-02-17 | \- | 1.9.0\<br />1.10.1\<br />1.10.0 | -| OPARI2 | perf | 2020-10-12 | \- | 2.0.5\<br />2.0.3 | -| ORCA | chem | 2020-03-04 | \- | 4.2.1\<br />4.1.1 | -| OTF2 | perf | 2020-10-12 | \- | 2.2\<br />2.1.1 | -| Octave | math | 2019-11-08 | \- | 5.1.0 | -| Octopus | chem | 2020-10-15 | \- | 8.4\<br />10.1 | -| OpenBLAS | numlib | 2021-02-17 | \- | 0.3.9\<br />0.3.7\<br />0.3.5\<br />0.3.12\<br />0.3.1\<br />0.2.20 | -| OpenBabel | chem | 2018-11-22 | \- | 2.4.1 | -| OpenCV | vis | 2020-01-13 | \- | 4.0.1\<br />3.4.1 | -| OpenFOAM | cae | 2020-09-23 | \- | v2006\<br />v1912\<br />v1806\<br />8\<br />7\<br />6\<br />5.0\<br />4.1\<br />2.3.1 | -| OpenFOAM-Extend | cae | 2020-02-12 | \- | 4.0 | -| OpenMPI | mpi | 2021-02-17 | \- | 4.0.5\<br />4.0.4\<br />4.0.3\<br />4.0.1\<br />3.1.6\<br />3.1.4\<br />3.1.3\<br />3.1.2\<br />3.1.1\<br />2.1.5\<br />2.1.2\<br />1.10.7 | -| OpenMX | phys | 2020-12-08 | \- | 3.9.2 | -| OpenMolcas | chem | 2020-10-12 | \- | 19.11\<br />18.09 | -| OpenPGM | system | 2019-11-05 | \- | 5.2.122 | -| OpenSSL | system | 2019-11-08 | \- | 1.1.1b\<br />1.0.2l | -| PAPI | perf | 2020-10-12 | \- | 6.0.0\<br />5.7.0\<br />5.6.0 | -| PCL | vis | 2019-08-13 | \- | 1.9.1 | -| PCRE | devel | 2021-03-17 | \- | 8.44\<br />8.43\<br />8.41 | -| PCRE2 | devel | 2020-08-11 | \- | 10.34\<br />10.33\<br />10.32 | -| PDFCrop | tools | 2018-11-22 | \- | 0.4b | -| PDT | perf | 2020-10-12 | \- | 3.25.1\<br />3.25 | -| PETSc | numlib | 2020-12-08 | \- | 3.9.4\<br />3.9.3\<br />3.8.3\<br />3.7.7\<br />3.13.3\<br />3.12.4\<br />3.11.0\<br />3.10.5 | -| PFFT | numlib | 2020-10-15 | \- | 1.0.8 | -| PGI | compiler | 2020-02-07 | \- | 19.4\<br />19.10\<br />18.7\<br />18.4\<br />18.10\<br />17.7\<br />17.10 | -| PLUMED | chem | 2020-08-11 | \- | 2.6.0\<br />2.5.1\<br />2.4.0 | -| PLY | lib | 2019-02-26 | \- | 3.11 | -| PMIx | lib | 2021-02-24 | \- | 3.1.5\<br />3.1.1 | -| PROJ | lib | 2020-10-06 | \- | 6.2.1\<br />6.0.0\<br />5.0.0 | -| Pandoc | tools | 2019-10-14 | \- | 2.5 | -| Pango | vis | 2020-09-02 | \- | 1.44.7\<br />1.43.0\<br />1.42.4\<br />1.41.1 | -| ParMETIS | math | 2020-02-12 | \- | 4.0.3 | -| ParMGridGen | math | 2020-02-12 | \- | 1.0 | -| ParaView | vis | 2020-11-12 | \- | 5.9.0-RC1\<br />5.8.0\<br />5.7.0\<br />5.6.2\<br />5.5.2\<br />5.4.1 | -| Perl | lang | 2021-02-17 | \- | 5.32.0\<br />5.30.2\<br />5.30.0\<br />5.28.1\<br />5.28.0\<br />5.26.1 | -| Pillow | vis | 2021-02-17 | \- | 8.0.1\<br />7.0.0\<br />6.2.1\<br />5.0.0 | -| Pillow-SIMD | vis | 2018-11-22 | \- | 5.0.0 | -| PnetCDF | data | 2019-03-06 | \- | 1.9.0 | -| PyQt5 | vis | 2018-11-22 | \- | 5.10.1 | -| PyTorch | devel | 2021-01-04 | \- | 1.6.0\<br />0.3.1 | -| PyYAML | lib | 2020-04-16 | \- | 5.1.2\<br />5.1\<br />3.13\<br />3.12 | -| Python | lang | 2021-02-17 | \- | 3.8.6\<br />3.8.2\<br />3.7.4\<br />3.7.2\<br />3.6.6\<br />3.6.4\<br />2.7.18\<br />2.7.16\<br />2.7.15\<br />2.7.14 | -| Qhull | math | 2019-11-08 | \- | 2019.1\<br />2015.2 | -| Qt | devel | 2018-11-22 | \- | 4.8.7 | -| Qt5 | devel | 2020-08-11 | \- | 5.9.3\<br />5.14.1\<br />5.13.1\<br />5.12.3\<br />5.10.1 | -| QuantumESPRESSO | chem | 2021-01-05 | \- | 6.6\<br />6.5\<br />6.4.1\<br />6.3\<br />6.2 | -| Qwt | lib | 2020-09-24 | \- | 6.1.4 | -| R | lang | 2020-10-06 | \- | 4.0.0\<br />3.6.2\<br />3.6.0\<br />3.5.1\<br />3.4.4 | -| RDFlib | lib | 2020-09-29 | \- | 4.2.2 | -| RELION | bio | 2019-02-27 | \- | 3.0\<br />2.1 | -| ROOT | data | 2019-06-03 | \- | 6.14.06 | -| Ruby | lang | 2019-11-14 | \- | 2.6.3\<br />2.6.1 | -| SCOTCH | math | 2020-12-08 | \- | 6.0.9\<br />6.0.6\<br />6.0.5\<br />6.0.4\<br />5.1.12b | -| SCons | devel | 2020-01-24 | \- | 3.1.1\<br />3.0.5\<br />3.0.1 | -| SHARC | chem | 2019-01-07 | \- | 2.0 | -| SIONlib | lib | 2020-10-12 | \- | 1.7.6\<br />1.7.4\<br />1.7.2 | -| SIP | lang | 2018-11-22 | \- | 4.19.8\<br />4.19.12 | -| SLEPc | numlib | 2020-02-27 | \- | 3.9.2\<br />3.12.2 | -| SPM | math | 2018-11-22 | \- | 12-r7219 | -| SQLite | devel | 2021-02-17 | \- | 3.33.0\<br />3.31.1\<br />3.29.0\<br />3.27.2\<br />3.26.0\<br />3.24.0\<br />3.21.0\<br />3.20.1 | -| STAR-CCM+ | cae | 2021-03-19 | \- | 15.06.008\<br />15.04.010-R8\<br />15.02.007\<br />14.04.011\<br />14.02.012\<br />13.06.012-R8\<br />13.04.011\<br />13.02.013-R8 | -| SUNDIALS | math | 2018-11-22 | \- | 2.7.0 | -| SWASH | phys | 2020-03-18 | \- | 6.01\<br />5.01 | -| SWIG | devel | 2020-10-06 | \- | 4.0.1\<br />3.0.12 | -| ScaFaCoS | math | 2020-08-11 | \- | 1.0.1 | -| ScaLAPACK | numlib | 2021-02-17 | \- | 2.1.0\<br />2.0.2 | -| Scalasca | perf | 2021-02-02 | \- | 2.5\<br />2.4 | -| SciPy-bundle | lang | 2021-02-17 | \- | 2020.11\<br />2020.03\<br />2019.10\<br />2019.03 | -| Score-P | perf | 2020-10-19 | \- | 6.0\<br />4.0 | -| Serf | tools | 2019-11-08 | \- | 1.3.9 | -| Siesta | phys | 2020-02-27 | \- | 4.1-b4\<br />4.1-b3\<br />4.1 | -| Six | lib | 2018-11-22 | \- | 1.11.0 | -| Spark | devel | 2020-10-22 | \- | 3.0.1\<br />2.4.4\<br />2.4.3 | -| Subversion | tools | 2019-11-08 | \- | 1.9.7\<br />1.12.0 | -| SuiteSparse | numlib | 2020-09-02 | \- | 5.7.1\<br />5.6.0\<br />5.4.0\<br />5.1.2 | -| SuperLU | numlib | 2019-01-15 | \- | 5.2.1 | -| SuperLU_DIST | numlib | 2019-01-16 | \- | 6.1.0 | -| SuperLU_MT | numlib | 2019-01-16 | \- | 3.1 | -| Szip | tools | 2021-03-03 | \- | 2.1.1 | -| Tcl | lang | 2021-02-17 | \- | 8.6.9\<br />8.6.8\<br />8.6.7\<br />8.6.10 | -| TensorFlow | lib | 2021-03-15 | \- | 2.4.1\<br />2.3.1\<br />2.1.0\<br />2.0.0\<br />1.8.0\<br />1.15.0\<br />1.10.0 | -| Tk | vis | 2021-02-17 | \- | 8.6.9\<br />8.6.8\<br />8.6.10 | -| Tkinter | lang | 2021-02-17 | \- | 3.8.6\<br />3.8.2\<br />3.7.4\<br />3.7.2\<br />3.6.6\<br />3.6.4\<br />2.7.15\<br />2.7.14 | -| TotalView | debugger | 2020-09-24 | \- | 8.14.1-8 | -| Trilinos | numlib | 2019-02-26 | \- | 12.12.1 | -| UCX | lib | 2021-02-24 | \- | 1.9.0\<br />1.8.0\<br />1.5.1 | -| UDUNITS | phys | 2020-08-28 | \- | 2.2.26 | -| UFL | cae | 2019-02-26 | \- | 2018.1.0 | -| UnZip | tools | 2021-02-17 | \- | 6.0 | -| VASP | phys | 2020-05-19 | \- | 5.4.4 | -| VMD | vis | 2018-11-22 | \- | 1.9.3 | -| VSEARCH | bio | 2018-11-22 | \- | 2.8.4 | -| VTK | vis | 2020-10-06 | \- | 8.2.0\<br />8.1.1\<br />8.1.0\<br />5.10.1 | -| VTune | tools | 2020-04-28 | \- | 2020\<br />2019\<br />2018 | -| Valgrind | debugger | 2019-11-05 | \- | 3.14.0\<br />3.13.0 | -| Vampir | tools | 2021-01-13 | \- | unstable\<br />9.9.0\<br />9.8.0\<br />9.7.1\<br />9.7.0\<br />9.6.1\<br />9.5.0\<br />9.11\<br />9.10.0 | -| Voro++ | math | 2020-08-11 | \- | 0.4.6 | -| WRF | geo | 2018-12-12 | \- | 3.8.1 | -| Wannier90 | chem | 2020-12-14 | \- | 2.1.0\<br />2.0.1.1\<br />1.2 | -| X11 | vis | 2021-02-17 | \- | 20201008\<br />20200222\<br />20190717\<br />20190311\<br />20180604\<br />20180131 | -| XML-Parser | data | 2018-11-22 | \- | 2.44-01 | -| XZ | tools | 2021-02-17 | \- | 5.2.5\<br />5.2.4\<br />5.2.3\<br />5.2.2 | -| YAXT | tools | 2020-10-06 | \- | 0.6.2\<br />0.6.0 | -| Yasm | lang | 2020-08-11 | \- | 1.3.0 | -| ZeroMQ | devel | 2019-11-05 | \- | 4.3.2\<br />4.2.5 | -| Zip | tools | 2021-03-17 | \- | 3.0 | -| ace | lib | 2018-11-22 | \- | 6.5.0 | -| ant | devel | 2020-09-02 | \- | 1.10.7\<br />1.10.1 | -| archspec | tools | 2020-08-11 | \- | 0.1.0 | -| arpack-ng | numlib | 2019-11-08 | \- | 3.7.0\<br />3.6.1\<br />3.5.0 | -| asciidoc | base | 2018-11-22 | \- | 8.6.9 | -| at-spi2-atk | vis | 2020-09-02 | \- | 2.34.1 | -| at-spi2-core | vis | 2020-09-02 | \- | 2.34.0 | -| auto_ml | lang | 2019-10-29 | \- | 2.9.9 | -| basemap | vis | 2019-04-02 | \- | 1.0.7 | -| binutils | tools | 2020-12-11 | \- | 2.35\<br />2.34\<br />2.32\<br />2.31.1\<br />2.30\<br />2.28\<br />2.27\<br />2.26 | -| bzip2 | tools | 2021-02-17 | \- | 1.0.8\<br />1.0.6 | -| cURL | tools | 2021-02-17 | \- | 7.72.0\<br />7.69.1\<br />7.66.0\<br />7.63.0\<br />7.60.0\<br />7.58.0 | -| cairo | vis | 2020-08-28 | \- | 1.16.0\<br />1.14.12 | -| cftime | data | 2019-07-17 | \- | 1.0.1 | -| chrpath | tools | 2018-11-22 | \- | 0.16 | -| ctags | devel | 2018-11-22 | \- | 5.8 | -| cuDNN | numlib | 2021-03-17 | \- | 8.0.4.30\<br />7.6.4.38\<br />7.4.2.24\<br />7.1.4.18\<br />7.1.4\<br />7.0.5 | -| ddt | tools | 2021-04-12 | \- | 20.2.1\<br />20.0.1\<br />18.2.2 | -| dftd3-lib | chem | 2021-02-17 | \- | 0.9 | -| dill | data | 2019-10-29 | \- | 0.3.1.1\<br />0.3.1 | -| double-conversion | lib | 2021-03-17 | \- | 3.1.5\<br />3.1.4 | -| ecCodes | tools | 2020-10-06 | \- | 2.8.2\<br />2.17.0 | -| expat | tools | 2021-02-17 | \- | 2.2.9\<br />2.2.7\<br />2.2.6\<br />2.2.5 | -| flair | vis | 2019-01-25 | \- | 2.3-0 | -| flair-geoviewer | vis | 2019-01-25 | \- | 2.3-0 | -| flatbuffers | devel | 2021-03-17 | \- | 1.12.0 | -| flatbuffers-python | devel | 2021-03-19 | \- | 1.12 | -| flex | lang | 2020-12-11 | \- | 2.6.4\<br />2.6.3\<br />2.6.0\<br />2.5.39 | -| fontconfig | vis | 2021-02-17 | \- | 2.13.92\<br />2.13.1\<br />2.13.0\<br />2.12.6 | -| foss | toolchain | 2021-02-17 | \- | 2020b\<br />2020a\<br />2019b\<br />2019a\<br />2018b\<br />2018a | -| fosscuda | toolchain | 2021-02-17 | \- | 2020b\<br />2020a\<br />2019b\<br />2019a\<br />2018b | -| freeglut | lib | 2019-11-08 | \- | 3.0.0 | -| freetype | vis | 2021-02-17 | \- | 2.9.1\<br />2.9\<br />2.10.3\<br />2.10.1 | -| future | lib | 2018-11-22 | \- | 0.16.0 | -| gc | lib | 2020-08-11 | \- | 7.6.12\<br />7.6.10\<br />7.6.0 | -| gcccuda | toolchain | 2021-02-17 | \- | 2020b\<br />2020a\<br />2019b\<br />2019a\<br />2018b | -| gettext | tools | 2021-02-17 | \- | 0.21\<br />0.20.1\<br />0.19.8.1\<br />0.19.8 | -| gflags | devel | 2020-09-29 | \- | 2.2.2 | -| giflib | lib | 2021-03-17 | \- | 5.2.1 | -| git | tools | 2021-03-17 | \- | 2.28.0\<br />2.23.0\<br />2.21.0\<br />2.19.1\<br />2.18.0\<br />2.16.1 | -| git-cola | tools | 2018-11-22 | \- | 3.2 | -| git-lfs | tools | 2019-06-26 | \- | 2.7.2 | -| glew | devel | 2020-09-23 | \- | 2.1.0 | -| glog | devel | 2020-09-29 | \- | 0.4.0 | -| gmsh | vis | 2019-11-26 | \- | 4.4.1 | -| gnuplot | vis | 2019-11-08 | \- | 5.2.6\<br />5.2.5\<br />5.2.4\<br />5.2.2 | -| golf | toolchain | 2018-11-22 | \- | 2018a | -| gomkl | toolchain | 2019-07-17 | \- | 2019a | -| gompi | toolchain | 2021-02-17 | \- | 2020b\<br />2020a\<br />2019b\<br />2019a\<br />2018b\<br />2018a | -| gompic | toolchain | 2021-02-17 | \- | 2020b\<br />2020a\<br />2019b\<br />2019a\<br />2018b | -| gperf | devel | 2021-02-17 | \- | 3.1 | -| gperftools | tools | 2018-11-22 | \- | 2.7 | -| grib_api | data | 2018-11-22 | \- | 1.27.0 | -| gzip | tools | 2020-08-11 | \- | 1.9\<br />1.8\<br />1.10 | -| h5py | data | 2020-08-11 | \- | 2.9.0\<br />2.8.0\<br />2.7.1\<br />2.10.0 | -| help2man | tools | 2020-12-11 | \- | 1.47.8\<br />1.47.6\<br />1.47.4\<br />1.47.16\<br />1.47.12\<br />1.47.10 | -| hwloc | system | 2021-02-17 | \- | 2.2.0\<br />2.0.3\<br />1.11.8\<br />1.11.12\<br />1.11.11\<br />1.11.10 | -| hypothesis | tools | 2021-02-17 | \- | 5.41.2\<br />4.44.2 | -| icc | compiler | 2019-10-11 | \- | 2019.1.144\<br />2019.0.117\<br />2018.3.222\<br />2018.1.163 | -| iccifort | toolchain | 2020-09-21 | \- | 2020.2.254\<br />2020.1.217\<br />2019.5.281\<br />2019.1.144\<br />2019.0.117\<br />2018.3.222\<br />2018.1.163 | -| ifort | compiler | 2019-10-11 | \- | 2019.1.144\<br />2019.0.117\<br />2018.3.222\<br />2018.1.163 | -| iimpi | toolchain | 2020-08-03 | \- | 2020a\<br />2019b\<br />2019a\<br />2018b\<br />2018a | -| imkl | numlib | 2021-01-05 | \- | 2020.1.217\<br />2019.5.281\<br />2019.1.144\<br />2018.3.222\<br />2018.1.163 | -| impi | mpi | 2020-08-13 | \- | 2019.7.217\<br />2018.5.288\<br />2018.4.274\<br />2018.3.222\<br />2018.1.163 | -| intel | toolchain | 2020-08-03 | \- | 2020a\<br />2019b\<br />2019a\<br />2018b\<br />2018a | -| intltool | devel | 2021-02-17 | \- | 0.51.0 | -| iomkl | toolchain | 2021-01-05 | \- | 2020a\<br />2019a\<br />2018a | -| iompi | toolchain | 2021-01-05 | \- | 2020a\<br />2019a\<br />2018a | -| itac | tools | 2018-11-22 | \- | 2018.3.022 | -| kim-api | chem | 2020-08-11 | \- | 2.1.3 | -| libGLU | vis | 2020-08-11 | \- | 9.0.1\<br />9.0.0 | -| libGridXC | chem | 2020-02-27 | \- | 0.8.5 | -| libPSML | data | 2020-02-27 | \- | 1.1.8 | -| libarchive | tools | 2021-02-17 | \- | 3.4.3 | -| libcerf | math | 2019-11-08 | \- | 1.7\<br />1.5\<br />1.11 | -| libcint | lib | 2019-01-09 | \- | 3.0.14 | -| libdap | lib | 2020-09-29 | \- | 3.20.6 | -| libdrm | lib | 2020-08-11 | \- | 2.4.99\<br />2.4.97\<br />2.4.92\<br />2.4.91\<br />2.4.100 | -| libelf | devel | 2020-12-11 | \- | 0.8.13 | -| libepoxy | lib | 2020-09-02 | \- | 1.5.4 | -| libevent | lib | 2021-02-24 | \- | 2.1.8\<br />2.1.12\<br />2.1.11 | -| libfabric | lib | 2021-02-17 | \- | 1.11.0 | -| libffi | lib | 2021-02-17 | \- | 3.3\<br />3.2.1 | -| libgd | lib | 2019-11-08 | \- | 2.2.5 | -| libgeotiff | lib | 2020-10-06 | \- | 1.5.1\<br />1.4.2 | -| libglvnd | lib | 2020-08-11 | \- | 1.2.0 | -| libharu | lib | 2018-11-22 | \- | 2.3.0 | -| libiconv | lib | 2021-03-03 | \- | 1.16 | -| libjpeg-turbo | lib | 2021-02-17 | \- | 2.0.5\<br />2.0.4\<br />2.0.3\<br />2.0.2\<br />2.0.0\<br />1.5.3\<br />1.5.2 | -| libmatheval | lib | 2020-08-11 | \- | 1.1.11 | -| libpciaccess | system | 2021-02-17 | \- | 0.16\<br />0.14 | -| libpng | lib | 2021-02-17 | \- | 1.6.37\<br />1.6.36\<br />1.6.34\<br />1.6.32 | -| libreadline | lib | 2021-02-17 | \- | 8.0\<br />7.0 | -| libsigc++ | devel | 2020-09-24 | \- | 2.10.1 | -| libsndfile | lib | 2020-08-28 | \- | 1.0.28 | -| libsodium | lib | 2019-11-05 | \- | 1.0.17\<br />1.0.16 | -| libssh2 | tools | 2018-11-22 | \- | 1.8.0 | -| libtirpc | lib | 2020-09-29 | \- | 1.2.6 | -| libtool | lib | 2021-02-17 | \- | 2.4.6 | -| libunistring | lib | 2020-08-11 | \- | 0.9.7\<br />0.9.10 | -| libunwind | lib | 2020-08-11 | \- | 1.3.1\<br />1.2.1 | -| libvdwxc | chem | 2020-01-15 | \- | 0.4.0 | -| libxc | chem | 2021-01-05 | \- | 4.3.4\<br />4.2.3\<br />3.0.1 | -| libxml++ | lib | 2020-09-24 | \- | 2.40.1 | -| libxml2 | lib | 2021-02-17 | \- | 2.9.9\<br />2.9.8\<br />2.9.7\<br />2.9.4\<br />2.9.10 | -| libxslt | lib | 2020-10-27 | \- | 1.1.34\<br />1.1.33\<br />1.1.32 | -| libxsmm | math | 2019-10-14 | \- | 1.8.3\<br />1.10 | -| libyaml | lib | 2020-04-16 | \- | 0.2.2\<br />0.2.1\<br />0.1.7 | -| likwid | tools | 2020-10-14 | \- | 5.0.1 | -| lo2s | perf | 2020-01-27 | \- | 1.3.0\<br />1.2.2\<br />1.1.1\<br />1.0.2\<br />1.0.1 | -| log4cxx | lang | 2020-02-18 | \- | 0.10.0 | -| lpsolve | math | 2018-11-22 | \- | 5.5.2.5 | -| lz4 | lib | 2020-08-11 | \- | 1.9.2\<br />1.9.1 | -| magma | math | 2021-01-04 | \- | 2.5.4\<br />2.3.0 | -| makedepend | devel | 2018-11-22 | \- | 1.0.5 | -| matplotlib | vis | 2021-02-17 | \- | 3.3.3\<br />3.2.1\<br />3.1.1\<br />3.0.3\<br />3.0.0\<br />2.1.2 | -| mkl-dnn | lib | 2018-11-22 | \- | 0.13 | -| molmod | math | 2020-08-11 | \- | 1.4.5 | -| motif | vis | 2018-11-22 | \- | 2.3.8 | -| ncdf4 | math | 2020-10-06 | \- | 1.17 | -| ncurses | devel | 2021-02-17 | \- | 6.2\<br />6.1\<br />6.0 | -| netCDF | data | 2021-03-03 | \- | 4.7.4\<br />4.7.1\<br />4.6.2\<br />4.6.1\<br />4.6.0 | -| netCDF-Fortran | data | 2021-03-04 | \- | 4.5.3\<br />4.5.2\<br />4.4.5\<br />4.4.4 | -| netcdf4-python | data | 2019-07-17 | \- | 1.4.3 | -| nettle | lib | 2020-01-24 | \- | 3.5.1\<br />3.4.1\<br />3.4 | -| nextstrain | bio | 2020-07-20 | \- | 2.0.0.post1 | -| nfft | math | 2018-11-28 | \- | 3.3.2DLR | -| nsync | devel | 2021-03-17 | \- | 1.24.0 | -| numactl | tools | 2021-02-17 | \- | 2.0.13\<br />2.0.12\<br />2.0.11 | -| numba | lang | 2020-09-25 | \- | 0.47.0 | -| numeca | cae | 2018-11-22 | \- | all | -| nvidia-nsight | tools | 2020-01-21 | \- | 2019.3.1 | -| p7zip | tools | 2018-11-22 | \- | 9.38.1 | -| parallel | tools | 2020-02-25 | \- | 20190922\<br />20190622\<br />20180822 | -| patchelf | tools | 2019-08-09 | \- | 0.9 | -| petsc4py | tools | 2019-02-26 | \- | 3.9.1 | -| pigz | tools | 2018-11-22 | \- | 2.4 | -| pixman | vis | 2020-08-28 | \- | 0.38.4\<br />0.38.0\<br />0.34.0 | -| pkg-config | devel | 2021-02-17 | \- | 0.29.2 | -| pkgconfig | devel | 2021-03-17 | \- | 1.5.1\<br />1.3.1 | -| pocl | lib | 2020-02-19 | \- | 1.4 | -| pompi | toolchain | 2018-11-22 | \- | 2018.04 | -| protobuf | devel | 2021-03-17 | \- | 3.6.1.2\<br />3.6.1\<br />3.14.0\<br />3.10.0 | -| protobuf-python | devel | 2021-03-17 | \- | 3.14.0\<br />3.10.0 | -| pybind11 | lib | 2021-02-17 | \- | 2.6.0\<br />2.4.3\<br />2.2.4 | -| pyscf | chem | 2019-11-13 | \- | 1.6.1\<br />1.6.0 | -| pytecplot | data | 2020-04-06 | \- | 1.0.0 | -| pytest | tools | 2019-02-26 | \- | 3.8.0 | -| qrupdate | numlib | 2019-11-08 | \- | 1.1.2 | -| re2c | tools | 2020-08-11 | \- | 1.3\<br />1.2.1 | -| rgdal | geo | 2019-11-27 | \- | 1.4-4 | -| rstudio | lang | 2020-02-18 | \- | 1.2.5001\<br />1.2.1335\<br />1.1.456 | -| scikit-learn | data | 2020-09-24 | \- | 0.21.3 | -| scorep_plugin_fileparser | perf | 2018-11-22 | \- | 1.3.1 | -| sf | lib | 2020-10-06 | \- | 0.9-5 | -| slepc4py | tools | 2019-02-26 | \- | 3.9.0 | -| snakemake | tools | 2020-04-16 | \- | 5.7.1\<br />5.14.0 | -| snappy | lib | 2021-03-17 | \- | 1.1.8\<br />1.1.7 | -| source-highlight | tools | 2018-11-22 | \- | 3.1.8 | -| spacy | lang | 2018-12-05 | \- | 2.0.18 | -| spglib | chem | 2019-12-12 | \- | 1.14.1 | -| tbb | lib | 2020-08-11 | \- | 2020.1\<br />2019-U4\<br />2018-U5 | -| tcsh | tools | 2018-11-22 | \- | 6.20.00 | -| tecplot360ex | vis | 2020-07-20 | \- | 2019r1 | -| texinfo | devel | 2020-08-11 | \- | 6.7\<br />6.6\<br />6.5 | -| tmux | tools | 2021-04-09 | \- | 3.1c\<br />2.3 | -| torchvision | vis | 2018-11-22 | \- | 0.2.1 | -| tqdm | lib | 2020-09-29 | \- | 4.41.1 | -| typing-extensions | devel | 2021-03-19 | \- | 3.7.4.3 | -| utf8proc | lib | 2019-11-08 | \- | 2.3.0 | -| util-linux | tools | 2021-02-17 | \- | 2.36\<br />2.35\<br />2.34\<br />2.33\<br />2.32\<br />2.31.1 | -| wheel | tools | 2018-11-22 | \- | 0.31.1\<br />0.31.0 | -| x264 | vis | 2020-08-11 | \- | 20191217\<br />20190925\<br />20190413\<br />20181203\<br />20180128 | -| x265 | vis | 2020-08-11 | \- | 3.3\<br />3.2\<br />3.0\<br />2.9\<br />2.6 | -| xbitmaps | devel | 2018-11-22 | \- | 1.1.1 | -| xmlf90 | data | 2020-02-27 | \- | 1.5.4 | -| xorg-macros | devel | 2021-02-17 | \- | 1.19.2\<br />1.19.1 | -| xprop | vis | 2019-11-08 | \- | 1.2.4\<br />1.2.3\<br />1.2.2 | -| xproto | devel | 2018-11-22 | \- | 7.0.31 | -| yaff | chem | 2020-08-11 | \- | 1.6.0 | -| zlib | lib | 2020-12-11 | \- | 1.2.8\<br />1.2.11 | -| zsh | tools | 2021-01-06 | \- | 5.8 | -| zstd | lib | 2020-08-11 | \- | 1.4.4 | - -### Classic Environment - -<span class="twiki-macro TABLE">headerrows= 1"</span> <span -class="twiki-macro EDITTABLE" -format="| text, 30, Software | text, 40, Kategorie | text, 30, Letzte Änderung | text, 30, SGI-UV | text, 30, Taurus | " -changerows="on"></span> - -| Software | Category | Last change | Venus | Taurus | -|:-------------------------|:-------------|:------------|:-----------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| AVS-Express | applications | 2015-06-26 | \- | mpepst8.2\<br />8.2 | -| FPLO | applications | 2018-04-13 | \- | 18.00-53\<br />14.00-49 | -| VirtualGL | tools | 2013-10-02 | \- | | -| abaqus | applications | 2018-04-19 | 2017\<br />2016 | *2018* \<br /> 6.9-EF1\<br />6.13\<br />6.12\<br />2017\<br />2016 | -| abinit | applications | 2013-11-21 | 7.2.1 | 7.2.1 | -| ace | lib | 2018-11-22 | \- | 6.3.3 | -| adolc | libraries | 2014-07-24 | *2.5.0* \<br /> 2.4.1 | *2.5.0* \<br /> 2.4.1 | -| afni | applications | 2014-07-01 | \- | 2011-12-21-1014 | -| amber | applications | 2017-10-06 | \- | 15 | -| ansys | applications | 2018-09-04 | 18.0\<br />17.1\<br />16.1 | 19.0\<br />18.2\<br />18.1\<br />18.0\<br />17.2\<br />17.1\<br />17.0\<br />16.1 | -| ansysem | applications | 2017-07-20 | \- | 16.0 | -| asm | tools | 2017-03-17 | \- | 5.2 | -| autoconf | tools | 2013-10-30 | 2.69 | 2.69 | -| automake | tools | 2014-09-11 | 1.14\<br />1.12.2\<br />1.12 | 1.14\<br />1.12.2 | -| autotools | tools | 2017-02-01 | \- | default\<br />2015 | -| bazel | compilers | 2017-07-13 | \- | 0.5.2 | -| bison | libraries | 2015-03-11 | \- | 3.0.4 | -| blcr | tools | 2016-03-02 | \- | | -| boost | libraries | 2019-03-29 | *1.49* \<br /> 1.69.0\<br />1.54\<br />1.51.0 | *1.54.0* \<br /> 1.66.0\<br />1.65.1\<br />1.65.0\<br />1.64.0\<br />1.63.0\<br />1.62.0\<br />1.61.0\<br />1.60.0\<br />1.59.0\<br />1.58.0\<br />1.57.0\<br />1.56.0\<br />1.55.0\<br />1.49 | -| bowtie | applications | 2013-01-16 | 0.12.8 | \- | -| bullxmpi | libraries | 2016-10-14 | \- | *1.2.8.4* \<br /> 1.2.9.2 | -| casita | tools | 2017-06-08 | \- | 1.9 | -| cdo | tools | 2013-04-08 | 1.6.0 | \- | -| ceph | libraries | 2017-01-13 | \- | 11.1 | -| cereal | libraries | 2016-12-07 | \- | 1.2.1 | -| cg | libraries | 2015-09-18 | \- | 3.1 | -| cgal | libraries | 2018-03-08 | \- | 4.11.1 | -| clFFT | libraries | 2017-07-12 | \- | 2.12.2dev\<br />2.12.2\<br />2.12.1\<br />2.12.0\<br />2.10.0 | -| clang | compilers | 2018-08-20 | \- | 4.0.0 | -| cmake | tools | 2019-04-03 | 3.3.1\<br />3.11.4\<br />2.8.2\<br />2.8.12.2\<br />2.8.11 | *3.10.1* \<br /> 3.9.0\<br />3.6.2\<br />3.3.1\<br />2.8.2\<br />2.8.12.2\<br />2.8.11 | -| collectl | tools | 2017-12-05 | 4.2.0\<br />4.1.2\<br />3.6.7\<br />3.5.1 | 4.2.0\<br />4.1.2\<br />3.6.7\<br />3.5.1 | -| comsol | applications | 2018-04-20 | \- | *5.3a* \<br /> 5.3 | -| conn | libraries | 2017-07-12 | \- | 17f | -| cp2k | applications | 2017-12-05 | 2.5 | *5.1* \<br /> r16298\<br />r15503\<br />r14075\<br />r13178\<br />3.0\<br />2.6.2\<br />2.6\<br />2.4\<br />2.3\<br />130326 | -| cpufrequtils | libraries | 2017-02-16 | \- | gcc5.3.0 | -| ctool | libraries | 2013-04-17 | 2.12 | 2.12 | -| cube | tools | 2018-05-31 | *4.3* | *4.3* \<br /> 4.4 | -| cuda | libraries | 2018-06-07 | \- | *9.2.88* \<br /> 9.1.85\<br />9.0.176\<br />8.0.61\<br />8.0.44\<br />7.5.18\<br />7.0.28 | -| curl | libraries | 2013-01-18 | 7.28.1 | \- | -| cusp | libraries | 2014-04-22 | \- | 0.4.0\<br />0.3.1 | -| cython | libraries | 2016-08-26 | \- | 0.24.1\<br />0.24\<br />0.19.2 | -| dalton | applications | 2016-04-07 | \- | 2016.0 | -| darshan | tools | 2017-09-02 | \- | darshan-3.1.4 | -| dash | libraries | 2019-02-14 | \- | dash | -| dataheap | libraries | 2017-01-12 | \- | 1.2\<br />1.1 | -| ddt | tools | 2021-04-12 | 4.2\<br />4.0\<br />3.2.1 | *18.0.1* \<br /> 6.0.5\<br />6.0 | -| dftb+ | applications | 2017-06-26 | \- | mpi\<br />1.3\<br />1.2.2 | -| dmtcp | tools | 2017-10-18 | \- | *2.5.1-ib* \<br /> ib-id | -| doxygen | tools | 2016-01-27 | \- | 1.8.11\<br />1.7.4 | -| dune | libraries | 2014-05-28 | \- | 2.2.1 | -| dyninst | libraries | 2016-02-16 | 8.1.1 | 8.2.1\<br />8.1.1 | -| eigen | libraries | 2017-03-23 | \- | 3.3.3\<br />3.2.0 | -| eirods | tools | 2013-12-11 | \- | 3.1 | -| eman2 | applications | 2017-06-21 | \- | 2.2 | -| ensight | applications | 2015-07-13 | \- | 10.1.5a\<br />10.0 | -| extrae | applications | 2017-01-03 | \- | 3.4.1 | -| fftw | libraries | 2017-03-18 | \- | 3.3.6pl1\<br />3.3.5\<br />3.3.4 | -| firestarter | applications | 2016-04-20 | \- | 1.4 | -| flex | lang | 2020-12-11 | \- | 2.5.39 | -| fme | applications | 2017-04-21 | \- | 2017 | -| freecad | applications | 2014-05-12 | \- | *0.14* \<br /> 0.13 | -| freeglut | lib | 2019-11-08 | 2.8.1 | 2.8.1 | -| freesurfer | applications | 2015-12-04 | \- | *5.3.0* \<br /> 5.1.0 | -| fsl | libraries | 2018-01-17 | \- | 5.0.5\<br />5.0.4\<br />5.0.10 | -| ga | libraries | 2015-06-22 | \- | 5.2 | -| gamess | applications | 2014-12-11 | \- | 2013 | -| gams | applications | 2017-07-10 | \- | 24.8\<br />24.3.3 | -| gaussian | applications | 2017-06-01 | g16\<br />g09d01\<br />g09b01\<br />g09\<br />g03 | *g16* \<br /> g09d01\<br />g09b01\<br />g09 | -| gautomatch | applications | 2017-06-21 | \- | 0.53 | -| gcc | compilers | 2019-04-02 | 8.3.0\<br />7.1.0\<br />6.3.0\<br />5.5.0\<br />4.9.3\<br />4.9.1\<br />4.8.2\<br />4.8.0\<br />4.7.1 | *7.1.0* \<br /> 6.3.0\<br />6.2.0\<br />5.5.0\<br />5.3.0\<br />5.2.0\<br />5.1.0\<br />4.9.3\<br />4.9.1\<br />4.8.2\<br />4.8.0\<br />4.7.1\<br />4.6.2 | -| gcl | compilers | 2017-03-13 | \- | 2.6.12 | -| gcovr | tools | 2015-02-12 | *3.2* | *3.2* | -| gctf | applications | 2017-06-21 | \- | 0.50 | -| gdb | tools | 2017-08-03 | 7.9.1 | 7.5 | -| gdk | tools | 2015-12-14 | \- | 352 | -| geany | tools | 2014-05-12 | \- | 1.24.1 | -| ghc | compilers | 2015-01-16 | 7.6.3 | 7.6.3 | -| git | tools | 2021-03-17 | *1.8.3.1* \<br /> 2.17.1\<br />1.7.7\<br />1.7.4.1\<br />1.7.3.2 | *2.15.1* \<br /> 2.7.3\<br />1.9.0 | -| glib | libraries | 2016-10-27 | \- | 2.50.1\<br />2.44.1 | -| gmap | tools | 2018-07-13 | \- | 2018-07-04 | -| gmock | tools | 2013-10-17 | \- | *1.6.0* | -| gnuplot | vis | 2019-11-08 | 4.6.1\<br />4.4.0 | 4.6.1\<br />4.4.0 | -| gpaw | applications | 2016-02-19 | \- | 0.11.0 | -| gperftools | tools | 2018-11-22 | \- | gperftools-2.6.1.lua | -| gpi2 | libraries | 2018-04-25 | \- | *git* \<br /> 1.3.0\<br />1.2.2\<br />1.1.0 | -| gpi2-mpi | libraries | 2015-03-30 | \- | *1.1.1* | -| gpudevkit | libraries | 2016-07-25 | \- | 352-79 | -| grads | applications | 2014-08-05 | 2.0.2 | \- | -| grid | tools | 2014-12-09 | \- | 2012 | -| gromacs | applications | 2018-01-22 | \- | *5.1.3* \<br /> 5.1.4\<br />5.1.1\<br />5.1\<br />4.6.7\<br />4.5.5\<br />3.3.3 | -| gsl | libraries | 2015-01-28 | \- | 1.16 | -| gulp | applications | 2015-10-09 | \- | 4.3 | -| gurobi | applications | 2017-06-28 | 7.0.2\<br />6.0.4 | 7.0.2\<br />6.0.4 | -| h5utils | tools | 2014-08-18 | \- | 1.12.1 | -| haskell-platform | tools | 2015-01-16 | 2013.2.0.0 | 2013.2.0.0 | -| hdeem | libraries | 2018-01-22 | \- | *deprecated* \<br /> 2.2.20ms\<br />2.2.2\<br />2.2.19ms\<br />2.2.16ms\<br />2.2.15ms\<br />2.2.13ms\<br />2.1.9ms\<br />2.1.5\<br />2.1.4\<br />2.1.10ms | -| hdf5 | libraries | 2018-02-09 | 1.8.14\<br />1.8.10 | hdfview\<br />1.8.19\<br />1.8.18\<br />1.8.16\<br />1.8.15\<br />1.8.14\<br />1.8.10\<br />1.6.5\<br />1.10.1 | -| hip | libraries | 2018-08-27 | \- | git | -| hoomd-blue | applications | 2016-07-29 | \- | 2.0.1 | -| hpc-x | libraries | 2017-04-10 | \- | 1.8.0 | -| hpctoolkit | tools | 2013-05-28 | 5.3.2 | \- | -| hpx | libraries | 2017-09-15 | \- | hpx | -| htop | tools | 2016-11-04 | 1.0.2 | 1.0.2 | -| hwloc | system | 2021-02-17 | \- | 1.11.8\<br />1.11.6 | -| hyperdex | tools | 2015-08-20 | \- | default\<br />1.8.1\<br />1.7.1 | -| hyperopt | libraries | 2018-03-19 | \- | 0.1 | -| imagemagick | applications | 2015-03-18 | \- | 6.9.0 | -| intel | toolchain | 2020-08-03 | *2013* \<br /> 2017.2.174\<br />2016.2.181\<br />2016.1.150\<br />2015.3.187\<br />2015.2.164\<br />2013-sp1\<br />11.1.069 | *2018.1.163* \<br /> 2018.0.128\<br />2017.4.196\<br />2017.2.174\<br />2017.1.132\<br />2017.0.020\<br />2016.2.181\<br />2016.1.150\<br />2015.3.187\<br />2015.2.164\<br />2015.1.133\<br />2013-sp1\<br />2013\<br />12.1\<br />11.1.069 | -| intelmpi | libraries | 2017-11-21 | \- | *2018.1.163* \<br /> 5.1.3.181\<br />5.1.2.150\<br />5.0.3.048\<br />5.0.1.035\<br />2018.0.128\<br />2017.3.196\<br />2017.2.174\<br />2017.1.132\<br />2017.0.098\<br />2013 | -| iotop | tools | 2013-07-16 | \- | 0.5 | -| iotrack | tools | 2013-07-16 | \- | 0.5 | -| java | tools | 2015-11-17 | jre1.6.0-21\<br />jdk1.8.0-66\<br />jdk1.7.0-25\<br />jdk1.7.0-03 | jdk1.8.0-66\<br />jdk1.7.0-25 | -| julia | compilers | 2018-05-15 | \- | *0.6.2* \<br /> 0.4.6\<br />0.4.1 | -| knime | applications | 2017-03-20 | \- | 3.3.1\<br />3.1.0\<br />2.11.3-24\<br />2.11.3 | -| lammps | applications | 2017-08-31 | 2014sep\<br />2014jun\<br />2013feb | *2016jul* \<br /> 2017aug\<br />2016may\<br />2015aug\<br />2014sep\<br />2014feb\<br />2013feb\<br />2013aug | -| lbfgsb | libraries | 2013-08-02 | *3.0* \<br /> 2.1 | *3.0* \<br /> 2.1 | -| libnbc | libraries | 2014-05-28 | \- | 1.1.1 | -| libssh2 | tools | 2018-11-22 | \- | 1.8.0 | -| libsvm | tools | 2015-11-20 | \- | 3.20 | -| libtool | lib | 2021-02-17 | \- | 2.4.2 | -| libunwind | lib | 2020-09-25 | \- | 1.1 | -| libxc | chem | 2021-01-05 | \- | 3.0.0\<br />2.2.2 | -| liggghts | applications | 2014-05-28 | \- | 2.3.8\<br />2.3.2 | -| llview | tools | 2015-01-28 | \- | | -| llvm | compilers | 2018-04-20 | \- | *4.0.0* \<br /> ykt\<br />3.9.1\<br />3.7\<br />3.4\<br />3.3.1 | -| lo2s | perf | 2020-01-27 | \- | 2018-02-13\<br />2017-12-06\<br />2017-08-07 | -| ls-dyna | applications | 2017-12-05 | *9.0.1* \<br /> 971\<br />7.0 | *10.0.0* \<br /> dev-121559\<br />971\<br />9.0.1\<br />8.1\<br />7.1.2\<br />7.1.1\<br />7.0\<br />6.0 | -| ls-dyna-usermat | applications | 2016-08-25 | \- | 9.0.1-s\<br />9.0.1-d\<br />7.1.2-d\<br />7.1.1-s\<br />7.1.1-d | -| ls-prepost | applications | 2016-08-22 | \- | *4.3* | -| lumerical | applications | 2016-06-01 | \- | fdtd-8.11.422 | -| m4 | tools | 2013-10-30 | \- | 1.4.16 | -| m4ri | libraries | 2017-03-27 | \- | 20140914 | -| make | tools | 2018-02-21 | \- | 4.2 | -| map | tools | 2016-11-22 | \- | 6.0.5 | -| mathematica | applications | 2015-10-16 | \- | *10.0* \<br /> 8.0 | -| matlab | applications | 2019-02-26 | deprecated.lua\<br />2017a.lua\<br />2016b.lua\<br />2015b.lua\<br />2014a.lua\<br />2013a.lua\<br />2012a.lua\<br />2010b.lua\<br />2010a.lua | \- | -| maxima | applications | 2017-03-15 | \- | 5.39.0 | -| med | libraries | 2017-09-27 | \- | 3.2.0 | -| meep | applications | 2015-04-23 | \- | 1.3\<br />1.2.1 | -| mercurial | tools | 2014-10-22 | 3.1.2 | 3.1.2 | -| metis | libraries | 2013-12-17 | 5.1.0 | 5.1.0\<br />4.0.3 | -| mkl | libraries | 2017-05-10 | 2013 | 2017\<br />2015\<br />2013 | -| modenv | environment | 2020-03-25 | \- | scs5.lua\<br />ml.lua\<br />hiera.lua\<br />classic.lua | -| mongodb | applications | 2018-03-19 | \- | 3.6.3 | -| motioncor2 | applications | 2017-06-21 | \- | 01-30-2017 | -| mpb | applications | 2014-08-19 | \- | 1.4.2 | -| mpi4py | libraries | 2016-05-02 | 1.3.1 | 2.0.0\<br />1.3.1 | -| mpirt | libraries | 2017-11-21 | \- | *2018.1.163* \<br /> 5.1.3.181\<br />5.1.2.150\<br />5.0.3.048\<br />5.0.1.035\<br />2018.0.128\<br />2017.3.196\<br />2017.2.174\<br />2017.1.132\<br />2017.0.098\<br />2013 | -| mumps | libraries | 2017-05-11 | \- | 5.1.1 | -| must | tools | 2018-02-01 | \- | 1.5.0\<br />1.4.0 | -| mvapich2 | libraries | 2017-05-23 | \- | 2.2 | -| mysql | tools | 2013-12-06 | \- | 6.0.11 | -| namd | applications | 2015-09-08 | \- | *2.10* \<br /> 2.9 | -| nco | tools | 2013-08-01 | 4.3.0 | \- | -| nedit | tools | 2013-04-30 | 5.6\<br />5.5 | 5.6\<br />5.5 | -| netbeans | applications | 2018-03-07 | \- | 8.2 | -| netcdf | libraries | 2018-02-09 | 4.1.3 | 4.6.0\<br />4.4.0\<br />4.3.3.1\<br />4.1.3 | -| netlogo | applications | 2017-08-08 | \- | 6.0.1\<br />5.3.0\<br />5.2.0 | -| nsys | tools | 2018-09-11 | \- | 2018.1.1.36\<br />2018.0.1.173 | -| numeca | cae | 2018-11-22 | \- | all | -| nwchem | applications | 2016-02-15 | 6.3 | *6.6* \<br /> custom\<br />6.5patched\<br />6.5\<br />6.3.r2\<br />6.3 | -| octave | applications | 2018-03-23 | \- | 3.8.1 | -| octopus | applications | 2017-03-07 | \- | 6.0 | -| openbabel | applications | 2014-03-07 | 2.3.2 | 2.3.2 | -| opencl | libraries | 2015-07-13 | \- | 1.2-4.4.0.117 | -| openems | applications | 2017-08-01 | \- | 0.0.35 | -| openfoam | applications | 2020-10-15 | \- | *2.3.0* \<br /> v1712\<br />v1706\<br />5.0\<br />4.0\<br />2.4.0\<br />2.3.1\<br />2.2.2 | -| openmpi | libraries | 2018-02-01 | \- | *1.10.2* \<br /> 3.0.0\<br />2.1.1\<br />2.1.0\<br />1.8.8\<br />1.10.4\<br />1.10.3 | -| opentelemac | applications | 2017-09-29 | \- | v7p2r3\<br />v7p1r1 | -| oprofile | tools | 2013-06-05 | 0.9.8 | \- | -| orca | applications | 2017-07-27 | \- | 4.0.1\<br />4.0.0.2\<br />3.0.3 | -| otf2 | libraries | 2018-02-12 | \- | *2.0* \<br /> 2.1\<br />1.4\<br />1.3 | -| papi | libraries | 2017-11-06 | 5.1.0 | 5.5.1\<br />5.4.3\<br />5.4.1 | -| parallel | tools | 2020-02-25 | \- | 20170222 | -| paraview | applications | 2016-03-03 | \- | *4.1.0* \<br /> 4.0.1 | -| parmetis | libraries | 2018-01-12 | \- | 4.0.3 | -| pasha | applications | 2013-11-14 | 1.0.9 | \- | -| pathscale | compilers | 2016-03-04 | \- | enzo-6.0.858\<br />enzo-6.0.749 | -| pdt | tools | 2015-09-18 | 3.18.1 | 3.18.1 | -| perf | tools | 2016-06-14 | | | -| perl | applications | 2015-01-29 | 5.20.1\<br />5.12.1 | 5.20.1\<br />5.12.1 | -| petsc | libraries | 2018-03-19 | *3.3-p6* \<br /> 3.1-p8 | *3.3-p6* \<br /> 3.8.3-64bit\<br />3.8.3\<br />3.4.4\<br />3.4.3\<br />3.3-p7-64bit\<br />3.3-p7\<br />3.2-p7\<br />3.1-p8-p\<br />3.1-p8 | -| pgi | compilers | 2018-09-07 | 14.9\<br />14.7\<br />14.6\<br />14.3\<br />13.4 | *18.3* \<br /> 17.7\<br />17.4\<br />17.1\<br />16.9\<br />16.5\<br />16.4\<br />16.10\<br />16.1\<br />15.9\<br />14.9 | -| pigz | tools | 2018-11-22 | \- | 2.3.4 | -| prope-env | tools | 2017-05-02 | \- | *.1.0* | -| protobuf | devel | 2021-03-17 | \- | 3.5.0\<br />3.2.0 | -| pycuda | libraries | 2016-10-28 | \- | *2016.1.2* \<br /> 2013.1.1\<br />2012.1 | -| pyslurm | libraries | 2017-11-09 | \- | 16.05.8 | -| python | libraries | 2018-01-17 | 3.6\<br />3.3.0\<br />2.7.5\<br />2.7 | *3.6* \<br /> intelpython3\<br />intelpython2\<br />3.5.2\<br />3.4.3\<br />3.3.0\<br />3.1.2\<br />2.7.6\<br />2.7.5\<br />2.7 | -| q-chem | applications | 2016-12-12 | \- | 4.4 | -| qt | libraries | 2016-10-26 | \- | *4.8.1* \<br /> 5.4.1\<br />4.8.7 | -| quantum_espresso | applications | 2016-09-13 | *5.0.3* \<br /> 5.0.2 | 5.3.0\<br />5.1.2\<br />5.0.3 | -| r | applications | 2016-02-18 | \- | 3.2.1\<br />2.15.3 | -| ramdisk | tools | 2016-07-21 | 1.0 | \- | -| read-nvml-clocks-pci | tools | 2018-02-22 | \- | 1.0 | -| readex | tools | 2018-06-13 | \- | pre\<br />beta-1806\<br />beta-1805\<br />beta-1804\<br />beta\<br />alpha | -| redis | tools | 2016-06-28 | \- | 3.2.1 | -| relion | applications | 2017-06-21 | \- | 2.1 | -| repoclient | applications | 2017-01-18 | \- | 1.4.1 | -| ripgrep | tools | 2017-02-17 | \- | 0.3.2 | -| robinhood | tools | 2017-05-04 | \- | 2.4.3 | -| root | applications | 2015-02-27 | \- | 6.02.05 | -| rstudio | lang | 2020-02-18 | \- | 0.98.1103 | -| ruby | tools | 2014-07-21 | \- | 2.1.2 | -| samrai | libraries | 2016-04-22 | \- | 3.10.0 | -| samtools | tools | 2018-07-13 | 0.1.18 | 1.8 | -| scafes | libraries | 2017-05-05 | *2.3.0* \<br /> 2.2.0\<br />2.0.0\<br />1.0.0 | *2.3.0* \<br /> 2.2.0\<br />2.1.0\<br />2.0.1\<br />2.0.0\<br />1.0.0 | -| scala | compilers | 2015-06-22 | \- | 2.11.4\<br />2.10.4 | -| scalapack | libraries | 2013-11-21 | 2.0.2 | \- | -| scalasca | tools | 2018-02-06 | \- | *2.3.1* | -| scons | tools | 2015-11-19 | 2.3.4 | 2.4.1\<br />2.3.4 | -| scorep | tools | 2018-10-01 | 1.3.0 | *3.0* \<br /> try\<br />trunk\<br />ompt\<br />java\<br />dev-io\<br />3.1 | -| scorep-apapi | libraries | 2018-01-09 | \- | gcc-2018-01-09 | -| scorep-cpu-energy | libraries | 2017-01-20 | \- | r217\<br />r211\<br />r117\<br />2017-01-20\<br />2016-04-07 | -| scorep-cpu-id | libraries | 2014-08-13 | \- | r117 | -| scorep-dataheap | libraries | 2015-07-28 | \- | *2015-07-28* \<br /> r191\<br />r122 | -| scorep-dev | tools | 2017-07-19 | \- | *05* | -| scorep-hdeem | libraries | 2018-06-19 | \- | *2016-12-20* \<br /> sync\<br />2017-12-08a\<br />2017-12-08.lua\<br />2016-11-21 | -| scorep-plugin-x86-energy | libraries | 2018-06-19 | \- | xmpi\<br />intelmpi\<br />2017-09-06\<br />2017-09-05 | -| scorep-printmetrics | libraries | 2018-02-26 | \- | 2018-02-26 | -| scorep-uncore | libraries | 2018-06-21 | \- | *2018-01-24* \<br /> 2016-03-29 | -| scorep_plugin_x86_energy | libraries | 2018-07-04 | \- | intel-2018\<br />gcc-7.1.0\<br />2017-07-14 | -| scout | compilers | 2015-06-22 | 1.6.0 | 1.6.0 | -| sed | tools | 2018-02-21 | \- | 4.4 | -| sftp | tools | 2014-04-10 | \- | 6.6 | -| shifter | tools | 2016-06-09 | \- | 16.04.0pre1 | -| siesta | applications | 2017-05-29 | 3.1-pl20 | 4.0\<br />3.2-pl4 | -| singularity | tools | 2019-02-13 | | ff69c5f3 | -| sionlib | tools | 2017-06-29 | \- | 1.6.1\<br />1.5.5 | -| siox | libraries | 2016-10-27 | \- | 2016-10-27\<br />2016-10-26 | -| spm | applications | 2014-07-09 | \- | 8-r4667 | -| spm12 | libraries | 2017-07-12 | \- | r6906 | -| spparks | applications | 2016-06-30 | \- | 2016feb | -| sqlite3 | libraries | 2016-07-27 | 3.8.2 | 3.8.10 | -| sra-tools | tools | 2018-07-19 | \- | 2.9.1 | -| stack | tools | 2016-06-23 | \- | 1.1.2 | -| star | applications | 2017-10-25 | \- | *12.06* \<br /> 9.06\<br />12.04\<br />12.02\<br />11.02\<br />10.04 | -| subread | tools | 2018-07-13 | \- | 1.6.2 | -| suitesparse | libraries | 2017-08-25 | \- | *4.5.4* \<br /> 4.2.1 | -| superlu | libraries | 2017-08-25 | \- | *5.2.1* | -| superlu_dist | libraries | 2017-03-21 | \- | 5.1.3 | -| superlu_mt | libraries | 2017-03-21 | \- | 3.1 | -| svn | tools | 2016-03-16 | 1.8.11\<br />1.7.3 | *1.9.3* \<br /> 1.8.8\<br />1.8.11 | -| swig | tools | 2016-04-08 | 2.0 | 3.0.8\<br />2.0 | -| swipl | tools | 2017-02-28 | \- | 7.4.0-rc2 | -| tcl | applications | 2017-08-07 | \- | 8.6.6 | -| tcltk | applications | 2015-02-06 | \- | 8.4.20 | -| tecplot360 | applications | 2018-05-22 | 2015\<br />2013 | 2018r1\<br />2017r2\<br />2017r1\<br />2016r2\<br />2015r2\<br />2015\<br />2013\<br />2010 | -| tesseract | libraries | 2016-06-28 | \- | 3.04 | -| texinfo | devel | 2020-08-11 | \- | 5.2 | -| theodore | libraries | 2016-05-27 | \- | 1.3 | -| tiff | libraries | 2013-09-16 | 3.9.2 | 3.9.2 | -| tinker | applications | 2014-03-04 | \- | 6.3 | -| tmux | tools | 2021-04-09 | \- | 2.2 | -| totalview | tools | 2017-11-03 | *2017.2.11* \<br /> 8.9.2-0\<br />8.8.0-1\<br />8.13.0-0\<br />8.11.0-3 | *2017.2.11* \<br /> 8.9.2-0\<br />8.8.0-1\<br />8.13.0-0\<br />8.11.0-3 | -| trilinos | applications | 2016-04-13 | \- | 12.6.1 | -| trinityrnaseq | applications | 2013-05-16 | r2013-02-25 | \- | -| turbomole | applications | 2017-01-10 | 7.1\<br />6.6\<br />6.5 | 7.1\<br />6.6\<br />6.5 | -| valgrind | tools | 2017-08-08 | 3.8.1 | *3.10.1* \<br /> r15216\<br />3.8.1\<br />3.13.0 | -| vampir | tools | 2019-03-04 | *9.6.1* \<br /> 9.7\<br />9.5.0\<br />9.4.0\<br />9.3.0\<br />8.5.0 | *9.5.0* \<br /> 9.4.0\<br />9.3.0\<br />9.3\<br />9.2.0\<br />9.1.0\<br />9.0.0\<br />8.5.0\<br />8.4.1\<br />8.3.0 | -| vampirlive | tools | 2018-02-27 | \- | | -| vampirtrace | tools | 2016-03-29 | *5.14.4* | *5.14.4* | -| vampirtrace-plugins | libraries | 2014-08-06 | \- | x86\<br />power-1.1\<br />power-1.0\<br />apapi | -| vasp | applications | 2017-11-08 | *5.3* \<br /> 5.2 | 5.4.4\<br />5.4.1\<br />5.3 | -| visit | applications | 2016-12-13 | \- | 2.4.2\<br />2.12.0 | -| vmd | applications | 2016-12-15 | \- | 1.9.3 | -| vt_dataheap | libraries | 2014-08-14 | \- | r190 | -| vtk | libraries | 2016-08-16 | 5.10.1 | 5.10.1 | -| wannier90 | libraries | 2016-04-21 | 1.2 | 2.0.1\<br />1.2 | -| wget | tools | 2015-05-12 | 1.16.3 | 1.16.3 | -| wxwidgets | libraries | 2017-03-15 | \- | 3.0.2 | -| yade | applications | 2014-05-22 | \- | | -| zlib | lib | 2020-12-11 | \- | 1.2.8 | - -### ML Environment - -<span class="twiki-macro TABLE">headerrows= 1"</span> <span -class="twiki-macro EDITTABLE" -format="| text, 30, Software | text, 40, Kategorie | text, 30, Letzte Änderung | text, 30, SGI-UV | text, 30, Taurus | " -changerows="on"></span> - -| Software | Category | Last change | Venus | Taurus | -|:--------------------------|:----------|:------------|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| ASE | chem | 2020-09-29 | \- | 3.19.0 | -| ATK | vis | 2020-10-12 | \- | 2.34.1\<br />2.28.1 | -| Anaconda3 | lang | 2019-11-13 | \- | 2019.07\<br />2019.03 | -| Arrow | data | 2020-11-30 | \- | 0.16.0\<br />0.14.1 | -| Autoconf | devel | 2020-10-14 | \- | 2.69 | -| Automake | devel | 2020-10-14 | \- | 1.16.1\<br />1.15.1 | -| Autotools | devel | 2020-10-14 | \- | 20180311\<br />20170619 | -| Bazel | devel | 2021-03-22 | \- | 3.7.1\<br />3.4.1\<br />2.0.0\<br />1.1.0\<br />0.29.1\<br />0.26.1\<br />0.25.2\<br />0.20.0\<br />0.18.0 | -| BigDataFrameworkConfigure | devel | 2019-09-16 | \- | 0.0.2\<br />0.0.1 | -| Bison | lang | 2020-10-14 | \- | 3.5.3\<br />3.3.2\<br />3.0.5\<br />3.0.4 | -| Boost | devel | 2020-11-30 | \- | 1.71.0\<br />1.70.0\<br />1.69.0\<br />1.67.0\<br />1.66.0 | -| CMake | devel | 2020-11-02 | \- | 3.9.5\<br />3.9.1\<br />3.16.4\<br />3.15.3\<br />3.13.3\<br />3.12.1\<br />3.11.4\<br />3.10.2 | -| CUDA | system | 2020-10-14 | \- | 9.2.88\<br />11.0.2\<br />10.1.243\<br />10.1.105 | -| CUDAcore | system | 2020-10-14 | \- | 11.0.2 | -| Check | lib | 2020-10-14 | \- | 0.15.2 | -| Clang | compiler | 2020-02-20 | \- | 9.0.1 | -| CubeLib | perf | 2020-07-23 | \- | 4.4.4\<br />4.4 | -| CubeW | perf | 2019-02-04 | \- | 4.4 | -| CubeWriter | perf | 2020-07-23 | \- | 4.4.3 | -| DBus | devel | 2019-09-11 | \- | 1.13.8 | -| Devel-NYTProf | perf | 2019-08-30 | \- | 6.06 | -| Doxygen | devel | 2019-07-17 | \- | 1.8.14\<br />1.8.13 | -| EasyBuild | tools | 2021-03-08 | \- | 4.3.3\<br />4.3.2\<br />4.3.1\<br />4.3.0\<br />4.2.2\<br />4.2.0\<br />4.1.2\<br />4.1.1\<br />4.1.0\<br />4.0.1\<br />3.9.4\<br />3.9.3\<br />3.9.2\<br />3.9.1\<br />3.8.1\<br />3.8.0\<br />3.7.1\<br />3.7.0\<br />3.6.2\<br />3.6.1 | -| Eigen | math | 2020-11-02 | \- | 3.3.7 | -| FFTW | numlib | 2020-10-14 | \- | 3.3.8\<br />3.3.7\<br />3.3.6 | -| FFmpeg | vis | 2020-09-29 | \- | 4.2.1\<br />4.1 | -| Flink | devel | 2019-09-16 | \- | 1.9.0\<br />1.8.1 | -| FriBidi | lang | 2020-09-29 | \- | 1.0.5 | -| GCC | compiler | 2020-10-14 | \- | 9.3.0\<br />8.3.0\<br />8.2.0-2.31.1\<br />7.3.0-2.30\<br />6.4.0-2.28 | -| GCCcore | compiler | 2020-10-14 | \- | 9.3.0\<br />8.3.0\<br />8.2.0\<br />7.3.0\<br />6.4.0 | -| GDAL | data | 2019-08-19 | \- | 2.2.3 | -| GDRCopy | lib | 2020-10-14 | \- | 2.1 | -| GEOS | math | 2019-08-19 | \- | 3.6.2 | -| GLib | vis | 2020-02-19 | \- | 2.62.0\<br />2.60.1\<br />2.54.3 | -| GMP | math | 2020-11-02 | \- | 6.2.0\<br />6.1.2 | -| GObject-Introspection | devel | 2020-10-12 | \- | 1.63.1\<br />1.54.1 | -| GSL | numlib | 2020-02-19 | \- | 2.6\<br />2.5 | -| GTK+ | vis | 2019-02-15 | \- | 2.24.32 | -| Gdk-Pixbuf | vis | 2019-02-15 | \- | 2.36.12 | -| Ghostscript | tools | 2020-02-19 | \- | 9.50\<br />9.27 | -| HDF5 | data | 2020-02-19 | \- | 1.10.5\<br />1.10.2\<br />1.10.1 | -| Hadoop | devel | 2019-09-16 | \- | 2.7.7 | -| HarfBuzz | vis | 2019-02-15 | \- | 2.2.0 | -| Horovod | tools | 2020-08-04 | \- | 0.19.5\<br />0.18.2 | -| Hyperopt | lib | 2020-02-19 | \- | 0.2.2 | -| ICU | lib | 2020-02-19 | \- | 64.2\<br />61.1\<br />56.1 | -| ImageMagick | vis | 2020-02-20 | \- | 7.0.9-5\<br />7.0.8-46 | -| JUnit | devel | 2020-01-21 | \- | 4.12 | -| JasPer | vis | 2020-02-19 | \- | 2.0.14 | -| Java | lang | 2020-02-26 | \- | 11.0.6\<br />1.8.0-162\<br />1.8-191-b26 | -| JsonCpp | lib | 2020-10-30 | \- | 1.9.3 | -| Keras | math | 2019-06-28 | \- | 2.2.4 | -| LAME | data | 2020-06-24 | \- | 3.100 | -| LLVM | compiler | 2020-09-24 | \- | 9.0.0\<br />8.0.1\<br />7.0.1\<br />6.0.0\<br />5.0.1 | -| LMDB | lib | 2020-10-30 | \- | 0.9.24 | -| LibTIFF | lib | 2020-02-19 | \- | 4.0.9\<br />4.0.10 | -| LibUUID | lib | 2019-08-02 | \- | 1.0.3 | -| LittleCMS | vis | 2020-02-20 | \- | 2.9 | -| M4 | devel | 2020-10-14 | \- | 1.4.18\<br />1.4.17 | -| MPFR | math | 2020-06-24 | \- | 4.0.2 | -| Mako | devel | 2020-02-19 | \- | 1.1.0\<br />1.0.8\<br />1.0.7 | -| Mesa | vis | 2020-02-19 | \- | 19.1.7\<br />19.0.1\<br />18.1.1\<br />17.3.6 | -| Meson | tools | 2020-11-03 | \- | 0.55.1\<br />0.51.2\<br />0.50.0 | -| MongoDB | data | 2019-08-05 | \- | 4.0.3 | -| NASM | lang | 2020-02-19 | \- | 2.14.02\<br />2.13.03 | -| NCCL | lib | 2020-02-18 | \- | 2.4.8\<br />2.4.2\<br />2.3.7 | -| NLopt | numlib | 2020-02-19 | \- | 2.6.1\<br />2.4.2 | -| NSPR | lib | 2019-09-11 | \- | 4.21 | -| NSS | lib | 2019-09-11 | \- | 3.42.1 | -| Ninja | tools | 2020-11-03 | \- | 1.9.0\<br />1.10.0 | -| OPARI2 | perf | 2020-07-23 | \- | 2.0.5\<br />2.0.3 | -| OTF2 | perf | 2020-07-23 | \- | 2.2\<br />2.1.1 | -| OpenBLAS | numlib | 2020-10-14 | \- | 0.3.9\<br />0.3.7\<br />0.3.5\<br />0.3.1\<br />0.2.20 | -| OpenCV | vis | 2019-02-21 | \- | 4.0.1 | -| OpenMPI | mpi | 2021-02-10 | \- | 4.0.3\<br />3.1.4\<br />3.1.3\<br />3.1.1 | -| OpenPGM | system | 2019-09-11 | \- | 5.2.122 | -| PAPI | perf | 2020-07-23 | \- | 6.0.0\<br />5.6.0 | -| PCRE | devel | 2020-02-19 | \- | 8.43\<br />8.41 | -| PCRE2 | devel | 2019-09-11 | \- | 10.33 | -| PDT | perf | 2020-07-23 | \- | 3.25 | -| PGI | compiler | 2019-05-14 | \- | 19.4 | -| PMIx | lib | 2020-10-14 | \- | 3.1.5 | -| PROJ | lib | 2019-08-19 | \- | 5.0.0 | -| Pango | vis | 2019-02-15 | \- | 1.42.4 | -| Perl | lang | 2020-10-14 | \- | 5.30.2\<br />5.30.0\<br />5.28.1\<br />5.28.0\<br />5.26.1 | -| Pillow | vis | 2020-06-24 | \- | 6.2.1 | -| PowerAI | data | 2019-12-10 | \- | 1.7.0.a0\<br />1.6.1 | -| PyTorch | devel | 2020-09-29 | \- | 1.6.0\<br />1.3.1\<br />1.1.0 | -| PyTorch-Geometric | devel | 2020-09-29 | \- | 1.6.1 | -| PyYAML | lib | 2020-02-18 | \- | 5.1.2\<br />3.13 | -| Python | lang | 2020-11-02 | \- | 3.8.2\<br />3.7.4\<br />3.7.2\<br />3.6.6\<br />3.6.4\<br />2.7.16\<br />2.7.15\<br />2.7.14 | -| PythonAnaconda | lang | 2019-12-10 | \- | 3.7\<br />3.6 | -| Qt5 | devel | 2019-09-12 | \- | 5.12.3 | -| R | lang | 2020-08-20 | \- | 3.6.2\<br />3.6.0\<br />3.4.4 | -| RDFlib | lib | 2020-09-29 | \- | 4.2.2 | -| SCons | devel | 2019-09-11 | \- | 3.0.5 | -| SIONlib | lib | 2020-07-23 | \- | 1.7.6 | -| SQLite | devel | 2020-11-02 | \- | 3.31.1\<br />3.29.0\<br />3.27.2\<br />3.24.0\<br />3.21.0\<br />3.20.1 | -| SWIG | devel | 2020-10-30 | \- | 4.0.1\<br />3.0.12 | -| ScaLAPACK | numlib | 2020-10-14 | \- | 2.1.0\<br />2.0.2 | -| SciPy-bundle | lang | 2020-11-02 | \- | 2020.03\<br />2019.10 | -| Score-P | perf | 2020-07-23 | \- | 6.0\<br />4.1 | -| Six | lib | 2019-02-05 | \- | 1.11.0 | -| Spark | devel | 2020-09-29 | \- | 3.0.1\<br />2.4.4\<br />2.4.3 | -| SpectrumMPI | mpi | 2019-01-14 | \- | system | -| Szip | tools | 2020-02-18 | \- | 2.1.1 | -| Tcl | lang | 2020-11-02 | \- | 8.6.9\<br />8.6.8\<br />8.6.7\<br />8.6.10 | -| TensorFlow | lib | 2020-10-30 | \- | 2.3.1\<br />2.2.0\<br />2.1.0\<br />2.0.0\<br />1.15.0\<br />1.14.0 | -| Tk | vis | 2020-02-19 | \- | 8.6.9\<br />8.6.8 | -| Tkinter | lang | 2020-09-23 | \- | 3.7.4\<br />3.6.6 | -| UCX | lib | 2020-10-14 | \- | 1.8.0 | -| UDUNITS | phys | 2020-02-19 | \- | 2.2.26 | -| UnZip | tools | 2020-11-02 | \- | 6.0 | -| Vampir | perf | 2020-11-30 | \- | 9.9.0\<br />9.8.0\<br />9.7.1\<br />9.11\<br />9.10.0 | -| X11 | vis | 2020-11-03 | \- | 20200222\<br />20190717\<br />20190311\<br />20180604\<br />20180131 | -| XML-Parser | data | 2019-02-15 | \- | 2.44-01 | -| XZ | tools | 2020-10-14 | \- | 5.2.5\<br />5.2.4\<br />5.2.3 | -| Yasm | lang | 2020-09-29 | \- | 1.3.0 | -| ZeroMQ | devel | 2019-09-11 | \- | 4.3.2 | -| Zip | tools | 2020-07-30 | \- | 3.0 | -| ant | devel | 2020-10-12 | \- | 1.10.7\<br />1.10.1 | -| binutils | tools | 2020-10-14 | \- | 2.34\<br />2.32\<br />2.31.1\<br />2.30\<br />2.28 | -| bokeh | tools | 2020-09-29 | \- | 1.4.0 | -| bzip2 | tools | 2020-11-02 | \- | 1.0.8\<br />1.0.6 | -| cURL | tools | 2020-11-02 | \- | 7.69.1\<br />7.66.0\<br />7.63.0\<br />7.60.0\<br />7.58.0 | -| cairo | vis | 2020-02-19 | \- | 1.16.0\<br />1.14.12 | -| cftime | data | 2019-07-17 | \- | 1.0.1 | -| cuDNN | numlib | 2020-02-18 | \- | 7.6.4.38\<br />7.4.2.24\<br />7.1.4.18 | -| dask | data | 2020-09-29 | \- | 2.8.0 | -| dill | data | 2019-10-29 | \- | 0.3.1.1 | -| double-conversion | lib | 2020-08-13 | \- | 3.1.4 | -| expat | tools | 2020-10-14 | \- | 2.2.9\<br />2.2.7\<br />2.2.6\<br />2.2.5 | -| flatbuffers | devel | 2020-10-30 | \- | 1.12.0 | -| flatbuffers-python | devel | 2021-04-10 | \- | 1.12 | -| flex | lang | 2020-10-14 | \- | 2.6.4\<br />2.6.3 | -| fontconfig | vis | 2020-11-03 | \- | 2.13.92\<br />2.13.1\<br />2.13.0\<br />2.12.6 | -| fosscuda | toolchain | 2020-10-14 | \- | 2020a\<br />2019b\<br />2019a\<br />2018b | -| freetype | vis | 2020-11-03 | \- | 2.9.1\<br />2.9\<br />2.10.1 | -| future | lib | 2019-02-05 | \- | 0.16.0 | -| gcccuda | toolchain | 2020-10-14 | \- | 2020a\<br />2019b\<br />2019a\<br />2018b | -| gettext | tools | 2020-11-03 | \- | 0.20.1\<br />0.19.8.1 | -| gflags | devel | 2020-06-24 | \- | 2.2.2 | -| giflib | lib | 2020-10-30 | \- | 5.2.1 | -| git | tools | 2020-02-18 | \- | 2.23.0\<br />2.18.0 | -| glog | devel | 2020-06-24 | \- | 0.4.0 | -| golf | toolchain | 2019-01-14 | \- | 2018a | -| gompic | toolchain | 2020-10-14 | \- | 2020a\<br />2019b\<br />2019a\<br />2018b | -| gperf | devel | 2020-11-03 | \- | 3.1 | -| gsmpi | toolchain | 2019-01-14 | \- | 2018a | -| gsolf | toolchain | 2019-01-14 | \- | 2018a | -| h5py | data | 2020-07-30 | \- | 2.8.0\<br />2.10.0 | -| help2man | tools | 2020-10-14 | \- | 1.47.8\<br />1.47.6\<br />1.47.4\<br />1.47.12 | -| hwloc | system | 2020-10-14 | \- | 2.2.0\<br />2.0.3\<br />1.11.12\<br />1.11.11\<br />1.11.10 | -| hypothesis | tools | 2020-06-24 | \- | 4.44.2 | -| intltool | devel | 2020-11-03 | \- | 0.51.0 | -| libGLU | vis | 2020-02-19 | \- | 9.0.1\<br />9.0.0 | -| libdrm | lib | 2020-02-19 | \- | 2.4.99\<br />2.4.97\<br />2.4.92\<br />2.4.91 | -| libevent | lib | 2020-10-14 | \- | 2.1.8\<br />2.1.11 | -| libfabric | lib | 2021-02-10 | \- | 1.11.0 | -| libffi | lib | 2020-11-02 | \- | 3.3\<br />3.2.1 | -| libgeotiff | lib | 2019-08-19 | \- | 1.4.2 | -| libjpeg-turbo | lib | 2020-02-19 | \- | 2.0.3\<br />2.0.2\<br />2.0.0\<br />1.5.3 | -| libpciaccess | system | 2020-10-14 | \- | 0.16\<br />0.14 | -| libpng | lib | 2020-11-03 | \- | 1.6.37\<br />1.6.36\<br />1.6.34 | -| libreadline | lib | 2020-10-14 | \- | 8.0\<br />7.0 | -| libsndfile | lib | 2020-02-19 | \- | 1.0.28 | -| libsodium | lib | 2019-09-11 | \- | 1.0.17 | -| libtool | lib | 2020-10-14 | \- | 2.4.6 | -| libunwind | lib | 2020-09-25 | \- | 1.3.1\<br />1.2.1 | -| libxml2 | lib | 2020-10-14 | \- | 2.9.9\<br />2.9.8\<br />2.9.7\<br />2.9.4\<br />2.9.10 | -| libxslt | lib | 2020-10-27 | \- | 1.1.34\<br />1.1.33\<br />1.1.32 | -| libyaml | lib | 2020-01-24 | \- | 0.2.2\<br />0.2.1 | -| magma | math | 2020-09-29 | \- | 2.5.1 | -| matplotlib | vis | 2020-09-23 | \- | 3.1.1\<br />3.0.3 | -| ncurses | devel | 2020-10-14 | \- | 6.2\<br />6.1\<br />6.0 | -| netCDF | data | 2019-07-17 | \- | 4.6.1\<br />4.6.0 | -| netcdf4-python | data | 2019-07-17 | \- | 1.4.3 | -| nettle | lib | 2020-02-19 | \- | 3.5.1\<br />3.4.1\<br />3.4 | -| nsync | devel | 2020-10-30 | \- | 1.24.0 | -| numactl | tools | 2020-10-14 | \- | 2.0.13\<br />2.0.12\<br />2.0.11 | -| numba | lang | 2020-09-25 | \- | 0.47.0 | -| pixman | vis | 2020-02-19 | \- | 0.38.4\<br />0.38.0\<br />0.34.0 | -| pkg-config | devel | 2020-10-14 | \- | 0.29.2 | -| pkgconfig | devel | 2020-02-18 | \- | 1.5.1\<br />1.3.1 | -| pocl | lib | 2020-04-22 | \- | 1.4 | -| protobuf | devel | 2020-01-24 | \- | 3.6.1.2\<br />3.6.1\<br />3.10.0 | -| protobuf-python | devel | 2020-10-30 | \- | 3.10.0 | -| pybind11 | lib | 2020-11-02 | \- | 2.4.3 | -| re2c | tools | 2019-09-11 | \- | 1.1.1 | -| rstudio | lang | 2020-01-21 | \- | 1.2.5001 | -| scikit-image | vis | 2020-09-29 | \- | 0.16.2 | -| scikit-learn | data | 2020-09-24 | \- | 0.21.3 | -| snappy | lib | 2020-10-30 | \- | 1.1.7 | -| spleeter | tools | 2020-10-05 | \- | 1.5.4 | -| torchvision | vis | 2021-03-11 | \- | 0.7.0 | -| tqdm | lib | 2020-09-29 | \- | 4.41.1 | -| typing-extensions | devel | 2021-04-10 | \- | 3.7.4.3 | -| util-linux | tools | 2020-11-03 | \- | 2.35\<br />2.34\<br />2.33\<br />2.32.1\<br />2.32\<br />2.31.1 | -| wheel | tools | 2019-01-30 | \- | 0.31.1 | -| x264 | vis | 2020-06-24 | \- | 20190925\<br />20181203 | -| x265 | vis | 2020-09-29 | \- | 3.2\<br />3.0 | -| xorg-macros | devel | 2020-10-14 | \- | 1.19.2 | -| zlib | lib | 2020-10-14 | \- | 1.2.11 | -| zsh | tools | 2021-01-06 | \- | 5.8 | diff --git a/twiki2md/root/PerformanceTools/IOTrack.md b/twiki2md/root/PerformanceTools/IOTrack.md deleted file mode 100644 index f20334c8ead2ae2fcb75e39a7b1096ec524e0cbc..0000000000000000000000000000000000000000 --- a/twiki2md/root/PerformanceTools/IOTrack.md +++ /dev/null @@ -1,27 +0,0 @@ -# Introduction - -IOTrack is a small tool developed at ZIH that tracks the I/O requests of -all processes and dumps a statistic per process at the end of the -program run. - -# How it works - -On taurus load the module via - - module load iotrack - -Then, instead of running your normal command, put "iotrack" in front of -it. So, - - python xyz.py arg1 arg2 - -changes to: - - iotrack python xyz.py arg1 arg2 - -# Technical Details - -The functionality is implemented in a library that is preloaded via -LD_PRELOAD. Thus, this will not work for static binaries. - --- Main.MichaelKluge - 2013-07-16 diff --git a/twiki2md/root/SoftwareDevelopment/PerformanceTools.md b/twiki2md/root/SoftwareDevelopment/PerformanceTools.md deleted file mode 100644 index eb353e8d62612b27761368ce9d6a6326918e1d2f..0000000000000000000000000000000000000000 --- a/twiki2md/root/SoftwareDevelopment/PerformanceTools.md +++ /dev/null @@ -1,15 +0,0 @@ -# Performance Tools - -- [Score-P](ScoreP) - tool suite for profiling, event tracing, and - online analysis of HPC applications -- [VampirTrace](VampirTrace) - recording performance relevant data at - runtime -- [Vampir](Vampir) - visualizing performance data from your program -- [Hardware performance counters - PAPI](PapiLibrary) - generic - performance counters -- [perf tools](PerfTools) - general performance statistic -- [IOTrack](IOTrack) - I/O statistics -- [EnergyMeasurement](EnergyMeasurement) - energy/power measurements - on taurus - --- Main.mark - 2009-12-16 diff --git a/twiki2md/root/SystemTaurus/EnergyMeasurement.md b/twiki2md/root/SystemTaurus/EnergyMeasurement.md deleted file mode 100644 index 607263056fd593b5f0ae62474a1c63961c8b31aa..0000000000000000000000000000000000000000 --- a/twiki2md/root/SystemTaurus/EnergyMeasurement.md +++ /dev/null @@ -1,310 +0,0 @@ -# Energy Measurement Infrastructure - -All nodes of the HPC machine Taurus are equipped with power -instrumentation that allow the recording and accounting of power -dissipation and energy consumption data. The data is made available -through several different interfaces, which will be described below. - -## System Description - -The Taurus system is split into two phases. While both phases are -equipped with energy-instrumented nodes, the instrumentation -significantly differs in the number of instrumented nodes and their -spatial and temporal granularity. - -### Phase 1 - -In phase one, the 270 Sandy Bridge nodes are equipped with node-level -power instrumentation that is stored in the Dataheap infrastructure at a -rate of 1Sa/s and further the energy consumption of a job is available -in SLURM (see below). - -### Phase 2 - -In phase two, all of the 1456 Haswell DLC nodes are equipped with power -instrumentation. In addition to the access methods of phase one, users -will also be able to access the measurements through a C API to get the -full temporal and spatial resolution, as outlined below: - -- ** Blade:**1 kSa/s for the whole node, includes both sockets, DRAM, - SSD, and other on-board consumers. Since the system is directly - water cooled, no cooling components are included in the blade - consumption. -- **Voltage regulators (VR):** 100 Sa/s for each of the six VR - measurement points, one for each socket and four for eight DRAM - lanes (two lanes bundled). - -The GPU blades of each Phase as well as the Phase I Westmere partition -also have 1 Sa/s power instrumentation but have a lower accuracy. - -HDEEM is now generally available on all nodes in the "haswell" -partition. - -## Summary of Measurement Interfaces - -| Interface | Sensors | Rate | Phase I | Phase II Haswell | -|:-------------------------------------------|:----------------|:--------------------------------|:--------|:-----------------| -| Dataheap (C, Python, VampirTrace, Score-P) | Blade, (CPU) | 1 Sa/s | yes | yes | -| HDEEM\* (C, Score-P) | Blade, CPU, DDR | 1 kSa/s (Blade), 100 Sa/s (VRs) | no | yes | -| HDEEM Command Line Interface | Blade, CPU, DDR | 1 kSa/s (Blade), 100 Sa/s (VR) | no | yes | -| SLURM Accounting (sacct) | Blade | Per Job Energy | yes | yes | -| SLURM Profiling (hdf5) | Blade | up to 1 Sa/s | yes | yes | - -Note: Please specify `-p haswell --exclusive` along with your job -request if you wish to use hdeem. - -## Accuracy - -HDEEM measurements have an accuracy of 2 % for Blade (node) -measurements, and 5 % for voltage regulator (CPU, DDR) measurements. - -## Command Line Interface - -The HDEEM infrastructure can be controlled through command line tools -that are made available by loading the **hdeem** module. They are -commonly used on the node under test to start, stop, and query the -measurement device. - -- **startHdeem**: Start a measurement. After the command succeeds, the - measurement data with the 1000 / 100 Sa/s described above will be - recorded on the Board Management Controller (BMC), which is capable - of storing up to 8h of measurement data. -- **stopHdeem**: Stop a measurement. No further data is recorded and - the previously recorded data remains available on the BMC. -- **printHdeem**: Read the data from the BMC. By default, the data is - written into a CSV file, whose name can be controlled using the - **-o** argument. -- **checkHdeem**: Print the status of the measurement device. -- **clearHdeem**: Reset and clear the measurement device. No further - data can be read from the device after this command is executed - before a new measurement is started. - -## Integration in Application Performance Traces - -The per-node power consumption data can be included as metrics in -application traces by using the provided metric plugins for Score-P (and -VampirTrace). The plugins are provided as modules and set all necessary -environment variables that are required to record data for all nodes -that are part of the current job. - -For 1 Sa/s Blade values (Dataheap): - -- [Score-P](ScoreP): use the module **`scorep-dataheap`** -- [VampirTrace](VampirTrace): use the module - **vampirtrace-plugins/power-1.1** - -For 1000 Sa/s (Blade) and 100 Sa/s (CPU{0,1}, DDR{AB,CD,EF,GH}): - -- [Score-P](ScoreP): use the module **\<span - class="WYSIWYG_TT">scorep-hdeem\</span>**\<br />Note: %ENDCOLOR%This - module requires a recent version of "scorep/sync-...". Please use - the latest that fits your compiler & MPI version.**\<br />** -- [VampirTrace](VampirTrace): not supported - -By default, the modules are set up to record the power data for the -nodes they are used on. For further information on how to change this -behavior, please use module show on the respective module. - - # Example usage with gcc - % module load scorep/trunk-2016-03-17-gcc-xmpi-cuda7.5 - % module load scorep-dataheap - % scorep gcc application.c -o application - % srun ./application - -Once the application is finished, a trace will be available that allows -you to correlate application functions with the component power -consumption of the parallel application. Note: For energy measurements, -only tracing is supported in Score-P/VampirTrace. The modules therefore -disables profiling and enables tracing, please use [Vampir](Vampir) to -view the trace. - -\<img alt="demoHdeem_high_low_vampir_3.png" height="262" -src="%ATTACHURL%/demoHdeem_high_low_vampir_3.png" width="695" /> - -%RED%Note<span class="twiki-macro ENDCOLOR"></span>: the power -measurement modules **`scorep-dataheap`** and **`scorep-hdeem`** are -dynamic and only need to be loaded during execution. However, -**`scorep-hdeem`** does require the application to be linked with a -certain version of Score-P. - -By default,** `scorep-dataheap`**records all sensors that are available. -Currently this is the total node consumption and for Phase II the CPUs. -**`scorep-hdeem`** also records all available sensors (node, 2x CPU, 4x -DDR) by default. You can change the selected sensors by setting the -environment variables: - - # For HDEEM - % export SCOREP_METRIC_HDEEM_PLUGIN=Blade,CPU* - # For Dataheap - % export SCOREP_METRIC_DATAHEAP_PLUGIN=localhost/watts - -For more information on how to use Score-P, please refer to the -[respective documentation](ScoreP). - -## Access Using Slurm Tools - -[Slurm](Slurm) maintains its own database of job information, including -energy data. There are two main ways of accessing this data, which are -described below. - -### Post-Mortem Per-Job Accounting - -This is the easiest way of accessing information about the energy -consumed by a job and its job steps. The Slurm tool `sacct` allows users -to query post-mortem energy data for any past job or job step by adding -the field `ConsumedEnergy` to the `--format` parameter: - - $> sacct --format="jobid,jobname,ntasks,submit,start,end,ConsumedEnergy,nodelist,state" -j 3967027 - JobID JobName NTasks Submit Start End ConsumedEnergy NodeList State - ------------ ---------- -------- ------------------- ------------------- ------------------- -------------- --------------- ---------- - 3967027 bash 2014-01-07T12:25:42 2014-01-07T12:25:52 2014-01-07T12:41:20 taurusi1159 COMPLETED - 3967027.0 sleep 1 2014-01-07T12:26:07 2014-01-07T12:26:07 2014-01-07T12:26:18 0 taurusi1159 COMPLETED - 3967027.1 sleep 1 2014-01-07T12:29:06 2014-01-07T12:29:06 2014-01-07T12:29:16 1.67K taurusi1159 COMPLETED - 3967027.2 sleep 1 2014-01-07T12:33:25 2014-01-07T12:33:25 2014-01-07T12:33:36 1.84K taurusi1159 COMPLETED - 3967027.3 sleep 1 2014-01-07T12:34:06 2014-01-07T12:34:06 2014-01-07T12:34:11 1.09K taurusi1159 COMPLETED - 3967027.4 sleep 1 2014-01-07T12:38:03 2014-01-07T12:38:03 2014-01-07T12:39:44 18.93K taurusi1159 COMPLETED - -The job consisted of 5 job steps, each executing a sleep of a different -length. Note that the ConsumedEnergy metric is only applicable to -exclusive jobs. - -### - -### Slurm Energy Profiling - -The `srun` tool offers several options for profiling job steps by adding -the `--profile` parameter. Possible profiling options are `All`, -`Energy`, `Task`, `Lustre`, and `Network`. In all cases, the profiling -information is stored in an hdf5 file that can be inspected using -available hdf5 tools, e.g., `h5dump`. The files are stored under -`/scratch/profiling/` for each job, job step, and node. A description of -the data fields in the file can be found -[here](http://slurm.schedmd.com/hdf5_profile_user_guide.html#HDF5). In -general, the data files contain samples of the current **power** -consumption on a per-second basis: - - $> srun -p sandy --acctg-freq=2,energy=1 --profile=energy sleep 10 - srun: job 3967674 queued and waiting for resources - srun: job 3967674 has been allocated resources - $> h5dump /scratch/profiling/jschuch/3967674_0_taurusi1073.h5 - [...] - DATASET "Energy_0000000002 Data" { - DATATYPE H5T_COMPOUND { - H5T_STRING { - STRSIZE 24; - STRPAD H5T_STR_NULLTERM; - CSET H5T_CSET_ASCII; - CTYPE H5T_C_S1; - } "Date_Time"; - H5T_STD_U64LE "Time"; - H5T_STD_U64LE "Power"; - H5T_STD_U64LE "CPU_Frequency"; - } - DATASPACE SIMPLE { ( 1 ) / ( 1 ) } - DATA { - (0): { - "", - 1389097545, # timestamp - 174, # power value - 1 - } - } - } - -## - -## Using the HDEEM C API - -Note: Please specify -p haswell --exclusive along with your job request -if you wish to use hdeem. - -Please download the offical documentation at \<font face="Calibri" -size="2"> [\<font -color="#0563C1">\<u>http://www.bull.com/download-hdeem-library-reference-guide\</u>\</font>](http://www.bull.com/download-hdeem-library-reference-guide)\</font> - -The HDEEM headers and sample code are made available via the hdeem -module. To find the location of the hdeem installation use - - % module show hdeem - ------------------------------------------------------------------- - /sw/modules/taurus/libraries/hdeem/2.1.9ms: - - conflict hdeem - module-whatis Load hdeem version 2.1.9ms - prepend-path PATH /sw/taurus/libraries/hdeem/2.1.9ms/include - setenv HDEEM_ROOT /sw/taurus/libraries/hdeem/2.1.9ms - ------------------------------------------------------------------- - -You can find an example of how to use the API under -\<span>$HDEEM_ROOT/sample.\</span> - -## Access Using the Dataheap Infrastructure - -In addition to the energy accounting data that is stored by Slurm, this -information is also written into our local data storage and analysis -infrastructure called -[Dataheap](http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/projekte/dataheap/). -From there, the data can be used in various ways, such as including it -into application performance trace data or querying through a Python -interface. - -The Dataheap infrastructure is designed to store various types of -time-based samples from different data sources. In the case of the -energy measurements on Taurus, the data is stored as a timeline of power -values which allows the reconstruction of the power and energy -consumption over time. The timestamps are stored as UNIX timestamps with -a millisecond granularity. The data is stored for each node in the form -of `nodename/watts`, e.g., `taurusi1073/watts`. Further metrics might -already be available or might be added in the future for which -information is available upon request. - -**Note**: The dataheap infrastructure can only be accessed from inside -the university campus network. - -### Using the Python Interface - -The module `dataheap/1.0` provides a Python module that can be used to -query the data in the Dataheap for personalized data analysis. The -following is an example of how to use the interface: - - import time - import os - from dhRequest import dhClient - - # Connect to the dataheap manager - dhc = dhClient() - dhc.connect(os.environ['DATAHEAP_MANAGER_ADDR'], int(os.environ['DATAHEAP_MANAGER_PORT'])) - - # take timestamps - tbegin = dhc.getTimeStamp() - # workload - os.system("srun -n 6 a.out") - tend = dhc.getTimeStamp() - - # wait for the data to get to the - # dataheap - time.sleep(5) - - # replace this with name of the node the job ran on - # Note: use multiple requests if the job used multiple nodes - countername = "taurusi1159/watts" - - # query the dataheap - integral = dhc.storageRequest("INTEGRAL(%d,%d,\"%s\", 0)"%(tbegin, tend, countername)) - # Remember: timestamps are stored in millisecond UNIX timestamps - energy = integral/1000 - - print energy - - timeline = dhc.storageRequest("TIMELINE(%d,%d,\"%s\", 0)"%(tbegin, tend, countername)) - - # output a list of all timestamp/power-value pairs - print timeline - -## More information and Citing - -More information can be found in the paper \<a -href="<http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7016382>" -title="HDEEM Paper E2SC 2014">HDEEM: high definition energy efficiency -monitoring\</a> by Hackenberg et al. Please cite this paper if you are -using HDEEM for your scientific work. diff --git a/twiki2md/root/SystemTaurus/RunningNxGpuAppsInOneJob.md b/twiki2md/root/SystemTaurus/RunningNxGpuAppsInOneJob.md deleted file mode 100644 index 2152522aeaa03d97841a8a9ca1d5ed844a1a0449..0000000000000000000000000000000000000000 --- a/twiki2md/root/SystemTaurus/RunningNxGpuAppsInOneJob.md +++ /dev/null @@ -1,85 +0,0 @@ -# Running Multiple GPU Applications Simultaneously in a Batch Job - -Keywords: slurm, job, gpu, multiple, instances, application, program, -background, parallel, serial, concurrently, simultaneously - -## Objective - -Our starting point is a (serial) program that needs a single GPU and -four CPU cores to perform its task (e.g. TensorFlow). The following -batch script shows how to run such a job on the Taurus partition called -"ml". - - #!/bin/bash - #SBATCH --ntasks=1 - #SBATCH --cpus-per-task=4 - #SBATCH --gres=gpu:1 - #SBATCH --gpus-per-task=1 - #SBATCH --time=01:00:00 - #SBATCH --mem-per-cpu=1443 - #SBATCH --partition=ml - - srun some-gpu-application - -When srun is used within a submission script, it inherits parameters -from sbatch, including --ntasks=1, --cpus-per-task=4, etc. So we -actually implicitly run the following - - srun --ntasks=1 --cpus-per-task=4 ... --partition=ml some-gpu-application - -Now, our goal is to run four instances of this program concurrently in a -**single** batch script. Of course we could also start the above script -multiple times with sbatch, but this is not what we want to do here. - -## Solution - -In order to run multiple programs concurrently in a single batch -script/allocation we have to do three things: - -1\. Allocate enough resources to accommodate multiple instances of our -program. This can be achieved with an appropriate batch script header -(see below). - -2\. Start job steps with srun as background processes. This is achieved -by adding an ampersand at the end of the srun command - -3\. Make sure that each background process gets its private resources. -We need to set the resource fraction needed for a single run in the -corresponding srun command. The total aggregated resources of all job -steps must fit in the allocation specified in the batch script header. -Additionally, the option --exclusive is needed to make sure that each -job step is provided with its private set of CPU and GPU resources. - -The following example shows how four independent instances of the same -program can be run concurrently from a single batch script. Each -instance (task) is equipped with 4 CPUs (cores) and one GPU. - - #!/bin/bash - #SBATCH --ntasks=4 - #SBATCH --cpus-per-task=4 - #SBATCH --gres=gpu:4 - #SBATCH --gpus-per-task=1 - #SBATCH --time=01:00:00 - #SBATCH --mem-per-cpu=1443 - #SBATCH --partition=ml - - srun --exclusive --gres=gpu:1 --ntasks=1 --cpus-per-task=4 --gpus-per-task=1 --mem-per-cpu=1443 some-gpu-application & - srun --exclusive --gres=gpu:1 --ntasks=1 --cpus-per-task=4 --gpus-per-task=1 --mem-per-cpu=1443 some-gpu-application & - srun --exclusive --gres=gpu:1 --ntasks=1 --cpus-per-task=4 --gpus-per-task=1 --mem-per-cpu=1443 some-gpu-application & - srun --exclusive --gres=gpu:1 --ntasks=1 --cpus-per-task=4 --gpus-per-task=1 --mem-per-cpu=1443 some-gpu-application & - - echo "Waiting for all job steps to complete..." - wait - echo "All jobs completed!" - -In practice it is possible to leave out resource options in srun that do -not differ from the ones inherited from the surrounding sbatch context. -The following line would be sufficient to do the job in this example: - - srun --exclusive --gres=gpu:1 --ntasks=1 some-gpu-application & - -Yet, it adds some extra safety to leave them in, enabling the SLURM -scheduler to complain if not enough resources in total were specified in -the header of the batch script. - --- Main.HolgerBrunst - 2021-04-16 diff --git a/twiki2md/root/WebHome/Accessibility.md b/twiki2md/root/WebHome/Accessibility.md deleted file mode 100644 index 022418cf2d58baf9e223c90117e744752bcc762e..0000000000000000000000000000000000000000 --- a/twiki2md/root/WebHome/Accessibility.md +++ /dev/null @@ -1,54 +0,0 @@ -# Erklrung zur Barrierefreiheit - -Diese Erklrung zur Barrierefreiheit gilt fr die unter -<https://doc.zih.tu-dresden.de> verffentlichte Website der Technischen -Universitt Dresden. - -Als ffentliche Stelle im Sinne des Barrierefreie-Websites-Gesetz -(BfWebG) ist die Technische Universitt Dresden bemht, ihre Websites und -mobilen Anwendungen im Einklang mit den Bestimmungen des -Barrierefreie-Websites-Gesetz (BfWebG) in Verbindung mit der -Barrierefreie-Informationstechnik-Verordnung (BITV 2.0) barrierefrei -zugnglich zu machen. - -## Erstellung dieser Erklrung zur Barrierefreiheit - -Diese Erklrung wurde am 17.09.2020 erstellt und zuletzt am 17.09.2020 -aktualisiert. Grundlage der Erstellung dieser Erklrung zur -Barrierefreiheit ist eine am 17.09.2020 von der TU Dresden durchgefhrte -Selbstbewertung. - -## Stand der Barrierefreiheit - -Es wurde bisher noch kein BITV-Test fr die Website durchgefhrt. Dieser -ist bis 30.11.2020 geplant. - -## Kontakt - -Sollten Ihnen Mngel in Bezug auf die barrierefreie Gestaltung auffallen, -knnen Sie uns diese ber das Formular [Barriere -melden](https://tu-dresden.de/barrierefreiheit/barriere-melden) -mitteilen und im zugnglichen Format anfordern. Alternativ knnen Sie sich -direkt an die Meldestelle fr Barrieren wenden (Koordinatorin: Mandy -Weickert, E-Mail: <barrieren@tu-dresden.de>, Telefon: [+49 351 -463-42022](tel:+49-351-463-42022), Fax: [+49 351 -463-42021](tel:+49-351-463-42021), Besucheradresse: Nthnitzer Strae 46, -APB 1102, 01187 Dresden). - -## Durchsetzungsverfahren - -Wenn wir Ihre Rckmeldungen aus Ihrer Sicht nicht befriedigend -bearbeiten, knnen Sie sich an die Schsische Durchsetzungsstelle wenden: - -**Beauftragter der Schsischen Staatsregierung fr die Belange von -Menschen mit Behinderungen**\<br /> Albertstrae 10\<br /> 01097 Dresden - -Postanschrift: Archivstrae 1, 01097 Dresden\<br /> E-Mail: -<info.behindertenbeauftragter@sk.sachsen.de>\<br /> Telefon: [+49 351 -564-12161](tel:+49-351-564-12161)\<br /> Fax: [+49 351 -564-12169](tel:+49-351-564-12169)\<br /> Webseite: -<https://www.inklusion.sachsen.de> - -\<div id="footer"> \</div> - --- Main.MatthiasKraeusslein - 2020-09-18 diff --git a/twiki2md/root/WebHome/FurtherDocumentation.md b/twiki2md/root/WebHome/FurtherDocumentation.md deleted file mode 100644 index 2e6586a43633c20327fae0132b90d3fe73c1bb7c..0000000000000000000000000000000000000000 --- a/twiki2md/root/WebHome/FurtherDocumentation.md +++ /dev/null @@ -1,81 +0,0 @@ -# Further Documentation - - - -## Libraries and Compiler - -- <http://www.intel.com/software/products/mkl/index.htm> -- <http://www.intel.com/software/products/ipp/index.htm> -- <http://www.ball-project.org/> -- <http://www.intel.com/software/products/compilers/> - Intel Compiler - Suite -- <http://www.pgroup.com/doc> - PGI Compiler -- <http://pathscale.com/ekopath.html> - PathScale Compilers - -## Tools - -- <http://www.allinea.com/downloads/userguide.pdf> - Allinea DDT - Manual -- <http://www.totalviewtech.com/support/documentation.html> - - Totalview Documentation -- <http://www.gnu.org/software/gdb/documentation/> - GNU Debugger -- <http://vampir-ng.de> - official homepage of Vampir, an outstanding - tool for performance analysis developed at ZIH. -- <http://www.fz-juelich.de/zam/kojak/> - homepage of KOJAK at the FZ - Jlich. Parts of this project are used by Vampirtrace. -- <http://www.intel.com/software/products/threading/index.htm> - -## OpenMP - -You will find a lot of information at the following web pages: - -- <http://www.openmp.org> -- <http://www.compunity.org> - -## MPI - -The following sites may be interesting: - -- <http://www.mcs.anl.gov/mpi/> - the MPI homepage. -- <http://www.mpi-forum.org/> - Message Passing Interface (MPI) Forum - Home Page -- <http://www.open-mpi.org/> - the dawn of a new standard for a more - fail-tolerant MPI. -- The manual for SGI-MPI (installed on Mars ) can be found at: - -<http://techpubs.sgi.com/library/manuals/3000/007-3773-003/pdf/007-3773-003.pdf> - -## SGI developer forum - -The web sites behind -<http://www.sgi.com/developers/resources/tech_pubs.html> are full of -most detailed information on SGI systems. Have a look onto the section -'Linux Publications'. You will be redirected to the public part of SGI's -technical publication repository. - -- Linux Application Tuning Guide -- Linux Programmer's Guide, The -- Linux Device Driver Programmer's Guide -- Linux Kernel Internals.... and more. - -## Intel Itanium - -There is a lot of additional material regarding the Itanium CPU: - -- <http://www.intel.com/design/itanium/manuals/iiasdmanual.htm> -- <http://www.intel.com/design/archives/processors/itanium/index.htm> -- <http://www.intel.com/design/itanium2/documentation.htm> - -You will find the following manuals: - -- Intel Itanium Processor Floating-Point Software Assistance handler - (FPSWA) -- Intel Itanium Architecture Software Developer's Manuals Volume 1: - Application Architecture -- Intel Itanium Architecture Software Developer's Manuals Volume 2: - System Architecture -- Intel Itanium Architecture Software Developer's Manuals Volume 3: - Instruction Set -- Intel Itanium 2 Processor Reference Manual for Software Development - and Optimization -- Itanium Architecture Assembly Language Reference Guide diff --git a/twiki2md/root/WebHome/TypicalProjectSchedule.md b/twiki2md/root/WebHome/TypicalProjectSchedule.md deleted file mode 100644 index c7b404ea81e20010d505392e1a67caec71f8b678..0000000000000000000000000000000000000000 --- a/twiki2md/root/WebHome/TypicalProjectSchedule.md +++ /dev/null @@ -1,546 +0,0 @@ -# Typical project schedule - - - -## \<span style="font-size: 1em;">0. Application for HPC login\</span> - -In order to use the HPC systems installed at ZIH, a project application -form has to be filled in. The HPC project manager should hold a -professorship (university) or head a research group. You may also apply -for the "Schnupperaccount" (trial account) for one year. Check the -[Access](Access) page for details. - -## \<span style="font-size: 1em;">1. Request for resources\</span> - -Important note: Taurus is based on the Linux system. Thus for the -effective work, you should have to know how to work with -[Linux](https://en.wikipedia.org/wiki/Linux) based systems and [Linux -Shell](https://ubuntu.com/tutorials/command-line-for-beginners#1-overview). -Beginners can find a lot of different tutorials on the internet, [for -example](https://swcarpentry.github.io/shell-novice/). - -### \<span style="font-size: 1em;">1.1 How do I determine the required CPU / GPU hours?\</span> - -Taurus is focused on data-intensive computing. The cluster is oriented -on the work with the high parallel code. Please keep it in mind for the -transfer sequential code from a local machine. So far you will have -execution time for the sequential program it is reasonable to use -[Amdahl's law](https://en.wikipedia.org/wiki/Amdahl%27s_law) to roughly -predict execution time in parallel. Think in advance about the -parallelization strategy for your project. - -### \<span style="font-size: 1em;">1.2 What software do I need? What is already available (in the correct version)?\</span> - -The good practice for the HPC clusters is use software and packages -where parallelization is possible. The open-source software is more -preferable than proprietary. However, the majority of popular -programming languages, scientific applications, software, packages -available or could be installed on Taurus in different ways. First of -all, check the [Software module list](SoftwareModulesList). There are -two different software environments: **scs5** (the regular one) and -**ml** (environment for the Machine Learning partition). Keep in mind -that Taurus has a Linux based operating system. - -## 2. Access to the cluster - -### SSH access - -%RED%Important note:%ENDCOLOR%\<span style="font-size: 1em;"> ssh to -Taurus is only possible from \</span> **inside** \<span -style="font-size: 1em;"> TU Dresden Campus. Users from outside should -use \</span> **VPN** \<span style="font-size: 1em;"> (see \</span>\<a -href="<https://tu-dresden.de/zih/dienste/service-katalog/arbeitsumgebung/zugang_datennetz/vpn>" -target="\_top">here\</a>\<span style="font-size: 1em;">).\</span> - -The recommended way to connect to the HPC login servers directly via -ssh: - - ssh <zih-login>@taurus.hrsk.tu-dresden.de - -Please put this command in the terminal and replace \<zih-login> with -your login that you received during the access procedure. Accept the -host verifying and enter your password. You will be loaded by login -nodes in your Taurus home directory. - -This method requires two conditions: Linux OS, workstation within the -campus network. For other options and details check the \<a href="Login" -target="\_blank">Login page\</a>. - -Useful links: [Access](Access), [Project Request -Form](ProjectRequestForm), [Terms Of Use](TermsOfUse) - -## 3. Available software, use of the software - -According to 1.2, first of all, check the [Software module -list](SoftwareModulesList). Keep in mind that there are two different -environments: **scs5** (for the x86 architecture) and **ml** -(environment for the Machine Learning partition based on the Power9 -architecture). - -\<span style="font-size: 1em;">Work with the software on Taurus could be -started only after allocating the resources by \</span> [batch -systems](BatchSystems)\<span style="font-size: 1em;">. By default, you -are in the login nodes. They are not specified for the work, only for -the login. Allocating resources will be done by batch system \</span> -[SLURM](Slurm). - -There are a lot of different possibilities to work with software on -Taurus: - -**a.** **Modules** - -\<span style="font-size: 1em;">The easiest way to start working with -software is using the \</span>\<a -href="RuntimeEnvironment#Module_Environments" target="\_blank">Modules -system\</a>\<span style="font-size: 1em;">. Modules are a way to use -frameworks, compilers, loader, libraries, and utilities. The module is a -user interface that provides utilities for the dynamic modification of a -user's module environment without manual modifications. You could use -them for **srun**, bath jobs (**sbatch**) and the Jupyterhub.\</span> - -**b. JupyterNotebook** - -The Jupyter Notebook is an open-source web application that allows -creating documents containing live code, equations, visualizations, and -narrative text. \<span style="font-size: 1em;">There is \</span>\<a -href="JupyterHub" target="\_self">jupyterhub\</a>\<span -style="font-size: 1em;"> on Taurus, where you can simply run your -Jupyter notebook on HPC nodes using modules, preloaded or custom virtual -environments. Moreover, you can run a [manually created remote jupyter -server](DeepLearning#Jupyter_notebook) for more specific cases.\</span> - -**c.** **Containers** - -\<span style="font-size: 1em;">Some tasks require using containers. It -can be done on Taurus by [Singularity](https://sylabs.io/). Details -could be found in the [following -chapter](TypicalProjectSchedule#Use_of_containers).\</span> - -Useful links: [Libraries](Libraries), [Deep Learning](DeepLearning), -[Jupyter Hub](JupyterHub), [Big Data -Frameworks](BigDataFrameworks:ApacheSparkApacheFlinkApacheHadoop), -[R](DataAnalyticsWithR), [Applications for various fields of -science](Applications) - -## 4. Create a project structure. Data management - -Correct organisation of the project structure is a straightforward way -to the efficient work of the whole team. There have to be rules and -regulations for working with the project that every member should -follow. \<span style="font-size: 1em;">The uniformity of the project -could be achieved by using for each member of a team: the same **data -storage** or set of them, the same **set of software** (packages, -libraries etc), **access rights** to project data should be taken into -account and set up correctly. \</span> - -### 4.1 Data storage and management - -#### 4.1.1 Taxonomy of file systems - -\<span style="font-size: 1em;">As soon as you have access to Taurus you -have to manage your data. The main [concept](HPCStorageConcept2019) of -working with data on Taurus is using [Workspaces](WorkSpaces). Use it -properly:\</span> - -- use a `/home` directory for the limited amount of personal data, - simple examples and the results of calculations. The home directory - is not a working directory! However, \<span - class="WYSIWYG_TT">/home\</span> file system is backed up using - snapshots; -- use **workspace** as a place for working data (i.e. datasets); - Recommendations of choosing the correct storage system for workspace - presented below. - -**Recommendations to choose of storage system:**\<span -style`"font-size: 1em;">For data that seldomly changes but consumes a lot of space, the </span> ==warm_archive=` -\<span style="font-size: 1em;"> can be used. (Note that this is -\</span>mounted** read-only**\<span style="font-size: 1em;">on the -compute nodes). For a series of calculations that works on the same data -please use a \</span> **scratch**\<span style="font-size: 1em;">based -workspace. \</span> **SSD** \<span style="font-size: 1em;">, in its -turn, is the fastest available file system made only for large parallel -applications running with millions of small I/O (input, output -operations).\</span>\<span style="font-size: 1em;"> If the batch job -needs a directory for temporary data then -\</span>**\<span>SSD\</span>**\<span style="font-size: 1em;"> is a good -choice as well. The data can be deleted afterwards.\</span> - -Note: Keep in mind that every working space has a storage duration ( -i.e. ssd - 30 days). Thus be careful with the expire date otherwise it -could vanish. The core data of your project should be [backed -up](FileSystems#Backup_and_snapshots_of_the_file_system) and -[archived](PreservationResearchData)(for the most -[important](https://www.dcc.ac.uk/guidance/how-guides/five-steps-decide-what-data-keep) -data). - -#### \<span style="font-size: 1em;">4.1.2 Backup \</span> - -\<span -style`"font-size: 1em;">The backup is a crucial part of any project. Organize it at the beginning of the project. If you will lose/delete your data in the "no back up" file systems it can not be restored! The b</span><span style="font-size: 13px;">ackup on Taurus is </span><b style="font-size: 1em;">only </b><span style="font-size: 13px;">available in the </span> =/home` -\<span style`"font-size: 13px;"> and the </span> =/projects` \<span -style="font-size: 13px;"> file systems! Backed up files could be -restored by the user. Details could be found -[here](FileSystems#Backup_and_snapshots_of_the_file_system).\</span> - -#### 4.1.3 Folder structure and organizing data - -\<span style="font-size: 1em;">Organizing of living data using the file -system helps for consistency and structuredness of the project. -\</span>\<span style="font-size: 1em;">We recommend following the rules -for your work regarding:\</span> - -- Organizing the data: Never change the original data; Automatize the - organizing the data; Clearly separate intermediate and final output - in the filenames; Carry identifier and original name along in your - analysis pipeline; Make outputs clearly identifiable; Document your - analysis steps. -- Naming Data: Keep s\<span style="font-size: 1em;">hort, but - meaningful names; \</span>\<span style="font-size: 1em;">Keep - standard file endings; File names dont replace documentation and - metadata; Use standards of your discipline; \</span>\<span - style="font-size: 1em;">Make rules for your project, document and - keep them (See the \</span> [README - recommendations](TypicalProjectSchedule#README_recommendation) - below) - -\<span style="font-size: 1em;">This is the example of an organisation -(hierarchical) for the folder structure. Use it as a visual illustration -of the above:\</span> - -\<img align="justify" alt="Organizing_Data-using_file_systems.png" -height="161" src="%ATTACHURL%/Organizing_Data-using_file_systems.png" -title="Organizing_Data-using_file_systems.png" width="300" /> - -Keep in mind [input-process-output -pattern](https://en.wikipedia.org/wiki/IPO_model#Programming) for the -work with folder structure. - -#### 4.1.4 README recommendation - -In general, [README](https://en.wikipedia.org/wiki/README) is just -simple general information of software/project that exists in the same -directory/repository of the project. README is used to explain the -details project and the **structure** of the project/folder in a short -way. We recommend using readme as for entire project as for every -important folder in the project. - -Example of the structure for the README:\<br />\<span style="font-size: -1em;">Think first: What is calculated why? (Description); \</span>\<span -style="font-size: 1em;">What is expected? (software and version)\<br -/>\</span>\<span style="font-size: 1em;">Example text file\<br -/>\</span>\<span style="font-size: 1em;"> Title:\<br />\</span>\<span -style="font-size: 1em;"> User:\<br />\</span>\<span style="font-size: -1em;"> Date:\<br />\</span>\<span style="font-size: 1em;"> -Description:\<br />\</span>\<span style="font-size: 1em;"> software:\<br -/>\</span>\<span style="font-size: 1em;"> version:\</span> - -#### 4.1.5 Metadata - -Another important aspect is the -[Metadata](http://dublincore.org/resources/metadata-basics/). It is -sufficient to use -[Metadata](PreservationResearchData#Why_should_I_add_Meta_45Data_to_my_data_63) -for your project on Taurus. [Metadata -standards](https://en.wikipedia.org/wiki/Metadata_standard) will help to -do it easier (i.e. [Dublin core](https://dublincore.org/), -[OME](https://www.openmicroscopy.org/)) - -#### 4.1.6 Data hygiene - -Don't forget about data hygiene: Classify your current data into -critical (need it now), necessary (need it later) or unnecessary -(redundant, trivial or obsolete); Track and classify data throughout its -lifecycle (from creation, storage and use to sharing, archiving and -destruction); Erase the data you dont need throughout its lifecycle. - -### \<span style="font-size: 1em;">4.2 Software packages\</span> - -As was written before the module concept is the basic concept for using -software on Taurus. Uniformity of the project has to be achieved by -using the same set of software on different levels. It could be done by -using environments. There are two types of environments should be -distinguished: runtime environment (the project level, use scripts to -load [modules](RuntimeEnvironment)), Python virtual environment. The -concept of the environment will give an opportunity to use the same -version of the software on every level of the project for every project -member. - -#### Private individual and project modules files - -[Private individual and project module -files](RuntimeEnvironment#Private_Project_Module_Files)\<span -style="font-size: 1em;"> will be discussed in [chapter -](TypicalProjectSchedule#A_7._Use_of_specific_software_40packages_44_libraries_44_etc_41)\</span> -[7](TypicalProjectSchedule#A_7._Use_of_specific_software_40packages_44_libraries_44_etc_41)\<span -style="font-size: 1em;">. Project modules list is a powerful instrument -for effective teamwork.\</span> - -#### Python virtual environment - -If you are working with the Python then it is crucial to use the virtual -environment on Taurus. The main purpose of Python virtual environments -(don't mess with the software environment for modules) is to create an -isolated environment for Python projects (self-contained directory tree -that contains a Python installation for a particular version of Python, -plus a number of additional packages). - -**Vitualenv (venv)** is a standard Python tool to create isolated Python -environments. We recommend using venv to work with Tensorflow and -Pytorch on Taurus. It has been integrated into the standard library -under the \<a href="<https://docs.python.org/3/library/venv.html>" -target="\_blank">venv module\</a>. **Conda** is the second way to use a -virtual environment on the Taurus. \<a -href="<https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html>" -target="\_blank">Conda\</a> is an open-source package management system -and environment management system from the Anaconda. - -[Detailed information](Python#Virtual_environment) about using the -virtual environment. - -### \<span style="font-size: 1em;">4.3 Application software availability\</span> - -Software created for the purpose of the project should be available for -all members of the group. The instruction of how to use the software: -installation of packages, compilation etc should be documented and gives -the opportunity to comfort efficient and safe work. - -### 4.4 Access rights - -The concept of **permissions** and **ownership** is crucial in Linux. -See the -[HPC-introduction](%PUBURL%/Compendium/WebHome/HPC-Introduction.pdf?t=1602081321) -slides for the understanding of the main concept. Standard Linux -changing permission command(i.e `chmod`) valid for Taurus as well. The -**group** access level contains members of your project group. Be -careful with 'write' permission and never allow to change the original -data. - -Useful links: [Data Management](DataManagement), [File -Systems](FileSystems), [Get Started with HPC-DA](GetStartedWithHPCDA), -[Project Management](ProjectManagement), [Preservation research -data](PreservationResearchData) - -## 5. Data moving - -### 5.1 Moving data to/from the HPC machines - -To copy data to/from the HPC machines, the Taurus [export -nodes](ExportNodes) should be used as a preferred way. There are three -possibilities to exchanging data between your local machine (lm) and the -HPC machines (hm):\<span> **SCP, RSYNC, SFTP**. \</span>\<span -style`"font-size: 1em;">Type following commands in the terminal of the local machine. The </span> ==SCP=` -\<span style="font-size: 1em;"> command was used for the following -example.\</span> - -#### Copy data from lm to hm - - scp <file> <zih-user>@taurusexport.hrsk.tu-dresden.de:<target-location> #Copy file from your local machine. For example: scp helloworld.txt mustermann@taurusexport.hrsk.tu-dresden.de:/scratch/ws/mastermann-Macine_learning_project/ - - scp -r <directory> <zih-user>@taurusexport.hrsk.tu-dresden.de:<target-location> #Copy directory from your local machine. - -#### Copy data from hm to lm - - scp <zih-user>@taurusexport.hrsk.tu-dresden.de:<file> <target-location> #Copy file. For example: scp mustermann@taurusexport.hrsk.tu-dresden.de:/scratch/ws/mastermann-Macine_learning_project/helloworld.txt /home/mustermann/Downloads - - scp -r <zih-user>@taurusexport.hrsk.tu-dresden.de:<directory> <target-location> #Copy directory - -### 5.2 Moving data inside the HPC machines. Datamover - -The best way to transfer data inside the Taurus is the \<a -href="DataMover" target="\_blank">datamover\</a>. It is the special data -transfer machine provides the best data speed. To load, move, copy etc. -files from one file system to another file system, you have to use -commands with **dt** prefix, such as: **\<span>dtcp, dtwget, dtmv, dtrm, -dtrsync, dttar, dtls. \</span>**\<span style="font-size: 1em;">These -commands submit a job to the data transfer machines that execute the -selected command. Except for the '\</span>\<span style="font-size: -1em;">dt'\</span>\<span style="font-size: 1em;"> prefix, their syntax is -the same as the shell command without the '\</span>\<span -style="font-size: 1em;">dt\</span>\<span style="font-size: -1em;">'\</span>**.** - -Keep in mind: The warm_archive is not writable for jobs. However, you -can store the data in the warm archive with the datamover. - -Useful links: [Data Mover](DataMover), [Export Nodes](ExportNodes) - -## 6. Use of hardware - -To run the software, do some calculations or compile your code compute -nodes have to be used. Login nodes which are using for login can not be -used for your computations. Submit your tasks (by using -[jobs](https://en.wikipedia.org/wiki/Job_(computing))) to compute nodes. -The [SLURM](Slurm) (scheduler to handle your jobs) is using on Taurus -for this purposes. [HPC -Introduction](%PUBURL%/Compendium/WebHome/HPC-Introduction.pdf) is a -good resource to get started with it. - -### 6.1 What do I need a CPU or GPU? - -The main difference between CPU and GPU architecture is that a CPU is -designed to handle a wide range of tasks quickly, but are limited in the -concurrency of tasks that can be running. While GPUs can process data -much faster than a CPU due to massive parallelism (but the amount of -data which single GPU's core can handle is small), GPUs are not as -versatile as CPUs. - -### 6.2 Selection of suitable hardware - -Available [hardware](HardwareTaurus): Normal compute nodes (Haswell\[ -[64,128,256](SystemTaurus#Run_45time_and_Memory_Limits)\], Broadwell, -[Rome](RomeNodes)), Large [SMP nodes](SDFlex), Accelerator(GPU) nodes: -(gpu2 partition, [ml partition](Power9)). - -The exact partition could be specified by `-p` flag with the srun -command or in your batch job. - -Majority of the basic task could be done on the conventional nodes like -a Haswell. SLURM will automatically select a suitable partition -depending on your memory and --gres (gpu) requirements. If you do not -specify the partition most likely you will be addressed to the Haswell -partition (1328 nodes in total). - -#### Parallel jobs: - -**MPI jobs**: For MPI jobs typically allocates one core per task. -Several nodes could be allocated if it is necessary. SLURM will -automatically find suitable hardware. Normal compute nodes are perfect -for this task. - -**OpenMP jobs**: An SMP-parallel job can only run **within a node**, so -it is necessary to include the options **-N 1** and **-n 1**. Using ---cpus-per-task N SLURM will start one task and you will have N CPUs. -The maximum number of processors for an SMP-parallel program is 896 on -Taurus ( [SMP](SDFlex) island). - -**GPUs** partitions are best suited for **repetitive** and -**highly-parallel** computing tasks. If you have a task with potential -[data -parallelism](https://en.wikipedia.org/wiki/Data_parallelism#:~:text=Data%20parallelism%20is%20parallelization%20across,on%20each%20element%20in%20parallel.) -most likely that you need the GPUs. Beyond video rendering, GPUs excel -in tasks such as machine learning, financial simulations and risk -modelling. Use the gpu2 and ml partition only if you need GPUs! -Otherwise using the x86 partitions (e.g Haswell) most likely would be -more beneficial. - -**Interactive jobs**: SLURM can forward your X11 credentials to the -first (or even all) node for a job with the --x11 option. To use an -interactive job you have to specify -X flag for the ssh login. - -### 6.3 Interactive vs. sbatch - -However, using srun directly on the shell will lead to blocking and -launch an interactive job. Apart from short test runs, it is -**recommended to launch your jobs into the background by using batch -jobs**. For that, you can conveniently put the parameters directly into -the job file which you can submit using `sbatch` \[options\] \<job -file>. - -### 6.4 Processing of data for input and output - -Pre-processing and post-processing of the data is a crucial part for the -majority of data-dependent projects. The quality of this work influence -on the computations. However, pre- and post-processing in many cases can -be done completely or partially on a local pc and then -[transferred](TypicalProjectSchedule#A_5._Data_moving) to the Taurus. -Please use Taurus for the computation-intensive tasks. - -Useful links: [Batch Systems](BatchSystems), [Hardware -Taurus](HardwareTaurus), [HPC-DA](HPCDA), [Slurm](Slurm) - -## 7. Use of specific software (packages, libraries, etc) - -### 7.1 Modular system - -The modular concept is the easiest way to work with the software on -Taurus. It allows to user to switch between different versions of -installed programs and provides utilities for the dynamic modification -of a user's environment. The information can be found -[here](RuntimeEnvironment#Modules). - -#### Private project and user modules files - -[Private project module -files](RuntimeEnvironment#Private_Project_Module_Files)\<span -style="font-size: 1em;"> allow you to load your group-wide installed -software into your environment and to handle different versions. It -allows creating your own software environment for the project. You can -create a list of modules that will be loaded for every member of the -team. It gives opportunity on unifying work of the team and defines the -reproducibility of results. Private modules can be loaded like other -modules with \</span>\<span class="WYSIWYG_TT">module load\</span>\<span -style="font-size: 1em;">.\</span> - -[Private user module -files](RuntimeEnvironment#Private_User_Module_Files) allow you to load -your own installed software into your environment. It works in the same -manner as to project modules but for your private use. - -### 7.2 Use of containers - -[Containerization](https://www.ibm.com/cloud/learn/containerization) -encapsulating or packaging up software code and all its dependencies to -run uniformly and consistently on any infrastructure. On Taurus -[Singularity](https://sylabs.io/) used as a standard container solution. -Singularity enables users to have full control of their environment. -This means that you dont have to ask an HPC support to install anything -for you - you can put it in a Singularity container and run! As opposed -to Docker (the most famous container solution), Singularity is much more -suited to being used in an HPC environment and more efficient in many -cases. Docker containers can easily be used in Singularity. Information -about the use of Singularity on Taurus can be found [here](Container). - -\<span style="font-size: 1em;">In some cases using Singularity requires -a Linux machine with root privileges (e.g. using the ml partition), the -same architecture and a compatible kernel. For many reasons, users on -Taurus cannot be granted root permissions. A solution is a Virtual -Machine (VM) on the ml partition which allows users to gain root -permissions in an isolated environment. There are two main options on -how to work with VM on Taurus:\<br />\</span>\<span style="font-size: -1em;">1. \</span> [VM tools](VMTools)\<span style="font-size: 1em;">. -Automative algorithms for using virtual machines;\<br />\</span>\<span -style="font-size: 1em;">2. \</span> [Manual method](Cloud)\<span -style="font-size: 1em;">. It required more operations but gives you more -flexibility and reliability.\<br />\</span>\<span style="font-size: -1em;">Additional Information: Examples of the definition for the -Singularity container (\</span> -[here](SingularityExampleDefinitions)\<span style="font-size: 1em;">) -and some hints (\</span> [here](SingularityRecipeHints)\<span -style="font-size: 1em;">).\</span> - -Useful links: [Containers](Container), [Custom EasyBuild -Environment](CustomEasyBuildEnvironment), [Cloud](Cloud) - -## 8. Structuring experiments - -- \<p>Input data\</p> -- \<p>Calculation results\</p> -- \<p>Log files\</p> -- \<p>Submission scripts (examples / code for survival)\</p> - -## What if everything didn't help? - -### Create a ticket: how do I do that? - -The best way to ask about the help is to create a ticket. In order to do -that you have to write a message to the <hpcsupport@zih.tu-dresden.de> -with a detailed description of your problem. If it possible please add -logs, used environment and write a minimal executable example for the -purpose to recreate the error or issue. - -### \<span style="font-size: 1em;">Communication with HPC support\</span> - -There is the HPC support team who is responsible for the support of HPC -users and stable work of the cluster. You could find the -[details](https://tu-dresden.de/zih/hochleistungsrechnen/support) in the -right part of any page of the compendium. However, please, before the -contact with the HPC support team check the documentation carefully -(starting points: [ main page](WebHome), [HPC-DA](HPCDA)), use a -[search](WebSearch) and then create a ticket. The ticket is a preferred -way to solve the issue, but in some terminable cases, you can call to -ask for help. - -Useful link: [Further Documentation](FurtherDocumentation) - -\<span style="font-size: 1em;">-- Main.AndreiPolitov - -2020-09-14\</span>