hpc-compendium issueshttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues2024-03-28T12:28:09+01:00https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/571Verweis auf SlurmGenerator in abaqus-slurm.pdf2024-03-28T12:28:09+01:00Sebastian DöbelVerweis auf SlurmGenerator in abaqus-slurm.pdfIn `doc.zih.tu-dresden.de/docs/software/misc/abaqus-slurm.pdf` wird auf https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/Slurmgenerator verwiesen. Der Link funktioniert nicht mehr.In `doc.zih.tu-dresden.de/docs/software/misc/abaqus-slurm.pdf` wird auf https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/Slurmgenerator verwiesen. Der Link funktioniert nicht mehr.https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/570alte Hardware aus der Git-Historie raussuchen2024-03-27T08:48:31+01:00Danny Marc Rotscheralte Hardware aus der Git-Historie raussuchenMich interessieren erstmal die Knotennamen zur folgenden CPU, meine Vermutung ist `taurusi[1001-1270]`, aber irgendwie steht in der Outage mehr drin.
`Intel Xeon E5-2680`Mich interessieren erstmal die Knotennamen zur folgenden CPU, meine Vermutung ist `taurusi[1001-1270]`, aber irgendwie steht in der Outage mehr drin.
`Intel Xeon E5-2680`Martin SchroschkMartin Schroschkhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/569Automate for commands examples2024-03-05T15:41:36+01:00Guilherme Pantoja Calandrini de Azevedo JuniorAutomate for commands examplesCreate a script for parsing the .md files, get what should be run/where and execute it at the specific cluster.
Save independent files with the results.Create a script for parsing the .md files, get what should be run/where and execute it at the specific cluster.
Save independent files with the results.Guilherme Pantoja Calandrini de Azevedo JuniorGuilherme Pantoja Calandrini de Azevedo Juniorhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/568How do we deal with new/future software?2024-03-04T09:17:51+01:00Jan FrenzelHow do we deal with new/future software?The new software installation cycle dictates new software only in April and October. Thus, we should also plan ahead the documentation of the software accordingly. In April, the module `development/24.04` is renamed to `release/24.04`, I...The new software installation cycle dictates new software only in April and October. Thus, we should also plan ahead the documentation of the software accordingly. In April, the module `development/24.04` is renamed to `release/24.04`, I assume. Thus, if we document software with `development/24.04`, we need to change it in April again. Alternatively, we could directly use `release/24.04` and postpone the publication of the changes. This would require the following steps:
- Add a new `next-release` branch, where we prepare changes in software documentation that need to be made once the new release is out.
- Then, we can merge it to `main` at the same time we make the new release available on all clusters (or to `preview` one week in advance when we want to look at it earlier).
- We could automate this merge with pipeline schedules if we know the exact date.
- We need to document the approach in the contribution guide, so that contributors select whether it is an urgent change (MR with target `preview`) or a planned change (MR with target `next-release`).
- We could use the half-yearly checks to also look for old releases and update if required.Martin SchroschkMartin Schroschkhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/567Update Big Data Analytics Page2024-03-02T14:50:26+01:00Apurv Deepak KulkarniUpdate Big Data Analytics PageBig Data Analytics Page needs updating, as few things have changed.
https://doc.zih.tu-dresden.de/software/big_data_frameworks/Big Data Analytics Page needs updating, as few things have changed.
https://doc.zih.tu-dresden.de/software/big_data_frameworks/Apurv Deepak KulkarniApurv Deepak Kulkarnihttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/566Page "CI/CD on HPC" should also document how to set up a group runner2024-02-29T10:17:46+01:00Jan FrenzelPage "CI/CD on HPC" should also document how to set up a group runnerDescribe how runners can be configured for a Gitlab (project) group. Configuration of group runners is fundamentally different from that of project runners.Describe how runners can be configured for a Gitlab (project) group. Configuration of group runners is fundamentally different from that of project runners.https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/565Update required: "CI/CD on HPC" page still uses haswell and ml2024-02-29T10:06:23+01:00Jan FrenzelUpdate required: "CI/CD on HPC" page still uses haswell and mlUpdate the page "CI/CD on HPC", so that it uses "barnard", "alpha", etc.Update the page "CI/CD on HPC", so that it uses "barnard", "alpha", etc.https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/563JupyterLab access alternative2024-02-26T08:36:40+01:00Danny Marc RotscherJupyterLab access alternativeWe described as workaround access to JupyterLab via X11 forwarding.
One user gave us an example how to access JupyterLab via port forwarding.
**In one Terminal Window:**
```console
marie@local$
ssh -XC marie@login4.barnard.hpc.tu-dresd...We described as workaround access to JupyterLab via X11 forwarding.
One user gave us an example how to access JupyterLab via port forwarding.
**In one Terminal Window:**
```console
marie@local$
ssh -XC marie@login4.barnard.hpc.tu-dresden.de
marie@login4.barnard$
ml release/23.10 GCCcore/12.2.0
ml Python/3.10.8
source /software/util/JupyterLab/vis/jupyterlab-4.0.4/bin/activate
export SLURM_CONF=/software/util/dcv/etc/slurm/slurm.conf
srun --nodes=1 --ntasks=1 --cpus-per-task=4 --mem-per-cpu=8192 --x11 --pty bash -l
marie@vis1.barnard$ jupyter lab --no-browser
```
As the jupyter lab starts it will print something like
[C 2024-02-25 19:01:53.163 ServerApp]
Or copy and paste one of these URLs:
http://localhost:8888/lab?token=d7759aea743fc848891bf2eb7e390f6d63be42fbbc92c859
To connect with our local web browser we need to forward a local port (e.g. 8888) to vis1.barnard.hpc.tu-dresden.de:8888
Therefore, in a second terminal window:
```console
marie@local$ ssh marie@vis1.barnard.hpc.tu-dresden.de -L 8888:localhost:8888
```
Finally, open a local web browser and navigate to
http://localhost:8888/lab?token=d7759aea743fc848891bf2eb7e390f6d63be42fbbc92c859
Would be good to have both alternatives to JupyterHub.Danny Marc RotscherDanny Marc Rotscherhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/562How to get CPU/GPU quotas for a project?2024-02-14T12:22:25+01:00Maximilian KnespelHow to get CPU/GPU quotas for a project?On Taurus, there is `showquota`, which show something like:
```
CPU-Quotas as of 2024-02-14 10:52
Project Used(h) Quota(h) % Comment
zihforschung 0 41667 0.0
GPU-Quotas as of 2024-02-1...On Taurus, there is `showquota`, which show something like:
```
CPU-Quotas as of 2024-02-14 10:52
Project Used(h) Quota(h) % Comment
zihforschung 0 41667 0.0
GPU-Quotas as of 2024-02-14 10:52
Project Used(h) Quota(h) % Comment
zihforschung 0 0 0.0 Limit reached (SOFT) *
* Job priority is minimal for this project
```
The compendium [still mentions it](https://doc.zih.tu-dresden.de/data_lifecycle/permanent/?h=quot#quotas) but it doesn't exist on Barnard.
This issue is different from #561 because I am interested in CPU/GPU project quotas, not disk quotas.
On the old system, it seems `showquota` was a custom script in `/usr/local/bin/showquota`, which reads out `/var/lib/zihacct/accounting`, which also does not exist on Barnard.
Is there any other way to query the project quota and used CPU/GPU hours, e.g., via sacct or sacctmgr or another custom script?https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/561Show Quota of /home2024-02-14T12:22:25+01:00Sebastian DöbelShow Quota of /home`doc.zih.tu-dresden.de/docs/data_lifecycle/permanent.md` states to get quota of `/home` by `quota -s -f /home`
Although I get
```
➜ showquota
zsh: command not found: showquota
➜ quota -s -f /home
quota: Mountpoint (or device) /home n...`doc.zih.tu-dresden.de/docs/data_lifecycle/permanent.md` states to get quota of `/home` by `quota -s -f /home`
Although I get
```
➜ showquota
zsh: command not found: showquota
➜ quota -s -f /home
quota: Mountpoint (or device) /home not found or has no quota enabled.
quota: Not all specified mountpoints are using quota.
➜ quota -v
Disk quotas for user s8916149 (uid 2108144):
Filesystem blocks quota limit grace files quota limit grace
141.76.10.15:/trcdata
0 107374182400 107374182400 0 50000000 50000000
141.76.10.12:/hrsk_projecthome/projects
16451428 9663676416 10737418240 37 4294967295 4294967295
```
I think we activated quota on `/home`? So whats the correct way to get these information?https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/560Update videos2024-02-07T17:35:54+01:00Martin SchroschkUpdate videosVideos in https://doc.zih.tu-dresden.de/software/virtual_desktops/#demonstration are showing the old URL.Videos in https://doc.zih.tu-dresden.de/software/virtual_desktops/#demonstration are showing the old URL.Guilherme Pantoja Calandrini de Azevedo JuniorGuilherme Pantoja Calandrini de Azevedo Juniorhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/559Slurm job file generator uses old partition names2024-01-29T09:48:16+01:00Jan FrenzelSlurm job file generator uses old partition namesPlease update Slurm Job File Generator. Remove the partition option?Please update Slurm Job File Generator. Remove the partition option?https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/557SINGULARITY_TMPDIR und SINGULARITY_CACHEDIR dokumentieren2024-01-12T10:17:17+01:00Danny Marc RotscherSINGULARITY_TMPDIR und SINGULARITY_CACHEDIR dokumentierenAuf Barnard haben wir ja nun auch Diskless-Knoten, die nur über ein kleines /tmp-Verzeichnis verfügen. Deshalb sollte man die Umgebungsvariable `SINGULARITY_TMPDIR` auf ein Verzeichnis im Workspace setzen.
`SINGULARITY_CACHEDIR` zeigt au...Auf Barnard haben wir ja nun auch Diskless-Knoten, die nur über ein kleines /tmp-Verzeichnis verfügen. Deshalb sollte man die Umgebungsvariable `SINGULARITY_TMPDIR` auf ein Verzeichnis im Workspace setzen.
`SINGULARITY_CACHEDIR` zeigt auch in Home-Verzeichnis vom Nutzer, will man vielleicht auch ins ein Workspace schieben.
https://docs.sylabs.io/guides/latest/user-guide/build_env.html#build-environmenthttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/556Mail-Template für Slurm-Reservierungen anpassen2024-01-12T07:40:57+01:00Danny Marc RotscherMail-Template für Slurm-Reservierungen anpassenEs wäre wünschenswert, gleich die Infos so an zu fordern, wie wir diese auch für das Reservierungs-Kommando benötigen.
Also mindestens das Zeitformat anpassen und noch besser wäre es die komplette Zeile vor zu geben.Es wäre wünschenswert, gleich die Infos so an zu fordern, wie wir diese auch für das Reservierungs-Kommando benötigen.
Also mindestens das Zeitformat anpassen und noch besser wäre es die komplette Zeile vor zu geben.Danny Marc RotscherDanny Marc Rotscherhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/555Analytics / Usage Statistics2024-01-10T16:26:40+01:00Martin SchroschkAnalytics / Usage Statistics* Site analytics via google analytics: https://squidfunk.github.io/mkdocs-material/setup/setting-up-site-analytics/
* Custom (non-google) analytics providers are also possible: https://squidfunk.github.io/mkdocs-material/setup/setting-up...* Site analytics via google analytics: https://squidfunk.github.io/mkdocs-material/setup/setting-up-site-analytics/
* Custom (non-google) analytics providers are also possible: https://squidfunk.github.io/mkdocs-material/setup/setting-up-site-analytics/#custom-site-analytics
## What is wanted?
* Internal site views (from within TU network)
* External site viewsMartin SchroschkMartin Schroschkhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/554Private easybuild modules?2024-01-10T14:06:45+01:00Jan FrenzelPrivate easybuild modules?Should we document how to create and use private modules with easybuild? https://doc.zih.tu-dresden.de/software/private_modules/ only documents non-Easybuild private modules.Should we document how to create and use private modules with easybuild? https://doc.zih.tu-dresden.de/software/private_modules/ only documents non-Easybuild private modules.https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/551Workspace extension code snippet uses old `scratch`, but should use e. g. `ho...2024-01-08T13:58:49+01:00Jan FrenzelWorkspace extension code snippet uses old `scratch`, but should use e. g. `horse`https://doc.zih.tu-dresden.de/data_lifecycle/workspaces/#extension-of-a-workspacehttps://doc.zih.tu-dresden.de/data_lifecycle/workspaces/#extension-of-a-workspacehttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/550Document possibility to suppress login message via .noshowjobsonlogin2024-01-08T10:08:55+01:00Jan FrenzelDocument possibility to suppress login message via .noshowjobsonloginhttps://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/549Slurm: document time limit2024-01-08T09:14:49+01:00Martin SchroschkSlurm: document time limitMax. time limit
* is not set via partition (`MaxTime=unlimited`)
* is set via QOS ( 7 days)Max. time limit
* is not set via partition (`MaxTime=unlimited`)
* is set via QOS ( 7 days)Martin SchroschkMartin Schroschk2024-01-12https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues/548doc.zih.tu-dresden.de/docs/accessibility.md (foot)2024-01-05T13:53:33+01:00Chunzhi Wudoc.zih.tu-dresden.de/docs/accessibility.md (foot)