diff --git a/Dockerfile b/Dockerfile
index 731e831c9b2fc1ff1068ae2b2a80c04bbf0039c7..f6bf9841524472c7af2522ce9cd641e9c5dbd824 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -4,9 +4,7 @@ FROM python:3.8-bullseye
# Base #
########
-COPY ./ /src/
-
-RUN pip install -r /src/doc.zih.tu-dresden.de/requirements.txt
+RUN pip install mkdocs>=1.1.2 mkdocs-material>=7.1.0
##########
# Linter #
@@ -16,6 +14,6 @@ RUN apt update && apt install -y nodejs npm aspell
RUN npm install -g markdownlint-cli markdown-link-check
-WORKDIR /src/doc.zih.tu-dresden.de
+WORKDIR /docs
CMD ["mkdocs", "build", "--verbose", "--strict"]
diff --git a/doc.zih.tu-dresden.de/README.md b/doc.zih.tu-dresden.de/README.md
index e7de9cd2eed1f6f72183047886655d522846bc34..dc65e18d33561b7e1bc5e50d73f8bfd4085f3f27 100644
--- a/doc.zih.tu-dresden.de/README.md
+++ b/doc.zih.tu-dresden.de/README.md
@@ -40,13 +40,7 @@ Now, create a local clone of your fork
#### Install Dependencies
-**TODO:** Description
-
-```Shell Session
-~ cd hpc-compendium/doc.zih.tu-dresden.de
-~ pip install -r requirements.txt
-```
-
+See [Installation with Docker](#preview-using-mkdocs-with-dockerfile).
**TODO:** virtual environment
**TODO:** What we need for markdownlinter and checks?
diff --git a/doc.zih.tu-dresden.de/docs/contrib/content_rules.md b/doc.zih.tu-dresden.de/docs/contrib/content_rules.md
index f5492e7f35ff26e425bff9c7b246f7c0d4a29fb0..b6b21e2315f7bbc6b81775f4c38a8021ddda48d5 100644
--- a/doc.zih.tu-dresden.de/docs/contrib/content_rules.md
+++ b/doc.zih.tu-dresden.de/docs/contrib/content_rules.md
@@ -51,6 +51,10 @@ should be highlighted, etc. Code examples, longer than half screen height should
## Writing Style
* Capitalize headings, e.g. *Exclusive Reservation of Hardware*
+* Use active over passive voice
+ * Write with confidence. This confidence should be reflected in the documentation, so that
+ the readers trust and follow it.
+ * Example: `We recommend something` instead of `Something is recommended.`
## Spelling and Technical Wording
@@ -64,6 +68,12 @@ there is a list of conventions w.r.t. spelling and technical wording.
* `ZIH system` and `ZIH systems` not `Taurus` etc. if possible
* `Workspace` not `work space`
+### Long Options
+
+* Use long over short options, e.g. `srun --nodes=2 --ntasks-per-node=4 ...` is preferred over
+ `srun -N 2 -n 4 ...`
+* Use `module` over the short front-end `ml` in documentation and examples
+
## Code Blocks and Command Prompts
Showing commands and sample output is an important part of all technical documentation. To make
diff --git a/doc.zih.tu-dresden.de/docs/contrib/howto_contribute.md b/doc.zih.tu-dresden.de/docs/contrib/howto_contribute.md
index 31105a5208932ff49ee86d939ed8faa744dad854..e9a1a20833cead3533ad08303ac8445c7ee54b0f 100644
--- a/doc.zih.tu-dresden.de/docs/contrib/howto_contribute.md
+++ b/doc.zih.tu-dresden.de/docs/contrib/howto_contribute.md
@@ -4,14 +4,21 @@
Ink is better than the best memory.
+In principle, there are three possible ways how to contribute to this documentation.
+
## Contribute via Issue
Users can contribute to the documentation via the
[GitLab issue tracking system](https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium/-/issues).
For that, open an issue to report typos and missing documentation or request for more precise
-wording etc. ZIH staff will get in touch with you to resolve the issue and improve the
+wording etc. ZIH staff will get in touch with you to resolve the issue and improve the
documentation.
+??? tip "GIF: Create GitLab Issue"
+
+ 
+ {: align=center}
+
!!! warning "HPC support"
Non-documentation issues and requests need to be send as ticket to
@@ -20,8 +27,15 @@ documentation.
## Contribute via Web IDE
GitLab offers a rich and versatile web interface to work with repositories. To fix typos and edit
-source files, just select the file of interest and click the `Edit` button. A text and commit
-editor are invoked: Do your changes, add a meaningful commit message and commit the changes.
+source files, follow these steps:
+
+1. Navigate to the repository at
+[https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium](https://gitlab.hrz.tu-chemnitz.de/zih/hpcsupport/hpc-compendium)
+and log in.
+1. Select the right branch.
+1. Select the file of interest in `doc.zih.tu-dresden.de/docs/...` and click the `Edit` button.
+1. A text and commit editor are invoked: Do your changes, add a meaningful commit message and commit
+ the changes.
The more sophisticated integrated Web IDE is reached from the top level menu of the repository or
by selecting any source file.
@@ -29,12 +43,9 @@ by selecting any source file.
Other git services might have an equivalent web interface to interact with the repository. Please
refer to the corresponding documentation for further information.
-<!--This option of contributing is only available for users of-->
-<!--[gitlab.hrz.tu-chemnitz.de](https://gitlab.hrz.tu-chemnitz.de). Furthermore, -->
-
## Contribute Using Git Locally
For experienced Git users, we provide a Docker container that includes all checks of the CI engine
used in the back-end. Using them should ensure that merge requests will not be blocked
due to automatic checking.
-For details, see [Work Locally Using Containers](contribute_container.md).
+For details, refer to the page [Work Locally Using Containers](contribute_container.md).
diff --git a/doc.zih.tu-dresden.de/docs/contrib/misc/create_gitlab_issue.gif b/doc.zih.tu-dresden.de/docs/contrib/misc/create_gitlab_issue.gif
new file mode 100644
index 0000000000000000000000000000000000000000..cb4910897903283e43b21feadcdb2acf2f42c15e
Binary files /dev/null and b/doc.zih.tu-dresden.de/docs/contrib/misc/create_gitlab_issue.gif differ
diff --git a/doc.zih.tu-dresden.de/docs/software/virtual_machines_tools.md b/doc.zih.tu-dresden.de/docs/software/virtual_machines_tools.md
index 0b03ddf927aeed68d8726797ed04db373d24b9b3..fbec2e51bc453cc17e2d131d7229c50ff90aa23f 100644
--- a/doc.zih.tu-dresden.de/docs/software/virtual_machines_tools.md
+++ b/doc.zih.tu-dresden.de/docs/software/virtual_machines_tools.md
@@ -12,12 +12,12 @@ it to ZIH systems for execution.
**This does not work on the partition `ml`** as it uses the Power9 architecture which your
workstation likely doesn't.
-For this we provide a Virtual Machine (VM) on the partition `ml` which allows users to gain root
-permissions in an isolated environment. The workflow to use this manually is described at
-[this page](virtual_machines.md) but is quite cumbersome.
+For this, we provide a Virtual Machine (VM) on the partition `ml` which allows users to gain root
+permissions in an isolated environment. The workflow to use this manually is described for
+[virtual machines](virtual_machines.md) but is quite cumbersome.
-To make this easier two programs are provided: `buildSingularityImage` and `startInVM` which do what
-they say. The latter is for more advanced use cases so you should be fine using
+To make this easier, two programs are provided: `buildSingularityImage` and `startInVM`, which do
+what they say. The latter is for more advanced use cases, so you should be fine using
`buildSingularityImage`, see the following section.
!!! note "SSH key without password"
@@ -28,43 +28,48 @@ they say. The latter is for more advanced use cases so you should be fine using
**The recommended workflow** is to create and test a definition file locally. You usually start from
a base Docker container. Those typically exist for different architectures but with a common name
(e.g. `ubuntu:18.04`). Singularity automatically uses the correct Docker container for your current
-architecture when building. So in most cases you can write your definition file, build it and test
+architecture when building. So, in most cases, you can write your definition file, build it and test
it locally, then move it to ZIH systems and build it on Power9 (partition `ml`) without any further
changes. However, sometimes Docker containers for different architectures have different suffixes,
in which case you'd need to change that when moving to ZIH systems.
## Build a Singularity Container in a Job
-To build a Singularity container on ZIH systems simply run:
+To build a Singularity container for the power9-architecture on ZIH systems simply run:
```console
marie@login$ buildSingularityImage --arch=power9 myContainer.sif myDefinition.def
```
-This command will submit a batch job and immediately return. Note that while Power9 is currently
-the only supported architecture, the parameter is still required. If you want it to block while the
+To build a singularity image on the x86-architecture, run:
+
+```console
+marie@login$ buildSingularityImage --arch=x86 myContainer.sif myDefinition.def
+```
+
+These commands will submit a batch job and immediately return. If you want it to block while the
image is built and see live output, add the option `--interactive`:
```console
marie@login$ buildSingularityImage --arch=power9 --interactive myContainer.sif myDefinition.def
```
-There are more options available which can be shown by running `buildSingularityImage --help`. All
-have reasonable defaults.The most important ones are:
+There are more options available, which can be shown by running `buildSingularityImage --help`. All
+have reasonable defaults. The most important ones are:
* `--time <time>`: Set a higher job time if the default time is not
enough to build your image and your job is canceled before completing. The format is the same as
for Slurm.
* `--tmp-size=<size in GB>`: Set a size used for the temporary
- location of the Singularity container. Basically the size of the extracted container.
+ location of the Singularity container, basically the size of the extracted container.
* `--output=<file>`: Path to a file used for (log) output generated
while building your container.
* Various Singularity options are passed through. E.g.
`--notest, --force, --update`. See, e.g., `singularity --help` for details.
-For **advanced users** it is also possible to manually request a job with a VM (`srun -p ml
+For **advanced users**, it is also possible to manually request a job with a VM (`srun -p ml
--cloud=kvm ...`) and then use this script to build a Singularity container from within the job. In
-this case the `--arch` and other Slurm related parameters are not required. The advantage of using
+this case, the `--arch` and other Slurm related parameters are not required. The advantage of using
this script is that it automates the waiting for the VM and mounting of host directories into it
(can also be done with `startInVM`) and creates a temporary directory usable with Singularity inside
the VM controlled by the `--tmp-size` parameter.
@@ -73,21 +78,22 @@ the VM controlled by the `--tmp-size` parameter.
**Read here if you have problems like "File not found".**
-As the build starts in a VM you may not have access to all your files. It is usually bad practice
+As the build starts in a VM, you may not have access to all your files. It is usually bad practice
to refer to local files from inside a definition file anyway as this reduces reproducibility.
-However common directories are available by default. For others, care must be taken. In short:
+However, common directories are available by default. For others, care must be taken. In short:
-* `/home/$USER`, `/scratch/$USER` are available and should be used `/scratch/\<group>` also works for
-* all groups the users is in `/projects/\<group>` similar, but is read-only! So don't use this to
- store your generated container directly, but rather move it here afterwards
-* /tmp is the VM local temporary directory. All files put here will be lost!
+* `/home/$USER`, `/scratch/$USER` are available and should be used `/scratch/<group>` also works for
+ all groups the users is in
+* `/projects/<group>` similar, but is read-only! So don't use this to store your generated
+ container directly, but rather move it here afterwards
+* `/tmp` is the VM local temporary directory. All files put here will be lost!
If the current directory is inside (or equal to) one of the above (except `/tmp`), then relative paths
for container and definition work as the script changes to the VM equivalent of the current
-directory. Otherwise you need to use absolute paths. Using `~` in place of `$HOME` does work too.
+directory. Otherwise, you need to use absolute paths. Using `~` in place of `$HOME` does work too.
-Under the hood, the filesystem of ZIH systems is mounted via SSHFS at `/host_data`, so if you need any
-other files they can be found there.
+Under the hood, the filesystem of ZIH systems is mounted via SSHFS at `/host_data`. So if you need any
+other files, they can be found there.
There is also a new SSH key named `kvm` which is created by the scripts and authorized inside the VM
to allow for password-less access to SSHFS. This is stored at `~/.ssh/kvm` and regenerated if it
@@ -98,26 +104,32 @@ needs to be re-generated on every script run.
## Start a Job in a VM
-Especially when developing a Singularity definition file it might be useful to get a shell directly
-on a VM. To do so simply run:
+Especially when developing a Singularity definition file, it might be useful to get a shell directly
+on a VM. To do so on the power9-architecture, simply run:
```console
startInVM --arch=power9
```
+To do so on the x86-architecture, run:
+
+```console
+startInVM --arch=x86
+```
+
This will execute an `srun` command with the `--cloud=kvm` parameter, wait till the VM is ready,
mount all folders (just like `buildSingularityImage`, see the Filesystem section above) and come
back with a bash inside the VM. Inside that you are root, so you can directly execute `singularity
build` commands.
-As usual more options can be shown by running `startInVM --help`, the most important one being
+As usual, more options can be shown by running `startInVM --help`, the most important one being
`--time`.
There are two special use cases for this script:
1. Execute an arbitrary command inside the VM instead of getting a bash by appending the command to
- the script. Example: `startInVM --arch=power9 singularity build \~/myContainer.sif \~/myDefinition.de`
+ the script. Example: `startInVM --arch=power9 singularity build ~/myContainer.sif ~/myDefinition.de`
1. Use the script in a job manually allocated via srun/sbatch. This will work the same as when
running outside a job but will **not** start a new job. This is useful for using it inside batch
- scripts, when you already have an allocation or need special arguments for the job system. Again
+ scripts, when you already have an allocation or need special arguments for the job system. Again,
you can run an arbitrary command by passing it to the script.
diff --git a/doc.zih.tu-dresden.de/requirements.txt b/doc.zih.tu-dresden.de/requirements.txt
deleted file mode 100644
index 272b09c7c7ffb6b945eaa66e14e2e695f5502f17..0000000000000000000000000000000000000000
--- a/doc.zih.tu-dresden.de/requirements.txt
+++ /dev/null
@@ -1,6 +0,0 @@
-# Documentation static site generator & deployment tool
-mkdocs>=1.1.2
-
-# Add custom theme if not inside a theme_dir
-# (https://github.com/mkdocs/mkdocs/wiki/MkDocs-Themes)
-mkdocs-material>=7.1.0
diff --git a/doc.zih.tu-dresden.de/util/check-bash-syntax.sh b/doc.zih.tu-dresden.de/util/check-bash-syntax.sh
index e5681413d14771e8fb144a2684161cf5e7c1edae..9f31effee3ebc3380af5ca892047aca6a9357139 100755
--- a/doc.zih.tu-dresden.de/util/check-bash-syntax.sh
+++ b/doc.zih.tu-dresden.de/util/check-bash-syntax.sh
@@ -47,12 +47,12 @@ branch="origin/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-preview}"
if [ $all_files = true ]; then
echo "Search in all bash files."
- files=$(git ls-tree --full-tree -r --name-only HEAD $basedir/docs/ | grep .sh)
+ files=`git ls-tree --full-tree -r --name-only HEAD $basedir/docs/ | grep .sh || true`
elif [[ ! -z $file ]]; then
files=$file
else
echo "Search in git-changed files."
- files=`git diff --name-only "$(git merge-base HEAD "$branch")" | grep .sh`
+ files=`git diff --name-only "$(git merge-base HEAD "$branch")" | grep .sh || true`
fi