Skip to content
Snippets Groups Projects
Commit d2d7a7d2 authored by Gitlab Bot's avatar Gitlab Bot
Browse files

Merge branch 'preview' into merge-preview-in-main

parents 755b24b3 a38f0285
No related branches found
No related tags found
2 merge requests!603Issue285,!575Automated merge from preview to main
Showing
with 66 additions and 30 deletions
...@@ -6,7 +6,7 @@ SHELL ["/bin/bash", "-c"] ...@@ -6,7 +6,7 @@ SHELL ["/bin/bash", "-c"]
# Base # # Base #
######## ########
RUN pip install mkdocs>=1.1.2 mkdocs-material>=7.1.0 RUN pip install mkdocs>=1.1.2 mkdocs-material>=7.1.0 mkdocs-htmlproofer-plugin==0.8.0
########## ##########
# Linter # # Linter #
......
...@@ -188,11 +188,11 @@ review and will be activated in case of a positive result. ...@@ -188,11 +188,11 @@ review and will be activated in case of a positive result.
### Application ### Application
- [Project Type](#project-type) - [Project Type](#project-type)
- [Principal Investigator (PI) and Person to Contact (PC)](#principal-investigator) - [Principal Investigator (PI) and Person to Contact (PC)](#principal-investigator-pi-and-person-to-contact-pc)
- [Resources](#resources) - [Resources](#resources)
- [Project Data](#project-data) - [Project Data](#project-data)
- [Software](#software) - [Software](#software)
- [Data Management](#data_management) - [Data Management](#data-management)
- [Upload Full Project description](#upload-full-project-description) - [Upload Full Project description](#upload-full-project-description)
No matter which application you submit, the procedure is always identical. No matter which application you submit, the procedure is always identical.
......
...@@ -6,10 +6,10 @@ ...@@ -6,10 +6,10 @@
| Software | Module | | Software | Module |
|-----------------------------------|-------------------------------------------| |-----------------------------------|-------------------------------------------|
| **[Infernal](#Infernal)** | infernal | | **[Infernal](#infernal)** | infernal |
| **[OpenProspect](#OpenProspect)** | openprospect, openprospect/885-mpi | | **[OpenProspect](#openprospect)** | openprospect, openprospect/885-mpi |
| **[Phylip](#Phylip)** | phylip | | **[Phylip](#phylip)** | phylip |
| **[PhyML](#PhyML)** | phyml/2.4.4, phyml/2.4.5-mpi, phyml/3.0.0 | | **[PhyML](#phyml)** | phyml/2.4.4, phyml/2.4.5-mpi, phyml/3.0.0 |
## Infernal ## Infernal
......
...@@ -57,13 +57,24 @@ actively used, especially for longer code examples, warnings, tips, important in ...@@ -57,13 +57,24 @@ actively used, especially for longer code examples, warnings, tips, important in
should be highlighted, etc. Code examples, longer than half screen height should collapsed should be highlighted, etc. Code examples, longer than half screen height should collapsed
(and indented): (and indented):
??? example ??? "Collapsed example"
```Bash ```Bash
[...] [...]
# very long example here # very long example here
[...] [...]
``` ```
### Brief How-To on Markdown
| Purpose | Markdown | Rendered HTML |
|---|---|---|
| Bold text | `**Bold Text**` | **Bold Text** |
| Italic text | `*Italic Text*` | *Italic Text* |
| Headings | `# Level 1`, `## Level 2`, `### Level 3`, ... | |
| External link | `[website of TU Dresden](https://tu-dresden.de)` | [website of TU Dresden](https://tu-dresden.de) |
| Internal link | `[Slurm page](../jobs_and_resources/slurm.md)` | [Slurm page](../jobs_and_resources/slurm.md)|
| Internal link to section | `[section on batch jobs](../jobs_and_resources/slurm.md#batch-jobs)` | [section on batch jobs](../jobs_and_resources/slurm.md#batch-jobs) |
## Writing Style ## Writing Style
* Avoid using tabs both in markdown files and in `mkdocs.yaml`. Type spaces instead. * Avoid using tabs both in markdown files and in `mkdocs.yaml`. Type spaces instead.
...@@ -130,7 +141,7 @@ We follow this rules regarding prompts: ...@@ -130,7 +141,7 @@ We follow this rules regarding prompts:
* **Always use a prompt**, even there is no output provided for the shown command. * **Always use a prompt**, even there is no output provided for the shown command.
* All code blocks should use long parameter names (e.g. Slurm parameters), if available. * All code blocks should use long parameter names (e.g. Slurm parameters), if available.
* All code blocks which specify some general command templates, e.g. containing `<` and `>` * All code blocks which specify some general command templates, e.g. containing `<` and `>`
(see [Placeholders](#mark-placeholders)), should use `bash` for the code block. Additionally, (see [Placeholders](#placeholders)), should use `bash` for the code block. Additionally,
an example invocation, perhaps with output, should be given with the normal `console` code block. an example invocation, perhaps with output, should be given with the normal `console` code block.
See also [Code Block description below](#code-blocks-and-syntax-highlighting). See also [Code Block description below](#code-blocks-and-syntax-highlighting).
* Using some magic, the prompt as well as the output is identified and will not be copied! * Using some magic, the prompt as well as the output is identified and will not be copied!
...@@ -157,7 +168,7 @@ bar ...@@ -157,7 +168,7 @@ bar
Make sure that shell session and console code blocks are executable on the login nodes of HPC system. Make sure that shell session and console code blocks are executable on the login nodes of HPC system.
Command templates use [Placeholders](#mark-placeholders) to mark replaceable code parts. Command Command templates use [Placeholders](#placeholders) to mark replaceable code parts. Command
templates should give a general idea of invocation and thus, do not contain any output. Use a templates should give a general idea of invocation and thus, do not contain any output. Use a
`bash` code block followed by an invocation example (with `console`): `bash` code block followed by an invocation example (with `console`):
...@@ -258,6 +269,10 @@ drwxr-xr-x 3 marie p_number_crunch 4096 Feb 12 2020 data ...@@ -258,6 +269,10 @@ drwxr-xr-x 3 marie p_number_crunch 4096 Feb 12 2020 data
-rw-rw---- 1 marie p_number_crunch 4096 Jan 24 2020 readme.md -rw-rw---- 1 marie p_number_crunch 4096 Jan 24 2020 readme.md
``` ```
### Placeholders
TODO!
## Mark Omissions ## Mark Omissions
If showing only a snippet of a long output, omissions are marked with `[...]`. If showing only a snippet of a long output, omissions are marked with `[...]`.
......
...@@ -200,9 +200,12 @@ wikiscript doc.zih.tu-dresden.de/util/check-no-floating.sh doc.zih.tu-dresden.de ...@@ -200,9 +200,12 @@ wikiscript doc.zih.tu-dresden.de/util/check-no-floating.sh doc.zih.tu-dresden.de
#### Link Checker #### Link Checker
No one likes dead links. Therefore, we check the internal and external links within the markdown !!! quote "Unknown programmer"
source files. To check a single file, e.g.
`doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md`, use: No one likes dead links.
Therefore, we check the internal and external links within the markdown source files. To check a
single file, e.g. `doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md`, use:
```bash ```bash
wikiscript doc.zih.tu-dresden.de/util/check-links.sh docs/software/big_data_frameworks.md wikiscript doc.zih.tu-dresden.de/util/check-links.sh docs/software/big_data_frameworks.md
......
...@@ -35,7 +35,7 @@ filesystems. ...@@ -35,7 +35,7 @@ filesystems.
!!! hint "Recommendations to choose of storage system" !!! hint "Recommendations to choose of storage system"
* For data that seldom changes but consumes a lot of space, the * For data that seldom changes but consumes a lot of space, the
[warm_archive](file_systems.md#warm_archive) can be used. [warm_archive](warm_archive.md) can be used.
(Note that this is mounted **read-only** on the compute nodes). (Note that this is mounted **read-only** on the compute nodes).
* For a series of calculations that works on the same data please use a `scratch` based * For a series of calculations that works on the same data please use a `scratch` based
[workspace](workspaces.md). [workspace](workspaces.md).
......
...@@ -137,7 +137,7 @@ the local machine. ...@@ -137,7 +137,7 @@ the local machine.
Windows 10 (1809 and higher) comes with a Windows 10 (1809 and higher) comes with a
[built-in OpenSSH support](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview) [built-in OpenSSH support](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview)
including the above described [SCP](#SCP) and [SFTP](#SFTP). including the above described [SCP](#scp) and [SFTP](#sftp).
### GUI - Using WinSCP ### GUI - Using WinSCP
......
...@@ -49,7 +49,7 @@ make it easier to utilize DMTCP together with Slurm. ...@@ -49,7 +49,7 @@ make it easier to utilize DMTCP together with Slurm.
## Using w.r.t. Chain Jobs ## Using w.r.t. Chain Jobs
For long-running jobs that you wish to split into multiple shorter jobs For long-running jobs that you wish to split into multiple shorter jobs
([chain jobs](../jobs_and_resources/slurm.md#chain-jobs)), thereby enabling the job scheduler to ([chain jobs](slurm_examples.md#chain-jobs)), thereby enabling the job scheduler to
fill the cluster much more efficiently and also providing some level of fault-tolerance, we have fill the cluster much more efficiently and also providing some level of fault-tolerance, we have
written a script that automatically creates a number of jobs for your desired runtime and adds the written a script that automatically creates a number of jobs for your desired runtime and adds the
checkpoint/restart bits transparently to your batch script. You just have to specify the targeted checkpoint/restart bits transparently to your batch script. You just have to specify the targeted
......
...@@ -6,9 +6,10 @@ research environment especially in the area of data analytics and machine learni ...@@ -6,9 +6,10 @@ research environment especially in the area of data analytics and machine learni
processing extremely large data sets. Moreover it is also a perfect platform for highly scalable, processing extremely large data sets. Moreover it is also a perfect platform for highly scalable,
data-intensive and compute-intensive applications. data-intensive and compute-intensive applications.
With shared [login nodes](#login-nodes) and [filesystems](../data_lifecycle/file_systems.md) our With shared [login nodes](hardware_overview.md#login-nodes) and
HPC system enables users to easily switch between [the components](hardware_overview.md), each [filesystems](../data_lifecycle/file_systems.md) our HPC system enables users to easily switch
specialized for different application scenarios. between [the components](hardware_overview.md), each specialized for different application
scenarios.
When log in to ZIH systems, you are placed on a login node where you can When log in to ZIH systems, you are placed on a login node where you can
[manage data life cycle](../data_lifecycle/overview.md), [manage data life cycle](../data_lifecycle/overview.md),
......
...@@ -33,7 +33,7 @@ environment. ...@@ -33,7 +33,7 @@ environment.
However, new containers can be created on your local workstation and moved to ZIH systems for However, new containers can be created on your local workstation and moved to ZIH systems for
execution. Follow the instructions for [locally installing Singularity](#local-installation) and execution. Follow the instructions for [locally installing Singularity](#local-installation) and
[container creation](#container-creation). Moreover, existing Docker container can easily be [container creation](#container-creation). Moreover, existing Docker container can easily be
converted, see [Import a docker container](#importing-a-docker-container). converted, see [Import a docker container](#import-a-docker-container).
If you are already familiar with Singularity, you might be more interested in our [singularity If you are already familiar with Singularity, you might be more interested in our [singularity
recipes and hints](singularity_recipe_hints.md). recipes and hints](singularity_recipe_hints.md).
......
...@@ -260,7 +260,7 @@ is limited to the number of cores on a single node. The maximum number of cores ...@@ -260,7 +260,7 @@ is limited to the number of cores on a single node. The maximum number of cores
be found in our [hardware documentation](../jobs_and_resources/hardware_overview.md). be found in our [hardware documentation](../jobs_and_resources/hardware_overview.md).
Submitting a multicore R job to Slurm is very similar to submitting an Submitting a multicore R job to Slurm is very similar to submitting an
[OpenMP Job](../jobs_and_resources/slurm.md#binding-and-distribution-of-tasks), [OpenMP Job](../jobs_and_resources/binding_and_distribution_of_tasks.md),
since both are running multicore jobs on a **single** node. Below is an example: since both are running multicore jobs on a **single** node. Below is an example:
```Bash ```Bash
...@@ -292,7 +292,7 @@ This way of the R parallelism uses the ...@@ -292,7 +292,7 @@ This way of the R parallelism uses the
[Rmpi](http://cran.r-project.org/web/packages/Rmpi/index.html) package and the [Rmpi](http://cran.r-project.org/web/packages/Rmpi/index.html) package and the
[MPI](https://en.wikipedia.org/wiki/Message_Passing_Interface) (Message Passing Interface) as a [MPI](https://en.wikipedia.org/wiki/Message_Passing_Interface) (Message Passing Interface) as a
"back-end" for its parallel operations. The MPI-based job in R is very similar to submitting an "back-end" for its parallel operations. The MPI-based job in R is very similar to submitting an
[MPI Job](../jobs_and_resources/slurm.md#binding-and-distribution-of-tasks) since both are running [MPI Job](../jobs_and_resources/binding_and_distribution_of_tasks.md) since both are running
multicore jobs on multiple nodes. Below is an example of running R script with the Rmpi on the ZIH multicore jobs on multiple nodes. Below is an example of running R script with the Rmpi on the ZIH
system: system:
......
...@@ -12,7 +12,7 @@ so called dotfiles in your home directory, e.g., `~/.bashrc` or `~/.bash_profile ...@@ -12,7 +12,7 @@ so called dotfiles in your home directory, e.g., `~/.bashrc` or `~/.bash_profile
## Software Environment ## Software Environment
There are different options to work with software on ZIH systems: [modules](#modules), There are different options to work with software on ZIH systems: [modules](#modules),
[Jupyter Notebook](#jupyternotebook) and [containers](#containers). Brief descriptions and related [Jupyter Notebook](#jupyter-notebook) and [containers](#containers). Brief descriptions and related
links on these options are provided below. links on these options are provided below.
!!! note !!! note
......
...@@ -96,7 +96,7 @@ the notebook by pre-loading a specific TensorFlow module: ...@@ -96,7 +96,7 @@ the notebook by pre-loading a specific TensorFlow module:
You can also define your own Jupyter kernel for more specific tasks. Please read about Jupyter You can also define your own Jupyter kernel for more specific tasks. Please read about Jupyter
kernels and virtual environments in our kernels and virtual environments in our
[JupyterHub](../access/jupyterhub.md#creating-and-using-your-own-environment) documentation. [JupyterHub](../access/jupyterhub.md#creating-and-using-a-custom-environment) documentation.
## TensorFlow in Containers ## TensorFlow in Containers
......
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
application. The ParaView package comprises different tools which are designed to meet interactive, application. The ParaView package comprises different tools which are designed to meet interactive,
batch and in-situ workflows. batch and in-situ workflows.
ParaView is available on ZIH systems from the [modules system](modules.md#modules-environment). The ParaView is available on ZIH systems from the [modules system](modules.md#module-environments). The
following command lists the available versions following command lists the available versions
```console ```console
......
...@@ -200,6 +200,27 @@ markdown_extensions: ...@@ -200,6 +200,27 @@ markdown_extensions:
# - mkdocs-video # - mkdocs-video
plugins:
- search
- markdown-caption
# https://github.com/manuzhang/mkdocs-htmlproofer-plugin
- htmlproofer:
# True by default
# Toggle via env. var. ENABLED_HTMLPROOFER=false
enabled: !ENV [ENABLED_HTMLPROOFER, True]
# Check URLs before serving. Build fails in case of bad URLs.
raise_error: True
# Ignore errors
#raise_error_excludes:
# 504: ['https://www.mkdocs.org/']
# Toggle validating external URLs
validate_external_urls: False
# Validate the entire rendered template, slow!
validate_rendered_template: True
# Enable cross-page anchor validation
use_directory_urls: False
extra: extra:
tud_homepage: https://tu-dresden.de tud_homepage: https://tu-dresden.de
tud_name: "TU Dresden" tud_name: "TU Dresden"
...@@ -218,8 +239,4 @@ extra: ...@@ -218,8 +239,4 @@ extra:
- link: /data_protection_declaration - link: /data_protection_declaration
name: "Data Protection Declaration / Datenschutzerklärung" name: "Data Protection Declaration / Datenschutzerklärung"
- link: https://tu-dresden.de/zertifikate - link: https://tu-dresden.de/zertifikate
name: "Certificates" name: "Certificates"
\ No newline at end of file
plugins:
- search
- markdown-caption
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment