Skip to content
Snippets Groups Projects
Commit 09d8b386 authored by Martin Schroschk's avatar Martin Schroschk
Browse files

Merge branch 'preview' of gitlab.hrz.tu-chemnitz.de:zih/hpcsupport/hpc-compendium into Filesystems

parents e3852a01 29ed9c5d
No related branches found
No related tags found
3 merge requests!322Merge preview into main,!319Merge preview into main,!234Aufgeteilt
Showing
with 245 additions and 330 deletions
...@@ -45,7 +45,7 @@ Check spelling for changed md-files: ...@@ -45,7 +45,7 @@ Check spelling for changed md-files:
stage: test stage: test
script: script:
- docker run --rm -w /src -e CI_MERGE_REQUEST_TARGET_BRANCH_NAME "${DOCKER_IMAGE}" - docker run --rm -w /src -e CI_MERGE_REQUEST_TARGET_BRANCH_NAME "${DOCKER_IMAGE}"
doc.zih.tu-dresden.de/util/check-spelling-changes.sh doc.zih.tu-dresden.de/util/check-spelling.sh
only: [ merge_requests ] only: [ merge_requests ]
Check links for changed md-files: Check links for changed md-files:
......
# JupyterHub # JupyterHub
With our JupyterHub service we offer you now a quick and easy way to With our JupyterHub service we offer you a quick and easy way to work with Jupyter notebooks on ZIH
work with jupyter notebooks on Taurus. systems. This page covers starting and stopping JuperterHub sessions, error handling and customizing
the environment.
Subpages: We also provide a comprehensive documentation on how to use
[JupyterHub for Teaching (git-pull feature, quickstart links, direct links to notebook files)](jupyterhub_for_teaching.md).
- [JupyterHub for Teaching (git-pull feature, quickstart links, direct
links to notebook files)](jupyterhub_for_teaching.md)
## Disclaimer ## Disclaimer
This service is provided "as-is", use at your own discretion. Please !!! warning
understand that JupyterHub is a complex software system of which we are
not the developers and don't have any downstream support contracts for, The JupyterHub service is provided *as-is*, use at your own discretion.
so we merely offer an installation of it but cannot give extensive
support in every case. Please understand that JupyterHub is a complex software system of which we are not the developers
and don't have any downstream support contracts for, so we merely offer an installation of it but
cannot give extensive support in every case.
## Access ## Access
<span style="color:red">**NOTE**</span> This service is only available for users with !!! note
an active HPC project. See [here](../access/overview.md) how to apply for an HPC This service is only available for users with an active HPC project.
project. See [here](../access/overview.md) how to apply for an HPC project.
JupyterHub is available here:\ JupyterHub is available at
<https://taurus.hrsk.tu-dresden.de/jupyter> [https://taurus.hrsk.tu-dresden.de/jupyter](https://taurus.hrsk.tu-dresden.de/jupyter).
## Start a session ## Start a Session
Start a new session by clicking on the **TODO ADD IMAGE** \<img alt="" height="24" Start a new session by clicking on the `Start my server` button.
src="%ATTACHURL%/start_my_server.png" /> button.
A form opens up where you can customize your session. Our simple form A form opens up where you can customize your session. Our simple form
offers you the most important settings to start quickly. offers you the most important settings to start quickly.
**TODO ADD IMAGE** \<a href="%ATTACHURL%/simple_form.png">\<img alt="session form" ![Simple form](misc/simple_form.png)
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/simple_form.png>" {: align="center"}
style="border: 1px solid #888;" title="simple form" width="400" />\</a>
For advanced users we have an extended form where you can change many For advanced users we have an extended form where you can change many
settings. You can: settings. You can:
- modify Slurm parameters to your needs ( [more about - modify batch system parameters to your needs ([more about batch system Slurm](../jobs_and_resources/slurm.md))
Slurm](../jobs_and_resources/slurm.md)) - assign your session to a project or reservation
- assign your session to a project or reservation - load modules from the [module system](../software/runtime_environment.md)
- load modules from the [LMOD module - choose a different standard environment (in preparation for future
system](../software/runtime_environment.md) software updates or testing additional features)
- choose a different standard environment (in preparation for future
software updates or testing additional features)
**TODO ADD IMAGE** \<a href="%ATTACHURL%/advanced_form_nov2019.png">\<img alt="session ![Advanced form](misc/advanced_form.png)
form" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/advanced_form_nov2019.png>"
style="border: 1px solid #888;" title="advanced form" width="400"
/>\</a>
You can save your own configurations as additional presets. Those are You can save your own configurations as additional presets. Those are
saved in your browser and are lost if you delete your browsing data. Use saved in your browser and are lost if you delete your browsing data. Use
the import/export feature (available through the button) to save your the import/export feature (available through the button) to save your
presets in text files. presets in text files.
Note: the [<span style="color:blue">**alpha**</span>] !!! info
(https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/AlphaCentauri) The partition [alpha](https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/AlphaCentauri)
partition is available only in the extended form. is available only in the extended form.
## Applications ## Applications
You can choose between JupyterLab or the classic notebook app. You can choose between JupyterLab or classic Jupyter notebooks as outlined in the following.
### JupyterLab ### JupyterLab
**TODO ADD IMAGE** \<a href="%ATTACHURL%/jupyterlab_app.png">\<img alt="jupyterlab app" ![JupyterLab overview](misc/jupyterlab_overview.png)
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/jupyterlab_app.png>" {: align="center"}
style="border: 1px solid #888;" title="JupyterLab overview" width="400"
/>\</a>
The main workspace is used for multiple notebooks, consoles or The main workspace is used for multiple notebooks, consoles or
terminals. Those documents are organized with tabs and a very versatile terminals. Those documents are organized with tabs and a very versatile
split screen feature. On the left side of the screen you can open split screen feature. On the left side of the screen you can open
several views: several views:
- file manager - file manager
- controller for running kernels and terminals - controller for running kernels and terminals
- overview of commands and settings - overview of commands and settings
- details about selected notebook cell - details about selected notebook cell
- list of open tabs - list of open tabs
### Classic notebook ### Classic Jupyter Notebook
**TODO ADD IMAGE** \<a href="%ATTACHURL%/jupyter_notebook_app_filebrowser.png">\<img Initially your `home` directory is listed. You can open existing notebooks or files by navigating to
alt="filebrowser in jupyter notebook server" width="400" the corresponding path and clicking on them.
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/jupyter_notebook_app_filebrowser.png>"
style="border: 1px solid #888;" title="Classic notebook (file browser)"
/>\</a>
**TODO ADD IMAGE** \<a href="%ATTACHURL%/jupyter_notebook_example_matplotlib.png">\<img ![Jupyter notebook file browser](misc/jupyter_notebook_file_browser.png)
alt="jupyter_notebook_example_matplotlib" width="400" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/jupyter_notebook_example_matplotlib.png>"
style="border: 1px solid #888;" title="Classic notebook (matplotlib
demo)" />\</a>
Initially you will get a list of your home directory. You can open Above the table on the right side is the button `New` which lets you create new notebooks, files,
existing notebooks or files by clicking on them. directories or terminals.
Above the table on the right side is the "New ⏷" button which lets you ![Jupyter notebook example matplotlib](misc/jupyter_notebook_example_matplotlib.png)
create new notebooks, files, directories or terminals. {: align="center"}
## The notebook ## Jupyter Notebooks in General
In JupyterHub you can create scripts in notebooks. In JupyterHub you can create scripts in notebooks. Notebooks are programs which are split into
Notebooks are programs which are split in multiple logical code blocks. multiple logical code blocks. In between those code blocks you can insert text blocks for
In between those code blocks you can insert text blocks for documentation and each block can be executed individually. Each notebook is paired with a kernel
documentation and each block can be executed individually. Each notebook running the code. We currently offer one for Python, C++, MATLAB and R.
is paired with a kernel which runs the code. We currently offer one for
Python, C++, MATLAB and R.
## Stop a session ## Stop a Session
It's good practise to stop your session once your work is done. This It is good practise to stop your session once your work is done. This releases resources for other
releases resources for other users and your quota is less charged. If users and your quota is less charged. If you just log out or close the window, your server continues
you just log out or close the window your server continues running and running and **will not stop** until the Slurm job runtime hits the limit (usually 8 hours).
will not stop until the Slurm job runtime hits the limit (usually 8
hours).
At first you have to open the JupyterHub control panel. At first you have to open the JupyterHub control panel.
**JupyterLab**: Open the file menu and then click on Logout. You can **JupyterLab**: Open the file menu and then click on `Logout`. You can
also click on "Hub Control Panel" which opens the control panel in a new also click on `Hub Control Panel` which opens the control panel in a new
tab instead. tab instead.
**TODO ADD IMAGE** \<a href="%ATTACHURL%/jupyterlab_logout.png">\<img alt="" height="400" ![JupyterLab logout](misc/jupyterlab_logout.png)
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/jupyterlab_logout.png>" {: align="center"}
style="border: 1px solid #888;" title="JupyterLab logout button"/>\</a>
**Classic notebook**: Click on the control panel button on the top right **Classic Jupyter notebook**: Click on the control panel button on the top right
of your screen. of your screen.
**TODO ADD IMAGE** \<img alt="" src="%ATTACHURL%/notebook_app_control_panel_btn.png" ![Jupyter notebook control panel button](misc/jupyter_notebook_control_panel_button.png)
style="border: 1px solid #888;" title="Classic notebook (control panel {: align="center"}
button)" />
Now you are back on the JupyterHub page and you can stop your server by
clicking on **TODO ADD IMAGE** \<img alt="" height="24"
src="%ATTACHURL%/stop_my_server.png" title="Stop button" />.
## Error handling Now you are back on the JupyterHub page and you can stop your server by clicking on
![Stop my server](misc/stop_my_server.png)
{: align="center"}
We want to explain some errors that you might face sooner or later. If ## Error Handling
you need help open a ticket at HPC support.
### Error while starting a session We want to explain some errors that you might face sooner or later.
If you need help open a ticket at [HPC support](mailto:hpcsupport@zih.tu-dresden.de).
**TODO ADD IMAGE** \<a href="%ATTACHURL%/error_batch_job_submission_failed.png">\<img ### Error at Session Start
alt="" width="400"
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/error_batch_job_submission_failed.png>"
style="border: 1px solid #888;" title="Error message: Batch job
submission failed."/>\</a>
This message often appears instantly if your Slurm parameters are not ![Error batch job submission failed](misc/error_batch_job_submission_failed.png)
valid. Please check those settings against the available hardware. {: align="center"}
Useful pages for valid Slurm parameters:
- [Slurm batch system (Taurus)] **TODO LINK** (../jobs_and_resources/SystemTaurus#Batch_System) This message appears instantly if your batch system parameters are not valid.
- [General information how to use Slurm](../jobs_and_resources/slurm.md) Please check those settings against the available hardware.
Useful pages for valid batch system parameters:
### Error message in JupyterLab - [Slurm batch system (Taurus)](../jobs_and_resources/system_taurus.md#batch-system)
- [General information how to use Slurm](../jobs_and_resources/slurm.md)
**TODO ADD IMAGE** \<a href="%ATTACHURL%/jupyterlab_error_directory_not_found.png">\<img ### Error Message in JupyterLab
alt="" width="400"
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/jupyterlab_error_directory_not_found.png>"
style="border: 1px solid #888;" title="Error message: Directory not
found"/>\</a>
If the connection to your notebook server unexpectedly breaks you maybe ![JupyterLab error directory not found](misc/jupyterlab_error_directory_not_found.png)
will get this error message. {: align="center"}
Sometimes your notebook server might hit a Slurm or hardware limit and
gets killed. Then usually the logfile of the corresponding Slurm job
might contain useful information. These logfiles are located in your
home directory and have the name "jupyter-session-\<jobid>.log".
------------------------------------------------------------------------ If the connection to your notebook server unexpectedly breaks, you will get this error message.
Sometimes your notebook server might hit a batch system or hardware limit and gets killed. Then
usually the logfile of the corresponding batch job might contain useful information. These logfiles
are located in your `home` directory and have the name `jupyter-session-<jobid>.log`.
## Advanced tips ## Advanced Tips
### Standard environments ### Standard Environments
The default python kernel uses conda environments based on the [Watson The default Python kernel uses conda environments based on the
Machine Learning Community Edition (formerly [Watson Machine Learning Community Edition (formerly PowerAI)](https://developer.ibm.com/linuxonpower/deep-learning-powerai/)
PowerAI)](https://developer.ibm.com/linuxonpower/deep-learning-powerai/)
package suite. You can open a list with all included packages of the package suite. You can open a list with all included packages of the
exact standard environment through the spawner form: exact standard environment through the spawner form:
**TODO ADD IMAGE** \<img alt="environment_package_list.png" ![Environment package list](misc/environment_package_list.png)
src="%ATTACHURL%/environment_package_list.png" style="border: 1px solid {: align="center"}
\#888;" title="JupyterHub environment package list" />
This list shows all packages of the currently selected conda This list shows all packages of the currently selected conda environment. This depends on your
environment. This depends on your settings for partition (cpu settings for partition (CPU architecture) and standard environment.
architecture) and standard environment.
There are three standard environments: There are three standard environments:
- production, - production
- test, - test
- python-env-python3.8.6. - python-env-python3.8.6
**Python-env-python3.8.6**virtual environment can be used for all x86 **Python-env-python3.8.6** virtual environment can be used for all x86 partitions(`gpu2`, `alpha`,
partitions(gpu2, alpha, etc). It gives the opportunity to create a user etc). It gives the opportunity to create a user kernel with the help of a Python environment.
kernel with the help of a python environment.
Here's a short list of some included software: Here is a short list of some included software:
| | | |
|------------|-----------|--------|
| | generic\* | ml | | | generic\* | ml |
|------------|-----------|--------|
| Python | 3.6.10 | 3.6.10 | | Python | 3.6.10 | 3.6.10 |
| R\*\* | 3.6.2 | 3.6.0 | | R\*\* | 3.6.2 | 3.6.0 |
| WML CE | 1.7.0 | 1.7.0 | | WML CE | 1.7.0 | 1.7.0 |
...@@ -226,155 +191,122 @@ Here's a short list of some included software: ...@@ -226,155 +191,122 @@ Here's a short list of some included software:
\*\* R is loaded from the [module system](../software/runtime_environment.md) \*\* R is loaded from the [module system](../software/runtime_environment.md)
### Creating and using your own environment ### Creating and Using a Custom Environment
Interactive code interpreters which are used by Jupyter Notebooks are !!! info
called kernels.
Creating and using your own kernel has the benefit that you can install
your own preferred python packages and use them in your notebooks.
We currently have two different architectures at Taurus. Build your Interactive code interpreters which are used by Jupyter notebooks are called *kernels*. Creating
kernel environment on the **same architecture** that you want to use and using your own kernel has the benefit that you can install your own preferred Python
packages and use them in your notebooks.
We currently have two different architectures at ZIH systems.
Build your kernel environment on the **same architecture** that you want to use
later on with the kernel. In the examples below we use the name later on with the kernel. In the examples below we use the name
"my-kernel" for our user kernel. We recommend to prefix your kernels "my-kernel" for our user kernel. We recommend to prefix your kernels
with keywords like "intel", "ibm", "ml", "venv", "conda". This way you with keywords like `haswell`, `ml`, `romeo`, `venv`, `conda`. This way you
can later recognize easier how you built the kernel and on which can later recognize easier how you built the kernel and on which hardware it will work.
hardware it will work.
**Intel nodes** (e.g. haswell, gpu2):
srun --pty -n 1 -c 2 --mem-per-cpu 2583 -t 08:00:00 bash -l **Intel nodes** (e.g. partition `haswell`, `gpu2`):
If you don't need Sandy Bridge support for your kernel you can create ```console
your kernel on partition 'haswell'. maria@login$ srun --pty --ntasks=1 --cpus-per-task=2 --mem-per-cpu=2541 --time=08:00:00 bash -l
```
**Power nodes** (ml partition):
srun --pty -p ml -n 1 -c 2 --mem-per-cpu 5772 -t 08:00:00 bash -l
Create a virtual environment in your home directory. You can decide
between python virtualenvs or conda environments.
<span class="twiki-macro RED"></span> **Note** <span **Power nodes** (partition `ml`):
class="twiki-macro ENDCOLOR"></span>: Please take in mind that Python
venv is the preferred way to create a Python virtual environment.
#### Python virtualenv ```console
maria@login$ srun --pty --partition=ml --ntasks=1 --cpus-per-task=2 --mem-per-cpu=1443 --time=08:00:00 bash -l
```
```bash Create a virtual environment in your `home` directory. You can decide between Python virtualenvs or
$ module load Python/3.8.6-GCCcore-10.2.0 conda environments.
$ mkdir user-kernel #please use Workspaces! !!! note
Please take in mind that Python venv is the preferred way to create a Python virtual environment.
$ cd user-kernel #### Python Virtualenv
$ virtualenv --system-site-packages my-kernel ```console
marie@compute$ module load Python/3.8.6-GCCcore-10.2.0
marie@compute$ mkdir user-kernel # please use workspaces!
marie@compute$ cd user-kernel
marie@compute$ virtualenv --system-site-packages my-kernel
Using base prefix '/sw/installed/Python/3.6.6-fosscuda-2018b' Using base prefix '/sw/installed/Python/3.6.6-fosscuda-2018b'
New python executable in .../user-kernel/my-kernel/bin/python New python executable in .../user-kernel/my-kernel/bin/python
Installing setuptools, pip, wheel...done. Installing setuptools, pip, wheel...done.
marie@compute$ source my-kernel/bin/activate
$ source my-kernel/bin/activate marie@compute$ pip install ipykernel
(my-kernel) $ pip install ipykernel
Collecting ipykernel Collecting ipykernel
... [...]
Successfully installed ... ipykernel-5.1.0 ipython-7.5.0 ... Successfully installed ... ipykernel-5.1.0 ipython-7.5.0 ...
marie@compute$ pip install --upgrade pip
(my-kernel) $ pip install --upgrade pip marie@compute$ python -m ipykernel install --user --name my-kernel --display-name="my kernel"
(my-kernel) $ python -m ipykernel install --user --name my-kernel --display-name="my kernel"
Installed kernelspec my-kernel in .../.local/share/jupyter/kernels/my-kernel Installed kernelspec my-kernel in .../.local/share/jupyter/kernels/my-kernel
marie@compute$ pip install [...] # now install additional packages for your notebooks
[now install additional packages for your notebooks] marie@compute$ deactivate
(my-kernel) $ deactivate
``` ```
#### Conda environment #### Conda Environment
Load the needed module for Intel nodes Load the needed module for Intel nodes
``` ```console
module load Anaconda3 marie@compute$ module load Anaconda3
``` ```
... or for IBM nodes (ml partition): ... or for IBM nodes (partition `ml`):
``` ```console
module load PythonAnaconda marie@ml$ module load PythonAnaconda
``` ```
Continue with environment creation, package installation and kernel Continue with environment creation, package installation and kernel registration:
registration:
```
$ mkdir user-kernel #please use Workspaces!
$ conda create --prefix /home/<USER>/user-kernel/my-kernel python=3.6 ```console
marie@compute$ mkdir user-kernel # please use workspaces!
marie@compute$ conda create --prefix /home/<USER>/user-kernel/my-kernel python=3.6
Collecting package metadata: done Collecting package metadata: done
Solving environment: done Solving environment: done
[...] [...]
marie@compute$ conda activate /home/<USER>/user-kernel/my-kernel
$ conda activate /home/<USER>/user-kernel/my-kernel marie@compute$ conda install ipykernel
$ conda install ipykernel
Collecting package metadata: done Collecting package metadata: done
Solving environment: done Solving environment: done
[...] [...]
marie@compute$ python -m ipykernel install --user --name my-kernel --display-name="my kernel"
$ python -m ipykernel install --user --name my-kernel --display-name="my kernel"
Installed kernelspec my-kernel in [...] Installed kernelspec my-kernel in [...]
marie@compute$ conda install [..] # now install additional packages for your notebooks
[now install additional packages for your notebooks] marie@compute$ conda deactivate
$ conda deactivate
``` ```
Now you can start a new session and your kernel should be available. Now you can start a new session and your kernel should be available.
**In JupyterLab**: **JupyterLab**: Your kernels are listed on the launcher page:
Your kernels are listed on the launcher page:
**TODO ADD IMAGE**\<a href="%ATTACHURL%/user-kernel_in_jupyterlab_launcher.png">\<img ![JupyterLab user kernel launcher](misc/jupyterlab_user_kernel_launcher.png)
alt="jupyterlab_app.png" height="410" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/user-kernel_in_jupyterlab_launcher.png>"
style="border: 1px solid #888;" title="JupyterLab kernel launcher
list"/>\</a>
You can switch kernels of existing notebooks in the menu: You can switch kernels of existing notebooks in the menu:
**TODO ADD IMAGE** \<a href="%ATTACHURL%/jupyterlab_change_kernel.png">\<img ![JupyterLab change kernel](misc/jupyterlab_change_kernel.png)
alt="jupyterlab_app.png" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/jupyterlab_change_kernel.png>"
style="border: 1px solid #888;" title="JupyterLab kernel switch"/>\</a>
**In classic notebook app**:
Your kernel is listed in the New menu: **Classic Jupyter notebook**: Your kernel is listed in the New menu:
**TODO ADD IMAGE** \<a href="%ATTACHURL%/user-kernel_in_jupyter_notebook.png">\<img ![Jupyter notebook user kernel launcher](misc/jupyter_notebook_user_kernel_launcher.png)
alt="jupyterlab_app.png" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/user-kernel_in_jupyter_notebook.png>"
style="border: 1px solid #888;" title="Classic notebook (create notebook
with new kernel)"/>\</a>
You can switch kernels of existing notebooks in the kernel menu: You can switch kernels of existing notebooks in the kernel menu:
**TODO ADD IMAGE** \<a href="%ATTACHURL%/switch_kernel_in_jupyter_notebook.png">\<img ![Jupyter notebook change kernel](misc/jupyter_notebook_change_kernel.png)
alt="jupyterlab_app.png" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHub/switch_kernel_in_jupyter_notebook.png>"
style="border: 1px solid #888;" title="Classic notebook (kernel
switch)"/>\</a>
**Note**: Both python venv and conda virtual environments will be !!! note
mention in the same list. Both python venv and conda virtual environments will be mention in the same list.
### Loading modules ### Loading Modules
You have now the option to preload modules from the LMOD module You have now the option to preload modules from the [module system](../software/modules.md).
system. Select multiple modules that will be preloaded before your notebook server starts. The list of
Select multiple modules that will be preloaded before your notebook available modules depends on the module environment you want to start the session in (`scs5` or
server starts. The list of available modules depends on the module `ml`). The right module environment will be chosen by your selected partition.
environment you want to start the session in (scs5 or ml). The right
module environment will be chosen by your selected partition.
...@@ -3,110 +3,100 @@ ...@@ -3,110 +3,100 @@
On this page we want to introduce to you some useful features if you On this page we want to introduce to you some useful features if you
want to use JupyterHub for teaching. want to use JupyterHub for teaching.
<span style="color:red">**PLEASE UNDERSTAND:** </span> JupyterHub uses compute resources !!! note
from the HPC system Taurus. Please be aware of the following notes:
JupyterHub uses compute resources from ZIH systems.
- The HPC system operates at a lower availability level than your
usual Enterprise Cloud VM. There can always be downtimes, e.g. of Please be aware of the following notes:
the file systems or the batch system.
- Scheduled downtimes are announced by email. Please plan your courses - ZIH systems operate at a lower availability level than your usual Enterprise Cloud VM. There
accordingly. can always be downtimes, e.g. of the filesystems or the batch system.
- Access to HPC resources is handled through projects. See your course - Scheduled downtimes are announced by email. Please plan your courses accordingly.
as a project. Projects need to be registered beforehand (more info - Access to HPC resources is handled through projects. See your course as a project. Projects need
on the page [Access](./../application/access.md)). to be registered beforehand (more info on the page [Access](../application/overview.md)).
- Don't forget to **TODO ANCHOR**(add your - Don't forget to **TODO ANCHOR**(add your users)
users)(ProjectManagement#manage_project_members_40dis_45_47enable_41) (ProjectManagement#manage_project_members_40dis_45_47enable_41) (eg. students or tutors) to
(eg. students or tutors) to your project. your project.
- It might be a good idea to **TODO ANCHOR**(request a - It might be a good idea to **TODO ANCHOR**(request a
reservation)(Slurm#Reservations) of part of the compute resources reservation)(Slurm#Reservations) of part of the compute resources for your project/course to
for your project/course to avoid unnecessary waiting times in the avoid unnecessary waiting times in the batch system queue.
batch system queue.
## Clone a repository with a link ## Clone a Repository With a Link
This feature bases on This feature bases on
[nbgitpuller](https://github.com/jupyterhub/nbgitpuller) ( [nbgitpuller](https://github.com/jupyterhub/nbgitpuller).
[documentation](https://jupyterhub.github.io/nbgitpuller/)) Documentation can be found at
[this page](https://jupyterhub.github.io/nbgitpuller/).
This extension for jupyter notebooks can clone every public git This extension for Jupyter notebooks can clone every public git repository into the users work
repository into the users work directory. It's offering a quick way to directory. It's offering a quick way to distribute notebooks and other material to your students.
distribute notebooks and other material to your students.
**TODO ADD IMAGE** \<a href="%ATTACHURL%/gitpull_progress.png">\<img alt="Git pull progress ![Git pull progress screen](misc/gitpull_progress.png)
screen" width="475" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHubForTeaching/gitpull_progress.png>"
style="border: 1px solid #888;" title="Git pull progress screen"/>\</a>
A sharable link for this feature looks like this: A shareable link for this feature looks like this:
<https://taurus.hrsk.tu-dresden.de/jupyter/hub/user-redirect/git-pull?repo=https://github.com/jdwittenauer/ipython-notebooks&urlpath=/tree/ipython-notebooks/notebooks/language/Intro.ipynb> <https://taurus.hrsk.tu-dresden.de/jupyter/hub/user-redirect/git-pull?repo=https://github.com/jdwittenauer/ipython-notebooks&urlpath=/tree/ipython-notebooks/notebooks/language/Intro.ipynb>
**TODO ADD IMAGE**\<a href="%ATTACHURL%/url-git-pull.png?t=1604588695">\<img alt="URL with ![URL with git-pull parameters](misc/url-git-pull.png)
git-pull parameters" width="100%" style="max-width: 2717px" {: align="center"}
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHubForTeaching/url-git-pull.png?t=1604588695>"
style="border: 1px solid #888;" title="URL with git-pull
parameters"/>\</a>
This example would clone the repository This example would clone the repository
[https://github.com/jdwittenauer/ipython-notebooks]( [https://github.com/jdwittenauer/ipython-notebooks](https://github.com/jdwittenauer/ipython-notebooks)
https://github.com/jdwittenauer/ipython-notebooks)
and afterwards open the **Intro.ipynb** notebook in the given path. and afterwards open the **Intro.ipynb** notebook in the given path.
The following parameters are available: The following parameters are available:
|parameter | info | | Parameter | Info |
|---|---| |---|---|
|repo|path to git repository| |`repo` | path to git repository|
|branch|branch in the repository to pull from default: `master`| |`branch` | branch in the repository to pull from default: `master`|
|urlpath|URL to redirect the user to a certain file [more info](https://jupyterhub.github.io/nbgitpuller/topic/url-options.html#urlpath)| |`urlpath` | URL to redirect the user to a certain file [more info](https://jupyterhub.github.io/nbgitpuller/topic/url-options.html#urlpath)|
|depth|clone only a certain amount of latest commits not recommended| |`depth` | clone only a certain amount of latest commits not recommended|
This [link This [link
generator](https://jupyterhub.github.io/nbgitpuller/link?hub=https://taurus.hrsk.tu-dresden.de/jupyter/) generator](https://jupyterhub.github.io/nbgitpuller/link?hub=https://taurus.hrsk.tu-dresden.de/jupyter/)
might help creating those links might help creating those links
## Spawner options passthrough with URL params ## Spawner Options Passthrough with URL Parameters
The spawn form now offers a quick start mode by passing url The spawn form now offers a quick start mode by passing URL parameters.
parameters.
An example: The following link would create a jupyter notebook session
on the `interactive` partition with the `test` environment being loaded:
``` !!! example
https://taurus.hrsk.tu-dresden.de/jupyter/hub/spawn#/~(partition~'interactive~environment~'test)
``` The following link would create a jupyter notebook session on the `interactive` partition with the `test`
environment being loaded:
**TODO ADD IMG** \<a href="%ATTACHURL%/url-quick-start.png?t=1604586059">\<img alt="URL ```
with quickstart parameters" width="100%" style="max-width: 800px" https://taurus.hrsk.tu-dresden.de/jupyter/hub/spawn#/~(partition~'interactive~environment~'test)
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHubForTeaching/url-quick-start.png?t=1604586059>" ```
style="border: 1px solid #888;" title="URL with quickstart
parameters"/>\</a>
Every parameter of the advanced form can ![URL with quickstart parameters](misc/url-quick-start.png)
be set with this parameter. If the parameter is not mentioned, the {: align="center"}
default value will be loaded.
| parameter | default value | Every parameter of the advanced form can be set with this parameter. If the parameter is not
mentioned, the default value will be loaded.
| Parameter | Default Value |
|:----------------|:-----------------------------------------| |:----------------|:-----------------------------------------|
| partition | default | | `partition` | default |
| nodes | 1 | | `nodes` | 1 |
| ntasks | 1 | | `ntasks` | 1 |
| cpuspertask | 1 | | `cpuspertask` | 1 |
| gres | *empty* (no generic resources) | | `gres` | *empty* (no generic resources) |
| mempercpu | 1000 | | `mempercpu` | 1000 |
| runtime | 8:00:00 | | `runtime` | 8:00:00 |
| reservation | *empty* (use no reservation) | | `reservation` | *empty* (use no reservation) |
| project | *empty* (use default project) | | `project` | *empty* (use default project) |
| modules | *empty* (do not load additional modules) | | `modules` | *empty* (do not load additional modules) |
| environment | production | | `environment` | production |
| launch | JupyterLab | | `launch` | JupyterLab |
| workspace_scope | *empty* (home directory) | | `workspace_scope` | *empty* (home directory) |
You can use the advanced form to generate a url for the settings you You can use the advanced form to generate a URL for the settings you want. The address bar contains
want. The address bar contains the encoded parameters starting with the encoded parameters starting with `#/`.
`#/`.
### Combination of Quickstart and Git-Pull Feature
### Combination of quickstart and git-pull feature
You can combine both features in a single link: You can combine both features in a single link:
...@@ -114,28 +104,21 @@ You can combine both features in a single link: ...@@ -114,28 +104,21 @@ You can combine both features in a single link:
https://taurus.hrsk.tu-dresden.de/jupyter/hub/user-redirect/git-pull?repo=https://github.com/jdwittenauer/ipython-notebooks&urlpath=/tree/ipython-notebooks/notebooks/language/Intro.ipynb#/~(partition~'interactive~environment~'test) https://taurus.hrsk.tu-dresden.de/jupyter/hub/user-redirect/git-pull?repo=https://github.com/jdwittenauer/ipython-notebooks&urlpath=/tree/ipython-notebooks/notebooks/language/Intro.ipynb#/~(partition~'interactive~environment~'test)
``` ```
**TODO ADD IMAGE**\<a ![URL with quickstart parameters](misc/url-quick-start.png)
href="%ATTACHURL%/url-git-pull-and-quick-start.png?t=1604588695">\<img {: align="center"}
alt="URL with git-pull and quickstart parameters" width="100%"
style="max-width: 3332px"
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHubForTeaching/url-git-pull-and-quick-start.png?t=1604588695>"
style="border: 1px solid #888;" title="URL with git-pull and quickstart
parameters"/>\</a>
## Open a notebook automatically with a single link ## Open a Notebook Automatically with a Single Link
With the following link you will be redirected to a certain file in your With the following link you will be redirected to a certain file in your
home directory. The file needs to exist, otherwise a 404 error will be home directory.
thrown.
[https://taurus.hrsk.tu-dresden.de/jupyter/user-redirect/notebooks/demo.ipynb]
(https://taurus.hrsk.tu-dresden.de/jupyter/user-redirect/notebooks/demo.ipynb)
[https://taurus.hrsk.tu-dresden.de/jupyter/user-redirect/notebooks/demo.ipynb](https://taurus.hrsk.tu-dresden.de/jupyter/user-redirect/notebooks/demo.ipynb) The file needs to exist, otherwise a 404 error will be thrown.
**TODO ADD IMAGE** \<a href="%ATTACHURL%/url-user-redirect.png">\<img alt="URL with ![URL with git-pull and quickstart parameters](misc/url-user-redirect.png)
git-pull and quickstart parameters" width="100%" style="max-width: {: align="center"}
700px"
src="<https://doc.zih.tu-dresden.de/hpc-wiki/pub/Compendium/JupyterHubForTeaching/url-user-redirect.png?t=1604587961>"
style="border: 1px solid #888;" title="URL with git-pull and quickstart
parameters"/>\</a>
This link would redirect to This link would redirect to
`https://taurus.hrsk.tu-dresden.de/jupyter/user/{login}/notebooks/demo.ipynb` . `https://taurus.hrsk.tu-dresden.de/jupyter/user/{login}/notebooks/demo.ipynb`.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment