Skip to content
Snippets Groups Projects
Commit 6b441c17 authored by Guilherme Pantoja Calandrini de Azevedo Junior's avatar Guilherme Pantoja Calandrini de Azevedo Junior
Browse files

Merge branch 'gupa977e_NEW_jupyterhub' into 'preview'

Gupa977e new jupyterhub

See merge request !945
parents ad9e3d5f a527d86d
No related branches found
No related tags found
2 merge requests!947Automated merge from preview to main,!945Gupa977e new jupyterhub
......@@ -26,71 +26,65 @@ case.
you need to apply for an HPC project.
JupyterHub is available at
[https://taurus.hrsk.tu-dresden.de/jupyter](https://taurus.hrsk.tu-dresden.de/jupyter).
## Start a Session
[https://jupyterhub.hpc.tu-dresden.de](https://jupyterhub.hpc.tu-dresden.de).
Start a new session by clicking on the `Start my server` button.
A form opens up where you can customize your session. Our simple form
offers you the most important settings to start quickly.
Old taurus
[https://taurus.hrsk.tu-dresden.de/jupyter](https://taurus.hrsk.tu-dresden.de/jupyter).
![Simple form](misc/simple_form.png)
{: align="center"}
## Login page
For advanced users, we have an extended form where you can change many settings. You can:
At login page please use your ZIH credential (without @tu-dresden.de).
- modify batch system parameters to your needs ([more about batch system Slurm](../jobs_and_resources/slurm.md))
- assign your session to a project or reservation
- load modules from the [module system](../software/modules.md)
- choose a different standard environment (in preparation for future
software updates or testing additional features)
For example:
Username: ddde223z
![Advanced form](misc/advanced_form.png)
![Simple form](misc/jupyterhub_loginpage.png)
{: align="center"}
You can save your own configurations as additional presets. Those are
saved in your browser and are lost if you delete your browsing data. Use
the import/export feature (available through the button) to save your
presets in text files.
!!! info
The partition [alpha](https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/AlphaCentauri)
is available only in the extended form.
## Applications
You can choose between JupyterLab or classic Jupyter notebooks as outlined in the following.
=== "JupyterLab"
![JupyterLab overview](misc/jupyterlab_overview.png)
{: align="center"}
## Start a Session
The main workspace is used for multiple notebooks, consoles or
terminals. Those documents are organized with tabs and a very versatile
split screen feature. On the left side of the screen you can open
several views:
Start a new session by clicking on the `Start my server` button.
- file manager
- controller for running kernels and terminals
- overview of commands and settings
- details about selected notebook cell
- list of open tabs
### Standard Profiles
=== "Classic Jupyter Notebook"
Our simple form offers you the most important settings to start quickly.
Initially, your home directory is listed. You can open existing notebooks or
files by navigating to the corresponding path and clicking on them.
![Advanced form](misc/jupyterhub_profile_selector.png)
{: align="center"}
We have created three profiles for each Cluster, namely:
| name | optimized for | specially recommended for |
|----------------------------------------------|-----------------|---------------------------|
| Alpha - 1 core, 1,5GB 1 hour | x86_64 (AMD) | python programming |
| Alpha - 2 Core, 3GB, 4 Hours | x86_64 (AMD) | R programming |
| Alpha - 4 Core, 8GB, 8 Hours | x86_64 (AMD) | |
| Barnard 1 core, 1,5GB 1 hour | x86_64 (Intel) | python programming |
| Barnard - 2 Core, 3GB, 4 Hours | x86_64 (Intel) | Julia and R programming |
| Barnard - 4 Core, 8GB, 8 Hours | x86_64 (Intel) | |
| Romeo - 1 core, 1,5GB 1 hour | x86_64 (AMD) | python programming |
| Romeo - 2 Core, 3GB, 4 Hours | x86_64 (AMD) | R programming |
| Romeo - 4 Core, 8GB, 8 Hours | x86_64 (AMD) | |
| VIS - 2 Core, 4GB, 2 Hours | Visualization | ANSYS |
| VIS - 4 Core, 8GB, 6 Hours | Visualization | ANSYS |
## JupyterLab
After your session it is spawned you will be redirected to JupyterLab. The main interface looks
like as following:
![JupyterLab overview](misc/jupyterlab_overview_2023-12-19.png)
{: align="center"}
![Jupyter notebook file browser](misc/jupyter_notebook_file_browser.png)
{: align="center"}
The main workspace is used for multiple notebooks, consoles or
terminals. Those documents are organized with tabs and a very versatile
split screen feature. On the left side of the screen you can open
several views:
Above the table on the right side, there is the button `New` which lets you create new
notebooks, files, directories or terminals.
![Jupyter notebook example Matplotlib](misc/jupyter_notebook_example_matplotlib.png)
{: align="center"}
- file manager
- controller for running kernels and terminals
- overview of commands and settings
- details about selected notebook cell
- list of open tabs
## Jupyter Notebooks in General
......@@ -133,40 +127,12 @@ At first, you have to open the JupyterHub control panel.
![JupyterLab logout](misc/jupyterlab_logout.png)
{: align="center"}
=== "Classic Jupyter notebook"
Click on the control panel button on the top right of your screen.
![Jupyter notebook control panel button](misc/jupyter_notebook_control_panel_button.png)
{: align="center"}
Now, you are back on the JupyterHub page and you can stop your server by clicking on
![Stop my server](misc/stop_my_server.png)
{: align="center"}
## Error Handling
We want to explain some errors that you might face sooner or later.
If you need help, open a ticket and ask for support as described in
[How to Ask for Support](../support/support.md).
### Error at Session Start
![Error batch job submission failed](misc/error_batch_job_submission_failed.png)
{: align="center"}
This message appears instantly, if your batch system parameters are not valid.
Please check those settings against the available hardware.
Useful pages for valid batch system parameters:
- [General information how to use Slurm](../jobs_and_resources/slurm.md)
- [Partitions and limits](../jobs_and_resources/slurm_limits.md)
!!! hint
This message might also appear for other Slurm related problems, e.g. quota issues.
That might be the case when the error appears for you but not for others while using the same
system parameters. In this case, please ask for support as described in
[How to Ask for Support](../support/support.md).
### Error Message in JupyterLab
![JupyterLab error directory not found](misc/jupyterlab_error_directory_not_found.png)
......@@ -176,79 +142,13 @@ If the connection to your notebook server unexpectedly breaks, you will get this
error message. Sometimes your notebook server might hit a batch system or
hardware limit and gets killed. Then, the log file of the corresponding
batch job usually contains useful information. These log files are located in your
home directory and have the name `jupyter-session-<jobid>.log`.
home directory and have the name `jupyterhub-<clustername>.log`.
## Advanced Tips
### Standard Environments
The default Python kernel uses conda environments based on the
[Watson Machine Learning Community Edition (formerly PowerAI)](https://developer.ibm.com/linuxonpower/deep-learning-powerai/)
package suite. You can open a list with all included python-packages that
are available within the standard environment through the spawner form:
![Environment package list](misc/environment_package_list.png)
{: align="center"}
This list shows all packages of the currently selected conda environment. This
depends on your settings for partition (CPU architecture) and standard
environment.
We have a number of standard environments, namely:
| name | optimized for | specially recommended for | Modenv | Partitions |
|------------------------------------------------------------|-----------------|---------------------------|-------------------------|----------------------------------------------------------------------------|
| foss-2020b | x86_64, ppc64le | | modenv/hiera, modenv/ml | `romeo`, `alpha`, `ml` |
| fosscuda-2019b | GPUs | | modenv/hiera, modenv/ml | `romeo`, `alpha`, `ml` |
| fosscuda-2020b | GPUs | | modenv/hiera, modenv/ml | `romeo`, `alpha`, `ml` |
| gcccore-10.3.0_python-3.9.5_matlab-2021b | x86_64 | MATLAB | modenv/hiera | `romeo`, `alpha` |
| hiera_gcccore-10.2.0_python-3.8.6 | x86_64 (AMD) | | modenv/hiera | `romeo`, `alpha` |
| hiera_gcccore-10.3.0_python-3.9.5_matlab-2021b | x86_64 (AMD) | MATLAB | modenv/hiera | `romeo`, `alpha` |
| hiera_gcccore-10.3.0_python-3.9.5_r-4.1.0_rstudio-1.4.1717 | x86_64 (AMD) | RStudio | modenv/hiera | `romeo`, `alpha` |
| **production** | x86_64, ppc64le | | modenv/scs5, modenv/ml | `default`, `haswell`, `interactive`, `gpu2`, `hpdlf`, `dcv`, `julia`, `ml` |
| scs5_gcccore-10.2.0_python-3.8.6 | x86_64 (Intel) | | modenv/scs5 | `default`, `haswell`, `interactive`, `gpu2`, `hpdlf`, `dcv`, `julia` |
| scs5_gcccore-10.3.0_python-3.9.5_matlab-2021b | x86_64 (Intel) | MATLAB | modenv/scs5 | `default`, `haswell`, `interactive`, `gpu2`, `hpdlf`, `dcv`, `julia` |
| scs5_gcccore-8.3.0_python-3.7.4 | x86_64 (Intel) | | modenv/scs5 | `default`, `haswell`, `interactive`, `gpu2`, `hpdlf`, `dcv`, `julia` |
<p style="font-size: 80%;">Actually 'romeo', 'alpha' and 'ml' refer to their respective interactive
partitions 'romeo-interactive', 'alpha-interactive', 'ml-interactive' to reduce job queue waiting time.</p>
The standard environments in Jupyter are also closely related to the available
[Module Environments](../software/modules.md#module-environments).
With these **standard environments**, we have tried to integrate a set of compatible software:
=== "production"
In this environment, we provide DCV and WebVNC access.
You can load more software using our module system,
by selecting these through the option ['Preload modules (module load)'](#loading-modules).
=== "...matlab"
We integrated MATLAB into it, such that you can use it in your browser.
You can load more software using our module system,
by selecting these through the option ['Preload modules (module load)'](#loading-modules).
=== "...rstudio"
We integrated R & RStudio into it, such that you can use it in your browser.
Please click on RStudio's Icon for it, when you are in the JupyterHub overview.
=== "fosscuda-2020b"
Can be utilized with the partitions `gpu2`, `alpha` and `ml`.
It's specially geared towards GPU support.
### Loading Modules
You have now the option to preload modules from the [module system](../software/modules.md).
Select multiple modules that will be preloaded before your notebook server
starts. The list of available modules depends on the module environment you want
to start the session in (`scs5`, `hiera` or `ml`). The right module environment will be
chosen by your selected partition.
Inside your terminal session you can load modules from the [module system](../software/modules.md).
### Custom Kernels
......
doc.zih.tu-dresden.de/docs/access/misc/jupyterhub_loginpage.png

25.6 KiB

doc.zih.tu-dresden.de/docs/access/misc/jupyterhub_profile_selector.png

42.1 KiB

doc.zih.tu-dresden.de/docs/access/misc/jupyterlab_overview_2023-12-19.png

146 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment