Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
hpc-compendium
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Deploy
Releases
Package registry
Container Registry
Model registry
Operate
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
ZIH
hpcsupport
hpc-compendium
Commits
bf1b7965
Commit
bf1b7965
authored
3 years ago
by
Taras Lazariv
Browse files
Options
Downloads
Patches
Plain Diff
Fix archive/install_jupyter.md
parent
8ea61535
No related branches found
Branches containing commit
No related tags found
3 merge requests
!412
Manual attempt to merge preview into main
,
!402
Solved issue-194. Added missing information regarding usage of SSH config for...
,
!397
Jupyterhub
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
doc.zih.tu-dresden.de/docs/archive/install_jupyter.md
+26
-24
26 additions, 24 deletions
doc.zih.tu-dresden.de/docs/archive/install_jupyter.md
with
26 additions
and
24 deletions
doc.zih.tu-dresden.de/docs/archive/install_jupyter.md
+
26
−
24
View file @
bf1b7965
# Jupyter Installation
# Jupyter Installation
!!! warning
This page is outdated!
Jupyter notebooks allow to analyze data interactively using your web browser. One advantage of
Jupyter notebooks allow to analyze data interactively using your web browser. One advantage of
Jupyter is, that code, documentation and visualization can be included in a single notebook, so that
Jupyter is, that code, documentation and visualization can be included in a single notebook, so that
it forms a unit. Jupyter notebooks can be used for many tasks, such as data cleaning and
it forms a unit. Jupyter notebooks can be used for many tasks, such as data cleaning and
...
@@ -41,17 +45,17 @@ one is to download Anaconda in your home directory.
...
@@ -41,17 +45,17 @@ one is to download Anaconda in your home directory.
1.
Load Anaconda module (recommended):
1.
Load Anaconda module (recommended):
```
console
```
console
marie@compute module load modenv/scs5
marie@compute
$
module load modenv/scs5
marie@compute module load Anaconda3
marie@compute
$
module load Anaconda3
```
```
1.
Download latest Anaconda release (see example below) and change the rights to make it an
1.
Download latest Anaconda release (see example below) and change the rights to make it an
executable script and run the installation script:
executable script and run the installation script:
```
console
```
console
marie@compute wget https://repo.continuum.io/archive/Anaconda3-2019.03-Linux-x86_64.sh
marie@compute
$
wget https://repo.continuum.io/archive/Anaconda3-2019.03-Linux-x86_64.sh
marie@compute chmod u+x Anaconda3-2019.03-Linux-x86_64.sh
marie@compute
$
chmod
u+x Anaconda3-2019.03-Linux-x86_64.sh
marie@compute ./Anaconda3-2019.03-Linux-x86_64.sh
marie@compute
$
./Anaconda3-2019.03-Linux-x86_64.sh
```
```
(during installation you have to confirm the license agreement)
(during installation you have to confirm the license agreement)
...
@@ -60,7 +64,7 @@ Next step will install the anaconda environment into the home
...
@@ -60,7 +64,7 @@ Next step will install the anaconda environment into the home
directory (
`/home/userxx/anaconda3`
). Create a new anaconda environment with the name
`jnb`
.
directory (
`/home/userxx/anaconda3`
). Create a new anaconda environment with the name
`jnb`
.
```
console
```
console
marie@compute conda create --name jnb
marie@compute
$
conda create
--name
jnb
```
```
## Set environmental variables
## Set environmental variables
...
@@ -69,15 +73,15 @@ In the shell, activate previously created python environment (you can
...
@@ -69,15 +73,15 @@ In the shell, activate previously created python environment (you can
deactivate it also manually) and install Jupyter packages for this python environment:
deactivate it also manually) and install Jupyter packages for this python environment:
```
console
```
console
marie@compute source activate jnb
marie@compute
$
source
activate jnb
marie@compute conda install jupyter
marie@compute
$
conda
install
jupyter
```
```
If you need to adjust the configuration, you should create the template. Generate configuration
If you need to adjust the configuration, you should create the template. Generate configuration
files for Jupyter notebook server:
files for Jupyter notebook server:
```
console
```
console
marie@compute jupyter notebook --generate-config
marie@compute
$
jupyter notebook
--generate-config
```
```
Find a path of the configuration file, usually in the home under
`.jupyter`
directory, e.g.
Find a path of the configuration file, usually in the home under
`.jupyter`
directory, e.g.
...
@@ -87,12 +91,12 @@ Set a password (choose easy one for testing), which is needed later on to log in
...
@@ -87,12 +91,12 @@ Set a password (choose easy one for testing), which is needed later on to log in
in browser session:
in browser session:
```
console
```
console
marie@compute jupyter notebook password Enter password: Verify password:
marie@compute
$
jupyter notebook password Enter password: Verify password:
```
```
You get a message like that:
You get a message like that:
```
console
```
bash
[
NotebookPasswordApp] Wrote
*
hashed password
*
to
[
NotebookPasswordApp] Wrote
*
hashed password
*
to
/home/<zih_user>/.jupyter/jupyter_notebook_config.json
/home/<zih_user>/.jupyter/jupyter_notebook_config.json
```
```
...
@@ -101,7 +105,7 @@ I order to create a certificate for secure connections, you can create a self-si
...
@@ -101,7 +105,7 @@ I order to create a certificate for secure connections, you can create a self-si
certificate:
certificate:
```
console
```
console
marie@compute openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mykey.key -out mycert.pem
marie@compute
$
openssl req
-x509
-nodes
-days
365
-newkey
rsa:1024
-keyout
mykey.key
-out
mycert.pem
```
```
Fill in the form with decent values.
Fill in the form with decent values.
...
@@ -128,7 +132,7 @@ c.NotebookApp.allow_remote_access = True
...
@@ -128,7 +132,7 @@ c.NotebookApp.allow_remote_access = True
## Slurm job file to run the Jupyter server on ZIH system with GPU (1x K80) (also works on K20)
## Slurm job file to run the Jupyter server on ZIH system with GPU (1x K80) (also works on K20)
```
console
```
bash
#!/bin/bash -l
#!/bin/bash -l
#SBATCH --gres=gpu:1 # request GPU
#SBATCH --gres=gpu:1 # request GPU
#SBATCH --partition=gpu2 # use partition GPU 2
#SBATCH --partition=gpu2 # use partition GPU 2
...
@@ -138,7 +142,7 @@ c.NotebookApp.allow_remote_access = True
...
@@ -138,7 +142,7 @@ c.NotebookApp.allow_remote_access = True
#SBATCH --time=02:30:00
#SBATCH --time=02:30:00
#SBATCH --mem=4000M
#SBATCH --mem=4000M
#SBATCH -J "jupyter-notebook" # job-name
#SBATCH -J "jupyter-notebook" # job-name
#
SBATCH
-A
<name_of_your_project>
#SBATCH -A
p_marie
unset
XDG_RUNTIME_DIR
# might be required when interactive instead of sbatch to avoid 'Permission denied error'
unset
XDG_RUNTIME_DIR
# might be required when interactive instead of sbatch to avoid 'Permission denied error'
srun jupyter notebook
srun jupyter notebook
...
@@ -146,7 +150,7 @@ srun jupyter notebook
...
@@ -146,7 +150,7 @@ srun jupyter notebook
Start the script above (e.g. with the name
`jnotebook`
) with sbatch command:
Start the script above (e.g. with the name
`jnotebook`
) with sbatch command:
```
console
```
bash
sbatch jnotebook.slurm
sbatch jnotebook.slurm
```
```
...
@@ -155,9 +159,7 @@ If you have a question about sbatch script see the article about [Slurm](../jobs
...
@@ -155,9 +159,7 @@ If you have a question about sbatch script see the article about [Slurm](../jobs
Check by the command:
`tail notebook_output.txt`
the status and the
**token**
of the server. It
Check by the command:
`tail notebook_output.txt`
the status and the
**token**
of the server. It
should look like this:
should look like this:
```
console
`https://(taurusi2092.taurus.hrsk.tu-dresden.de or 127.0.0.1):9999/`
https://(taurusi2092.taurus.hrsk.tu-dresden.de or 127.0.0.1):9999/
```
You can see the
**server node's hostname**
by the command:
`squeue -u <username>`
.
You can see the
**server node's hostname**
by the command:
`squeue -u <username>`
.
...
@@ -169,7 +171,7 @@ There are two options on how to connect to the server:
...
@@ -169,7 +171,7 @@ There are two options on how to connect to the server:
solution above. Open the other terminal and configure ssh
solution above. Open the other terminal and configure ssh
tunnel: (look up connection values in the output file of Slurm job, e.g.) (recommended):
tunnel: (look up connection values in the output file of Slurm job, e.g.) (recommended):
```
console
```
bash
node
=
taurusi2092
#see the name of the node with squeue -u <your_login>
node
=
taurusi2092
#see the name of the node with squeue -u <your_login>
localport
=
8887
#local port on your computer
localport
=
8887
#local port on your computer
remoteport
=
9999
#pay attention on the value. It should be the same value as value in the notebook_output.txt
remoteport
=
9999
#pay attention on the value. It should be the same value as value in the notebook_output.txt
...
@@ -183,12 +185,12 @@ pgrep -f "ssh -fNL ${localport}" #verify that tunnel is alive
...
@@ -183,12 +185,12 @@ pgrep -f "ssh -fNL ${localport}" #verify that tunnel is alive
You can connect directly if you know the IP address (just ping the node's hostname while logged on
You can connect directly if you know the IP address (just ping the node's hostname while logged on
ZIH system).
ZIH system).
```
console
```
bash
#
comand on remote terminal
#com
m
and on remote terminal
taurusi2092$
>
host taurusi2092
taurusi2092
$
host taurusi2092
#
copy IP address from output
# copy IP address from output
# paste IP to your browser or call on local terminal e.g.:
# paste IP to your browser or call on local terminal e.g.:
local$
>
firefox https://<IP>:<PORT>
# https important to use SSL cert
local
$
firefox https://<IP>:<PORT>
# https important to use SSL cert
```
```
To login into the Jupyter notebook site, you have to enter the
**token**
.
To login into the Jupyter notebook site, you have to enter the
**token**
.
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment