Skip to content
Snippets Groups Projects
Commit 767f7529 authored by Natalie Breidenbach's avatar Natalie Breidenbach
Browse files

Update tensorflow.md

parent 7aff028c
No related branches found
No related tags found
2 merge requests!938Automated merge from preview to main,!936Update to Five-Cluster-Operation
...@@ -17,13 +17,16 @@ to find out, which TensorFlow modules are available on your partition. ...@@ -17,13 +17,16 @@ to find out, which TensorFlow modules are available on your partition.
On ZIH systems, TensorFlow 2 is the default module version. For compatibility hints between On ZIH systems, TensorFlow 2 is the default module version. For compatibility hints between
TensorFlow 2 and TensorFlow 1, see the corresponding [section below](#compatibility-tf2-and-tf1). TensorFlow 2 and TensorFlow 1, see the corresponding [section below](#compatibility-tf2-and-tf1).
We recommend using partitions `alpha` and/or `ml` when working with machine learning workflows We recommend using the clusters `alpha` and/or `power` when working with machine learning workflows
and the TensorFlow library. You can find detailed hardware specification in our and the TensorFlow library. You can find detailed hardware specification in our
[Hardware](../jobs_and_resources/hardware_overview.md) documentation. [Hardware](../jobs_and_resources/hardware_overview.md) documentation.
## TensorFlow Console ## TensorFlow Console
On the partition `alpha`, load the module environment: _The module environments /hiera, /scs5, /classic and /ml originated from the old taurus system are momentarily under construction. The script will be updated after completion of the redesign accordingly_
On the cluster `alpha`, load the module environment:
```console ```console
marie@alpha$ module load modenv/scs5 marie@alpha$ module load modenv/scs5
...@@ -47,17 +50,17 @@ marie@alpha$ module avail TensorFlow ...@@ -47,17 +50,17 @@ marie@alpha$ module avail TensorFlow
[...] [...]
``` ```
On the partition `ml` load the module environment: On the cluster `power` load the module environment:
```console ```console
marie@ml$ module load modenv/ml marie@power$ module load modenv/ml
The following have been reloaded with a version change: 1) modenv/scs5 => modenv/ml The following have been reloaded with a version change: 1) modenv/scs5 => modenv/ml
``` ```
This example shows how to install and start working with TensorFlow using the modules system. This example shows how to install and start working with TensorFlow using the modules system.
```console ```console
marie@ml$ module load TensorFlow marie@power$ module load TensorFlow
Module TensorFlow/2.3.1-fosscuda-2019b-Python-3.7.4 and 47 dependencies loaded. Module TensorFlow/2.3.1-fosscuda-2019b-Python-3.7.4 and 47 dependencies loaded.
``` ```
...@@ -68,16 +71,16 @@ import TensorFlow: ...@@ -68,16 +71,16 @@ import TensorFlow:
!!! example !!! example
```console ```console
marie@ml$ ws_allocate -F scratch python_virtual_environment 1 marie@power$ ws_allocate -F /data/horse python_virtual_environment 1
Info: creating workspace. Info: creating workspace.
/scratch/ws/1/python_virtual_environment /data/horse/ws/1/python_virtual_environment
[...] [...]
marie@ml$ which python #check which python are you using marie@power$ which python #check which python are you using
/sw/installed/Python/3.7.2-GCCcore-8.2.0 /sw/installed/Python/3.7.2-GCCcore-8.2.0
marie@ml$ virtualenv --system-site-packages /scratch/ws/1/marie-python_virtual_environment/env marie@power$ virtualenv --system-site-packages /data/horse/ws/1/marie-python_virtual_environment/env
[...] [...]
marie@ml$ source /scratch/ws/1/marie-python_virtual_environment/env/bin/activate marie@power$ source /data/horse/ws/1/marie-python_virtual_environment/env/bin/activate
marie@ml$ python -c "import tensorflow as tf; print(tf.__version__)" marie@power$ python -c "import tensorflow as tf; print(tf.__version__)"
[...] [...]
2.3.1 2.3.1
``` ```
...@@ -105,7 +108,7 @@ Another option to use TensorFlow are containers. In the HPC domain, the ...@@ -105,7 +108,7 @@ Another option to use TensorFlow are containers. In the HPC domain, the
following example, we use the tensorflow-test in a Singularity container: following example, we use the tensorflow-test in a Singularity container:
```console ```console
marie@ml$ singularity shell --nv /scratch/singularity/powerai-1.5.3-all-ubuntu16.04-py3.img marie@power$ singularity shell --nv /data/horse/singularity/powerai-1.5.3-all-ubuntu16.04-py3.img
Singularity>$ export PATH=/opt/anaconda3/bin:$PATH Singularity>$ export PATH=/opt/anaconda3/bin:$PATH
Singularity>$ source activate /opt/anaconda3 #activate conda environment Singularity>$ source activate /opt/anaconda3 #activate conda environment
(base) Singularity>$ . /opt/DL/tensorflow/bin/tensorflow-activate (base) Singularity>$ . /opt/DL/tensorflow/bin/tensorflow-activate
...@@ -156,5 +159,5 @@ marie@compute$ module spider Keras ...@@ -156,5 +159,5 @@ marie@compute$ module spider Keras
[...] [...]
``` ```
to find out, which Keras modules are available on your partition. TensorFlow should be automatically to find out, which Keras modules are available on your cluster. TensorFlow should be automatically
loaded as a dependency. After loading the module, you can use Keras as usual. loaded as a dependency. After loading the module, you can use Keras as usual.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment