diff --git a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md
index 87263d924319cb7d3aae6bdf16c55eb3d0551afc..0c980b79f9135a775d0135ce943d3e2e8c494a8e 100644
--- a/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md
+++ b/doc.zih.tu-dresden.de/docs/data_lifecycle/lustre.md
@@ -108,7 +108,7 @@ line mode within this documentation.
!!! hint "Filesystem vs. Path"
If you provide a path to the lfs commands instead of a filesystem, the lfs option is applied to
- the filesystem this path is in. Thus, the provied information refers to the whole filesystem,
+ the filesystem this path is in. Thus, the passed information refers to the whole filesystem,
not the path.
You can retrieve a complete list of available options:
diff --git a/doc.zih.tu-dresden.de/docs/data_lifecycle/workspaces.md b/doc.zih.tu-dresden.de/docs/data_lifecycle/workspaces.md
index 12fa5ff4a4f8cb117228e702435e6455109a5642..7e3049bca90705b4a74644d030490306bef6527d 100644
--- a/doc.zih.tu-dresden.de/docs/data_lifecycle/workspaces.md
+++ b/doc.zih.tu-dresden.de/docs/data_lifecycle/workspaces.md
@@ -27,7 +27,8 @@ times.
### List Available Filesystems
-To list all available filesystems for using workspaces, use:
+To list all available filesystems for using workspaces, you can either invoke `ws_list -l` or
+`ws_find -l`, e.g.,
```console
marie@login$ ws_find -l
@@ -41,12 +42,13 @@ beegfs
!!! note "Default is `scratch`"
- The default filesystem is `scratch`. If you prefer another filesystem, provide the option `-F
- <fs>` to the workspace commands.
+ The default filesystem is `scratch`. If you prefer another filesystem (cf. section
+ [List Available Filesystems](#list-available-filesystems)), you have to explictly
+ provide the option `-F <fs>` to the workspace commands.
### List Current Workspaces
-To list all workspaces you currently own, use:
+The command `ws_list` lists all your currently active (,i.e, not expired) workspaces, e.g.
```console
marie@login$ ws_list
@@ -59,13 +61,84 @@ id: test-workspace
available extensions : 10
```
+The output of `ws_list` can be customized via several options. The following switch tab provides a
+overview of some of these options. All available options can be queried by `ws_list --help`.
+
+=== "Certain filesystem"
+
+ ```
+ marie@login$ ws_list --filesystem scratch_fast
+ id: numbercrunch
+ workspace directory : /lustre/ssd/ws/marie-numbercrunch
+ remaining time : 2 days 23 hours
+ creation time : Thu Mar 2 14:15:33 2023
+ expiration date : Sun Mar 5 14:15:33 2023
+ filesystem name : ssd
+ available extensions : 2
+ ```
+
+=== "Verbose output"
+
+ ```
+ marie@login$ ws_list -v
+ id: test-workspace
+ workspace directory : /scratch/ws/0/marie-test-workspace
+ remaining time : 89 days 23 hours
+ creation time : Thu Jul 29 10:30:04 2021
+ expiration date : Wed Oct 27 10:30:04 2021
+ filesystem name : scratch
+ available extensions : 10
+ acctcode : p_numbercrunch
+ reminder : Sat Oct 20 10:30:04 2021
+ mailaddress : marie@tu-dresden.de
+ ```
+
+=== "Terse output"
+
+ ```
+ marie@login$ ws_list -t
+ id: test-workspace
+ workspace directory : /scratch/ws/0/marie-test-workspace
+ remaining time : 89 days 23 hours
+ available extensions : 10
+ id: foo
+ workspace directory : /scratch/ws/0/marie-foo
+ remaining time : 3 days 22 hours
+ available extensions : 10
+ ```
+
+=== "Show only names"
+
+ ```
+ marie@login$ ws_list -s
+ test-workspace
+ foo
+ ```
+
+=== "Sort by remaining time"
+
+ You can list your currently allocated workspace by remaining time. This is especially useful
+ for housekeeping tasks, such as extending soon expiring workspaces if necessary.
+
+ ```
+ marie@login$ ws_list -R -t
+ id: test-workspace
+ workspace directory : /scratch/ws/0/marie-test-workspace
+ remaining time : 89 days 23 hours
+ available extensions : 10
+ id: foo
+ workspace directory : /scratch/ws/0/marie-foof
+ remaining time : 3 days 22 hours
+ available extensions : 10
+ ```
+
### Allocate a Workspace
-To create a workspace in one of the listed filesystems, use `ws_allocate`. It is necessary to
+To allocate a workspace in one of the listed filesystems, use `ws_allocate`. It is necessary to
specify a unique name and the duration of the workspace.
```console
-marie@login$ ws_allocate: [options] workspace_name duration
+ws_allocate: [options] workspace_name duration
Options:
-h [ --help] produce help message
@@ -94,10 +167,19 @@ Options:
This will create a workspace with the name `test-workspace` on the `/scratch` filesystem for 90
days with an email reminder for 7 days before the expiration.
-!!! Note
+!!! Note "Email reminder"
Setting the reminder to `7` means you will get a reminder email on every day starting `7` days
- prior to expiration date.
+ prior to expiration date. We strongly recommend to set this email reminder.
+
+!!! Note "Name of a workspace"
+
+ The workspace name should help you to remember the experiment and data stored here. It has to
+ be unique on a certain filesystem. On the other hand it is possible to use the very same name
+ for workspaces on different filesystems.
+
+Please refer to the section [section Cooperative Usage](#cooperative-usage-group-workspaces) for
+group workspaces.
### Extension of a Workspace
@@ -255,7 +337,7 @@ There are three typical options for the use of workspaces:
### Per-Job Storage
-The idea of a "workspace per-job storage" adresses the need of a batch job for a directory for
+The idea of a "workspace per-job storage" addresses the need of a batch job for a directory for
temporary data which can be deleted afterwards. To help you to write your own
[(Slurm) job file](../jobs_and_resources/slurm.md#job-files), suited to your needs, we came up with
the following example (which works [for the program g16](../software/nanoscale_simulations.md)).
@@ -391,6 +473,57 @@ marie@login$ qinfo quota /warm_archive/ws/
Note that the workspaces reside under the mountpoint `/warm_archive/ws/` and not `/warm_archive`
anymore.
+## Cooperative Usage (Group Workspaces)
+
+When a workspace is created with the option `-g, --group`, it gets a group workspace that is visible
+to others (if in the same group) via `ws_list -g`.
+
+!!! hint "Chose group"
+
+ If you are member of multiple groups, than the group workspace is visible for your primary
+ group. You can list all groups you belong to via `groups`, and the first entry is your
+ primary group.
+
+ Nevertheless, you can create a group workspace for any of your groups following these two
+ steps:
+
+ 1. Change to the desired group using `newgrp <other-group>`.
+ 1. Create the group workspace as usual, i.e., `ws_allocate --group [...]`
+
+ The [page on Sharing Data](data_sharing.md) provides
+ information on how to grant access to certain colleagues and whole project groups.
+
+!!! Example "Allocate and list group workspaces"
+
+ If Marie wants to share results and scripts in a workspace with all of her colleagues
+ in the project `p_number_crunch`, she can allocate a so-called group workspace.
+
+ ```console
+ marie@login$ ws_allocate --group --name numbercrunch --duration 30
+ Info: creating workspace.
+ /scratch/ws/0/marie-numbercrunch
+ remaining extensions : 10
+ remaining time in days: 30
+ ```
+
+ This workspace directory is readable for the group, e.g.,
+
+ ```console
+ marie@login$ ls -ld /scratch/ws/0/marie-numbercrunch
+ drwxr-x--- 2 marie p_number_crunch 4096 Mar 2 15:24 /scratch/ws/0/marie-numbercrunch
+ ```
+
+ All members of the project group `p_number_crunch` can now list this workspace using
+ `ws_list -g` and access the data (read-only).
+
+ ```console
+ martin@login$ ws_list -g -t
+ id: numbercrunch
+ workspace directory : /scratch/ws/0/marie-numbercrunch
+ remaining time : 29 days 23 hours
+ available extensions : 10
+ ```
+
## FAQ and Troubleshooting
**Q**: I am getting the error `Error: could not create workspace directory!`
diff --git a/doc.zih.tu-dresden.de/util/check-TOC-equals-page-headings.py b/doc.zih.tu-dresden.de/util/check-TOC-equals-page-headings.py
deleted file mode 100644
index 16097c70ac1aff3ab819205d7289ffb512b7f486..0000000000000000000000000000000000000000
--- a/doc.zih.tu-dresden.de/util/check-TOC-equals-page-headings.py
+++ /dev/null
@@ -1,44 +0,0 @@
-""" #!/usr/bin/env python
-Check for consistency between TOC and page headings.
-Provide as an command line argument the path to the mkdocs.yml file.
-
-Author: Taras Lazariv
-"""
-
-import os
-import sys
-import yaml
-import pandas as pd
-
-def list_and_read_files(path):
- "List files in a directory recursively and read the first line of each file"
- files = []
- firstline = []
- for root, _, filenames in os.walk(path):
- for filename in filenames:
- if filename.endswith('.md'):
- files.append(os.path.join(root.split('/')[-1], filename))
- firstline.append(open(os.path.join(root, filename)).readline().strip().replace('# ',''))
- df = pd.DataFrame({'file': files, 'firstline': firstline})
- return df
-
-def main():
- "Main function"
- path = os.getcwd()
- df = list_and_read_files(path)
-
- nav_section = dict()
-
- with open(sys.argv[1], 'r') as file:
- for line in file:
- line = line.rstrip()
- if line.endswith('.md'):
- nav_section.update(yaml.safe_load(line)[0])
-
- nav_df = pd.DataFrame(nav_section.items(), columns=['title', 'file'])
- with pd.option_context('display.max_rows', None): # more options can be specified also
- complete_nav_df = pd.merge(df, nav_df, on='file', how='outer')
- print(complete_nav_df.loc[~(complete_nav_df['firstline'] == complete_nav_df['title'])])
-
-if __name__ == '__main__':
- main()
\ No newline at end of file
diff --git a/doc.zih.tu-dresden.de/util/check-toc-equals-page-headings.py b/doc.zih.tu-dresden.de/util/check-toc-equals-page-headings.py
new file mode 100755
index 0000000000000000000000000000000000000000..ec66904b419c857fb194a38daf47e1a5b4677bb1
--- /dev/null
+++ b/doc.zih.tu-dresden.de/util/check-toc-equals-page-headings.py
@@ -0,0 +1,72 @@
+""" #!/usr/bin/env python
+Check for consistency between TOC and page headings.
+Provide as an command line argument the path to the mkdocs.yml file.
+
+Author: Michael Bommhardt-Richter
+"""
+
+import argparse
+import sys
+from pathlib import Path
+
+# {path/filename.md: [toc_heading, file_heading], ... }
+TOCData = dict()
+
+whitelist = ["index.md"] # ["archive"]
+
+
+def get_heading_in_file(filename, docs_path):
+ # Read until first level one heading is found
+ f = Path.joinpath(docs_path, filename)
+ with open(f, "r") as file:
+ for line in file:
+ if line.startswith("#"):
+ # TODO Make sure it is really a level one heading!
+ # Will be empty if there is more than one "#".
+ return line.split("#")[1].strip()
+
+
+def main():
+ scriptpath = Path(__file__).resolve().parent
+ mkdocsyaml = Path.joinpath(scriptpath, "../", "mkdocs.yml")
+ if Path.exists(mkdocsyaml):
+
+ docs_path = Path.joinpath(scriptpath, "../", "docs")
+ with open(mkdocsyaml, "r") as file:
+ c = file.readlines()
+
+ for line in c:
+ line = line.rstrip()
+
+ # "headline: path/file.md" -> "Headline" = "path/file.md"
+ if line.endswith(".md"):
+ line = line.split(" - ")[1]
+ line = line.split(": ")
+
+ key = line[1]
+ file_heading = get_heading_in_file(line[1], docs_path)
+ TOCData[line[1]] = [line[0], file_heading]
+
+ # Check TOC vs heading in corresponding md-file
+ cnt = 0
+ for key, value in TOCData.items():
+ if key in whitelist:
+ continue
+ if value[0] == "Overview":
+ continue
+ if value[0] != value[1]:
+ cnt += 1
+ print(f"{key:<40}{value[0]:<30} != {value[1]}")
+ sys.exit(cnt)
+ else:
+ print("Error: Could not find mkdocs.yml file.")
+ sys.exit(-1)
+
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser(
+ description="Find differences in TOC and top level headings of md-files."
+ )
+ args = parser.parse_args()
+
+ main()