Skip to content
Snippets Groups Projects
Commit 45a7832e authored by Martin Schroschk's avatar Martin Schroschk
Browse files

Merge branch 'issue-496' into 'preview' 

Solve issue 496: Refine workspace documentation

See merge request !876
parents 05bc49e3 de5f038b
No related branches found
No related tags found
3 merge requests!884Automated merge from preview to main,!883Draft: Solve issue 496,!876Solve issue 496: Refine workspace documentation
Hide whitespace changes
Inline Side-by-side
doc.zih.tu-dresden.de/docs/data_lifecycle/workspaces.md
+ 89
− 37
View file @ 45a7832e
......@@ -28,10 +28,10 @@ times.
### List Available Filesystems
To list all available filesystems for using workspaces, you can either invoke `ws_list -l` or
`ws_find -l`, e.g.,
`ws_find --list`, e.g.,
```console
marie@login$ ws_find -l
marie@login$ ws_find --list
available filesystems:
scratch (default)
warm_archive
......@@ -44,7 +44,7 @@ beegfs
The default filesystem is `scratch`. If you prefer another filesystem (cf. section
[List Available Filesystems](#list-available-filesystems)), you have to explictly
provide the option `-F <fs>` to the workspace commands.
provide the option `--filesystem=<fs>` to the workspace commands.
### List Current Workspaces
......@@ -67,7 +67,7 @@ overview of some of these options. All available options can be queried by `ws_l
=== "Certain filesystem"
```
marie@login$ ws_list --filesystem scratch_fast
marie@login$ ws_list --filesystem=scratch_fast
id: numbercrunch
workspace directory : /lustre/ssd/ws/marie-numbercrunch
remaining time : 2 days 23 hours
......@@ -135,7 +135,7 @@ overview of some of these options. All available options can be queried by `ws_l
### Allocate a Workspace
To allocate a workspace in one of the listed filesystems, use `ws_allocate`. It is necessary to
specify a unique name and the duration of the workspace.
specify a unique name and the duration (in days) of the workspace.
```console
ws_allocate: [options] workspace_name duration
......@@ -154,31 +154,54 @@ Options:
-c [ --comment ] arg comment
```
!!! example
!!! example "Simple workspace allocation"
The simple way to allocate a workspace is calling `ws_allocate` command with two arguments,
where the first specifies the workspace name and the second the duration. This allocates a
workspace on the default filesystem with no e-mail reminder.
```console
marie@login$ ws_allocate -F scratch -r 7 -m marie.testuser@tu-dresden.de test-workspace 90
marie@login$ ws_allocate test-workspace 90
Info: creating workspace.
/scratch/ws/marie-test-workspace
remaining extensions : 10
remaining time in days: 90
```
This will create a workspace with the name `test-workspace` on the `/scratch` filesystem for 90
days with an email reminder for 7 days before the expiration.
!!! example "Workspace allocation on specific filesystem"
In order to allocate a workspace on a non-default filesystem, the option
`--filesystem <filesystem>` is required.
```console
marie@login$ ws_allocate --filesystem scratch_fast test-workspace 3
Info: creating workspace.
/lustre/ssd/ws/marie-test-workspace
remaining extensions : 2
remaining time in days: 3
```
!!! example "Workspace allocation with e-mail reminder"
!!! Note "Email reminder"
This command will create a workspace with the name `test-workspace` on the `/scratch` filesystem
with a duration of 90 days and send an e-mail reminder. The e-mail reminder will be sent every
day starting 7 days prior to expiration. We strongly recommend setting this e-mail reminder.
Setting the reminder to `7` means you will get a reminder email on every day starting `7` days
prior to expiration date. We strongly recommend to set this email reminder.
```console
marie@login$ ws_allocate --reminder 7 --mailaddress marie.testuser@tu-dresden.de test-workspace 90
Info: creating workspace.
/scratch/ws/marie-test-workspace
remaining extensions : 10
remaining time in days: 90
```
!!! Note "Name of a workspace"
The workspace name should help you to remember the experiment and data stored here. It has to
be unique on a certain filesystem. On the other hand it is possible to use the very same name
for workspaces on different filesystems.
The workspace name should help you to remember the experiment and data stored here. It has to
be unique on a certain filesystem. On the other hand it is possible to use the very same name
for workspaces on different filesystems.
Please refer to the section [section Cooperative Usage](#cooperative-usage-group-workspaces) for
Please refer to the [section Cooperative Usage](#cooperative-usage-group-workspaces) for
group workspaces.
### Extension of a Workspace
......@@ -186,7 +209,7 @@ group workspaces.
The lifetime of a workspace is finite and different filesystems (storage systems) have different
maximum durations. A workspace can be extended multiple times, depending on the filesystem.
| Filesystem (use with parameter `-F <fs>`) | Duration, days | Extensions | [Filesystem Feature](../jobs_and_resources/slurm.md#filesystem-features) | Remarks |
| Filesystem (use with parameter `--filesystem=<fs>`) | Duration, days | Extensions | [Filesystem Feature](../jobs_and_resources/slurm.md#filesystem-features) | Remarks |
|:-------------------------------------|---------------:|-----------:|:-------------------------------------------------------------------------|:--------|
| `scratch` (default) | 100 | 10 | `fs_lustre_scratch2` | Scratch filesystem (`/lustre/scratch2`, symbolic link: `/scratch`) with high streaming bandwidth, based on spinning disks |
| `ssd` | 30 | 2 | `fs_lustre_ssd` | High-IOPS filesystem (`/lustre/ssd`, symbolic link: `/ssd`) on SSDs. |
......@@ -205,7 +228,7 @@ remaining extensions : 1
remaining time in days: 100
```
Mail reminder settings are retained. I.e., previously set mail alerts apply to the extended
E-mail reminder settings are retained. I.e., previously set e-mail alerts apply to the extended
workspace, too.
!!! attention
......@@ -221,28 +244,57 @@ marie@login$ ws_extend -F scratch my-workspace 40
it will now expire in 40 days **not** 130 days.
### Send Reminder for Workspace Expiry Date
### Send Reminder for Workspace Expiration Date
We strongly recommend using one of the two provided ways to ensure that the expiration date of a
workspace is not forgotten.
#### Send Daily Reminder
Send a calendar invitation by Email to ensure that the expiration date of a workspace is not
forgotten
An e-mail reminder can be set at workspace allocation using
```console
marie@login$ ws_send_ical -F scratch my-workspace -m marie.testuser@tu-dresden.de
ws_allocate --reminder=<N> --mailaddress=<mail> [...]
```
This will send an e-mail every day starting `N` days prior to the expiration date.
See the [example above](#allocate-a-workspace) for reference.
If you missed setting an e-mail reminder at workspace allocation, you can add a reminder later, e.g.
```
marie@login$ ws_allocate --name=FancyExp --duration=17
[...]
marie@login$ ws_allocate --name=FancyExp --duration=17 --reminder=7 --mailaddress=marie@dlr.de
--extension
```
This will reallocate the workspace, which counts against your maximum number of reallocations (Note:
No data is deleted, but the database entry is modified).
#### Send Calender Invitation
The command `ws_send_ical` sends you an ical event on the expiration date of a specified workspace. This
calender invitation can be further managed according to your personal preferences. The syntax is
as follows:
```console
ws_send_ical --filesystem=<filesystem> --mail=<e-mail-address> --workspace=<workspace name>
```
### Deletion of a Workspace
To delete a workspace use the `ws_release` command. It is mandatory to specify the name of the
workspace and the filesystem in which it is located:
```console
marie@login$ ws_release -F scratch my-workspace
marie@login$ ws_release --filesystem=scratch --name=my-workspace
```
You can list your already released or expired workspaces using the `ws_restore -l` command.
You can list your already released or expired workspaces using the `ws_restore --list` command.
```console
marie@login$ ws_restore -l
marie@login$ ws_restore --list
warm_archive:
scratch:
marie-my-workspace-1665014486
......@@ -257,9 +309,9 @@ beegfs:
```
In this example, the user `marie` has three inactive, i.e., expired, workspaces namely
`my-workspace` in `scratch`, as well as `foo` and `bar` in `ssd` filesystem. The command `ws_restore
-l` lists the name of the workspace and the expiration date. As you can see, the expiration date is
added to the workspace name as Unix timestamp.
`my-workspace` in `scratch`, as well as `foo` and `bar` in `ssd` filesystem. The command
`ws_restore --list` lists the name of the workspace and the expiration date. As you can see, the
expiration date is added to the workspace name as Unix timestamp.
!!! hint "Deleting data in in an expired workspace"
......@@ -275,7 +327,7 @@ It performs the following steps once per day and filesystem:
- Check for remaining life time of all workspaces.
- If the workspaces expired, move it to a hidden directory so that it becomes inactive.
- Send reminder Emails to users if the reminder functionality was configured for their particular
- Send reminder e-mails to users if the reminder functionality was configured for their particular
workspaces.
- Scan through all workspaces in grace period.
- If a workspace exceeded the grace period, the workspace and its data are deleted.
......@@ -295,7 +347,7 @@ warm_archive: 2 months), you can still restore your data **into an existing work
Use
```console
marie@login$ ws_restore -l -F scratch
marie@login$ ws_restore --list --filesystem=scratch
scratch:
marie-my-workspace-1665014486
unavailable since Thu Oct 6 02:01:26 2022
......@@ -305,12 +357,12 @@ to get a list of your expired workspaces, and then restore them like that into a
workspace 'new_ws':
```console
marie@login$ ws_restore -F scratch marie-my-workspace-1665014486 new_ws
marie@login$ ws_restore --filesystem=scratch marie-my-workspace-1665014486 new_ws
```
The expired workspace has to be specified by its full name as listed by `ws_restore -l`, including
username prefix and timestamp suffix (otherwise, it cannot be uniquely identified). The target
workspace, on the other hand, must be given with just its short name, as listed by `ws_list`,
The expired workspace has to be specified by its full name as listed by `ws_restore --list`,
including username prefix and timestamp suffix (otherwise, it cannot be uniquely identified). The
target workspace, on the other hand, must be given with just its short name, as listed by `ws_list`,
without the username prefix.
Both workspaces must be on the same filesystem. The data from the old workspace will be moved into
......@@ -555,9 +607,9 @@ wrong name. Use only the short name that is listed after `id:` when using `ws_li
----
**Q**: Man, I've missed to specify mail alert when allocating my workspace. How can I add the mail
alert functionality to an existing workspace?
**Q**: I forgot to specify an e-mail alert when allocating my workspace. How can I add the
e-mail alert functionality to an existing workspace?
**A**: You can add the mail alert by "overwriting" the workspace settings via `ws_allocate -x -m
**A**: You can add the e-mail alert by "overwriting" the workspace settings via `ws_allocate -x -m
<mail address> -r <days> -n <ws-name> -d <duration> -F <fs>`. (This will lower the remaining
extensions by one.)
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
Nutzungsbedingungen | Datenschutzerklärung | Barrierefreiheitserklärung