Commit bbd12c54 authored by Robert Dietrich's avatar Robert Dietrich

updated documentation

parent 100f7a0f
......@@ -5,10 +5,8 @@ It uses the collection daemon collectd, InfluxDB to store time-series data and M
Furthermore, it provides a powerful web-frontend for the visualization of job data.
Files that are required for the execution of the monitoring daemon (collectd), are located in the daemon folder. This includes the collectd configuration file and the LIKWID event group files as well as scripts that are periodically triggered to perform log rotation and error detection.
Prolog and epilog scripts ensure that the PIKA package is installed and the daemon is running. Corresponding files are located in the folder job_control.
Scripts for post processing, such as the generation of footprints, are located in the post-processing folder.
Scripts for post processing, such as the generation of footprints, are located in the post_processing folder.
Scripts to determine the scalability and the overhead of the monitoring as well as regression tests are located in the test folder.
## Installation
......@@ -20,13 +18,13 @@ For detailed install instructions see the [README.md](install/README.md) in the
Five files are used to configure the software stack.
*pika.conf*
contains the global version independent configuration variables. It also sets some environment variables that are used in the job prolog and epilog. It sources *.pika_access*.
contains the global version independent configuration variables. It also sets some environment variables that are used in the job prolog and epilog. It uses `source` to read the environment variables from *.pika_access*.
*.pika_access*
exports the environment variables with the access parameters for the databases. Thus, this file should have restricted read access.
*pika-VERSION.conf*
is used for versioning of the PIKA package. It sets the PIKA package version along with the used version of collectd, LIKWID and Python. Finally, it sources *pika.conf*.
is used for versioning of the PIKA package. It sets the PIKA package version along with the used version of collectd, LIKWID and Python. Finally, it uses `source` to read the environment variables from *pika.conf*.
*pika_utils.conf*
provides utility functions for prolog, epilog and other bash scripts.
......@@ -4,15 +4,26 @@ The installation consists of two parts, the package for the node monitoring daem
## Node Monitoring Daemon
We prepared simple bash scripts that can be used to generate the monitoring package.
*pika_install.conf*, a symbolic link to a *pika_install-VERSION.conf* file, sets a few environment variables that are used by the install scripts located in the *compute_node* directory.
*pika_install.conf*, a symbolic link to a *pika_install-VERSION.conf* file, sets some environment variables that are used by the install scripts located in the *compute_node* directory.
*install_pika.sh* is the main script. It deletes any old installation of the PIKA package. Then it runs *install_python3.sh*, *install_likwid.sh* and *install_collectd.sh* (For a manual installation, make sure that collectd is installed last.) Finally, a tarball is created which can be unpacked on the target nodes.
All scripts use the `source` command to read the environment from *pika_install.conf* and can be executed standalone.
Called as *root*, the *PIKA_INSTALL_PATH* is used as build and install directory, otherwise *PIKA_BUILD_PATH* is used.
All scripts source *pika_install.conf* and can be executed standalone. Called as *root*, the *PIKA_INSTALL_PATH* is used as build and install directory, otherwise *PIKA_BUILD_PATH* is used.
*install_pika.sh* is the main script. It deletes any old installation of the PIKA package.
Then it runs *install_python3.sh*, *install_likwid.sh* and *install_collectd.sh*.
(For a manual installation, make sure that collectd is installed last.)
Finally, a tarball is created which can be unpacked on the target nodes.
Depending on the configuration (OS, software, hardware, etc.) different packages have to be created for each system.
This may require adding more if branches to the configuration files.
A PIKA package should be created on a node of the partition or system on which it will later run.
The default LIKWID access mode is perf_event. *install_pika.sh* and *install_likwid.sh* optionally accept the paramter *direct* to use the direct access mode to the MSR registers.
Depending on the configuration (OS, software, hardware, etc.) different packages have to be created for each system. This may require adding more if branches to the configuration files. A PIKA package should be created on a node of the partition or system on which it will later run.
The folder compute_node/patches contains patches that can be applied to collectd and LIKWID.
The [collectd daemon patch](compute_node/patches/collectd-5.10.0_daemon.patch) adds the read alignment options (https://github.com/collectd/collectd/pull/3327).
The [LIKWID Set-Counters-Patch](compute_node/patches/pika_likwid-5.0.0_src.patch) adds an API function to (re)set the active counters.
All other patches are deprecated with the most recent versions of collectd and LIKWID.
## Databases for Time-Series Data and Metadata
It is recommended to install the databases on different systems or virtual machines, each having access to fast storage.
......@@ -62,12 +73,14 @@ Create users, a database and a retention policy using the InfluxDB shell (*influ
CREATE USER admin WITH PASSWORD 'password' WITH ALL PRIVILEGES
CREATE DATABASE pika
CREATE RETENTION POLICY shortterm ON pika DURATION 28d REPLICATION 1 SHARD DURATION 7d DEFAULT
CREATE DATABASE pika_lt
CREATE RETENTION POLICY longterm ON pika DURATION INF REPLICATION 1 SHARD DURATION 7d
CREATE USER readonly WITH PASSWORD 'password'
GRANT READ ON "pika" TO "readonly"
GRANT READ ON "pika_lt" TO "readonly"
### MariaDB
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment