Commit f8744557 authored by Robert Dietrich's avatar Robert Dietrich

improved PIKA conf files and added more documentation

parent e5c43593
......@@ -4,8 +4,29 @@ PIKA is an infrastructure for continuous monitoring and analysis of HPC systems.
It uses the collection daemon collectd, InfluxDB to store time-series data and MariaDB to store job metadata.
Furthermore, it provides a powerful web-frontend for the visualization of job data.
Files that are required for the execution of the monitoring daemon (collectd), are located in the daemon folder. This includes the collectd configuration file and the LIKWID event group files as well as scripts that are periodically triggered to perform log rotation and error detection.
Prolog and epilog scripts ensure that the PIKA package is installed and the daemon is running. Corresponding files are located in the folder job_control.
Scripts for post processing, such as the generation of footprints, are located in the post-processing folder.
Scripts to determine the scalability and the overhead of the monitoring as well as regression tests are located in the test folder.
## Installation
The software stack consists of several components and tools.
To simplify the installation, appropriate install scripts are available.
For detailed install instructions see the README.md in the install directory.
For detailed install instructions see the [README.md](install/README.md) in the install directory.
## Configuration
Five files are used to configure the software stack.
*pika.conf*
contains the global version independent configuration variables. It also sets some environment variables that are used in the job prolog and epilog. It sources *.pika_access*.
*.pika_access*
exports the environment variables with the access parameters for the databases. Thus, this file should have restricted read access.
*pika-VERSION.conf*
is used for versioning of the PIKA package. It sets the PIKA package version along with the used version of collectd, LIKWID and Python. Finally, it sources *pika.conf*.
*pika_utils.conf*
provides utility functions for prolog, epilog and other bash scripts.
......@@ -6,6 +6,7 @@ export LIKWID_VERSION=git #5.0.1
export PIKA_ROOT=/sw/taurus/tools/pika
# different configuration for different partitions
if [[ $(hostname -s) = taurusml* ]]; then
export PYTHON_VERSION=3.6.10
export PIKA_PACKAGE_PATH=${PIKA_ROOT}/archives/pika-${PIKA_VERSION}-ml.tar.gz
......
#!/bin/bash
# connection data for databases
source /sw/taurus/tools/pika/.pika_access
# install path of PIKA package
export PIKA_INSTALL_PATH=/opt/pika/${PIKA_VERSION}
#export PIKA_INSTALL_PATH=/sw/taurus/tools/prope/sw/prope/${PIKA_VERSION}
#software dependencies
export PYTHON_ROOT=${PIKA_INSTALL_PATH}/python/${PYTHON_VERSION}
export PYTHONPATH=$PYTHON_ROOT
export COLLECTD_PATH=${PIKA_INSTALL_PATH}/collectd/${COLLECTD_VERSION}
if [[ ${PYTHON_VERSION} = "3.6."* ]]; then
export LD_LIBRARY_PATH=${PYTHON_ROOT}/lib:${PYTHON_ROOT}/lib/python3.6:${LD_LIBRARY_PATH}
elif [[ ${PYTHON_VERSION} = "3.7."* ]]; then
export LD_LIBRARY_PATH=${PYTHON_ROOT}/lib:${PYTHON_ROOT}/lib/python3.7:${LD_LIBRARY_PATH}
else
export LD_LIBRARY_PATH=${PYTHON_ROOT}/lib:${LD_LIBRARY_PATH}
fi
export PATH=$PYTHON_ROOT/bin:$PATH
### global configuration ###
### variables used in prolog and epilog scripts
# variables used in prolog and epilog scripts
export PIKA_DEBUG=1
export LOCAL_STORE=/dev/shm
export REDIS_HOST="10.1.129.243"
......@@ -33,3 +13,25 @@ export PIKA_COLLECTD_LOGFILE=${PIKA_LOGPATH}/pika_collectd.log
export PIKA_COLLECTD_CONF=/tmp/pika_collectd.conf
export PIKA_COLLECTD_PID_FILE=/tmp/pika_collectd.pid
export PIKA_COLLECTD_SOCKET=/tmp/pika_collectd.sock
### END: global configuration ###
### usually there is no need to touch the following ###
# access parameters for databases
source /sw/taurus/tools/pika/.pika_access
# install path of PIKA package (on compute node)
export PIKA_INSTALL_PATH=/opt/pika/${PIKA_VERSION}
# set paths to find the PIKA python
export PYTHON_ROOT=${PIKA_INSTALL_PATH}/python/${PYTHON_VERSION}
export PYTHONPATH=$PYTHON_ROOT
python_major_version=`echo ${PYTHON_VERSION} | cut -d '.' -f1,2`
export LD_LIBRARY_PATH=${PYTHON_ROOT}/lib:${PYTHON_ROOT}/lib/python${python_major_version}:${LD_LIBRARY_PATH}
export PATH=$PYTHON_ROOT/bin:$PATH
# expose the collectd install path
export COLLECTD_PATH=${PIKA_INSTALL_PATH}/collectd/${COLLECTD_VERSION}
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment