Skip to content
Snippets Groups Projects
Commit 07c51d4c authored by Moe Jette's avatar Moe Jette
Browse files

wrap up what I can of the accounting web page

parent 9010ce1d
No related branches found
No related tags found
No related merge requests found
...@@ -13,7 +13,7 @@ records to a text file. ...@@ -13,7 +13,7 @@ records to a text file.
Beginning in SLURM version 1.3 accounting records can be written to Beginning in SLURM version 1.3 accounting records can be written to
a database. </p> a database. </p>
<p>There are three disticnt plugins associated with resource accounting. <p>There are three distinct plugins associated with resource accounting.
The configuration parameters associated with these plugins include: The configuration parameters associated with these plugins include:
<ul> <ul>
<li><b>JobCompType</b> controls how job completion information is <li><b>JobCompType</b> controls how job completion information is
...@@ -61,15 +61,19 @@ sacctmgr). ...@@ -61,15 +61,19 @@ sacctmgr).
Making information available to all users makes database security Making information available to all users makes database security
more difficult to provide, sending the data through an intermediate more difficult to provide, sending the data through an intermediate
daemon can provide better security. daemon can provide better security.
<a href="http://www.clusterresources.com/pages/products/gold-allocation-manager.php">Gold</a> Gold and SlurmDBD are two such services.
and SlurmDBD are two such services.
Our initial implementation relied upon Gold, but we found its Our initial implementation relied upon Gold, but we found its
performance to be inadequate for our needs and developed SlurmDBD. performance to be inadequate for our needs and developed SlurmDBD.
SlurmDBD (SLURM DataBase Deaemon) is written in C, multi-threaded, SlurmDBD (SLURM DataBase Deaemon) is written in C, multi-threaded,
secure, and orders of magnitude faster than Gold. secure, and much faster than Gold.
The configuration required to use SlurmDBD will be described below. The configuration required to use SlurmDBD will be described below.
Direct database or Gold use would be similar.</p> Direct database or Gold use would be similar.</p>
<p>Note that SlurmDBD relies upon existing SLURM plugins
for authentication and database use, but the other SLURM
commands and daemons are not required on the host where
SlurmDBD is installed.</p>
<h2>Infrastructure</h2> <h2>Infrastructure</h2>
<p>If the SlurmDBD is executed on a different cluster than the <p>If the SlurmDBD is executed on a different cluster than the
...@@ -161,11 +165,87 @@ Set to "jobcomp/slurmdbd".</li> ...@@ -161,11 +165,87 @@ Set to "jobcomp/slurmdbd".</li>
<p>SlurmDBD requires its own configuration file called "slurmdbd.conf". <p>SlurmDBD requires its own configuration file called "slurmdbd.conf".
This file should be only on the computer where SlurmDBD executes and This file should be only on the computer where SlurmDBD executes and
should only be readable by the user which executes SlurmDBD (e.g. "slurm"). should only be readable by the user which executes SlurmDBD (e.g. "slurm").
The configuration parameters required by SlurmDBD include:</p> This file should be protected from unauthorized access since it
contains a database password.
See "man slurmdbd.conf" for a more complete description of the
configuration parameters.
Some of the more important parameters include:</p>
<ul> <ul>
<li><b>TBD</b>: <li><b>AuthInfo</b>:
TBD.</li> If using SlurmDBD with a second Munge daemon, store the pathname of
the named socket used by Munge to provide enterprise-wide.
Otherwise the default Munge daemon will be used.</li>
<li><b>AuthType</b>:
Define the authentication method for communications between SLURM
components. A value of "auth/munge" is recommended.</li>
<li><b>DbdHost</b>:
The name of the machine where the Slurm Database Daemon is executed.
This should be a node name without the full domain name (e.g. "lx0001").
This value must be specified.</li>
<li><b>DbdPort</b>:
The port number that the Slurm Database Daemon (slurmdbd) listens
to for work. The default value is SLURMDBD_PORT as established at system
build time. If none is explicitly specified, it will be set to 6819.
This value must be equal to the \fBSlurmDbdPort\fR parameter in the
slurm.conf file.</li>
<li><b>LogFile</b>:
Fully qualified pathname of a file into which the Slurm Database Daemon's
logs are written.
The default value is none (performs logging via syslog).</li>
<li><b>PluginDir</b>:
Identifies the places in which to look for SLURM plugins.
This is a colon\-separated list of directories, like the PATH
environment variable.
The default value is "/usr/local/lib/slurm".</li>
<li><b>SlurmUser</b>:
The name of the user that the \fBslurmctld\fR daemon executes as.
This user must exist on the machine executing the Slurm Database Daemon
and have the same user ID as the hosts on which \fBslurmctld\fR execute.
For security purposes, a user other than "root" is recommended.
The default value is "root". </li>
<li><b>StorageHost</b>:
Define the name of the host the database is running where we are going
to store the data.
Ideally this should be the host on which slurmdbd executes.</li>
<li><b>StorageLoc</b>:
Specifies the location of the database where accounting
records are written.</li>
<li><b>StoragePass</b>:
Define the password used to gain access to the database to store
the job accounting data.</li>
<li><b>StoragePort</b>:
Define the password used to gain access to the database to store
the job accounting data.</li>
<li><b>StorageType</b>:
Define the accounting storage mechanism type.
Acceptable values at present include
"accounting_storage/gold", "accounting_storage/mysql", and
"accounting_storage/pgsql".
The value "accounting_storage/gold" indicates that account records
will be written to Gold, which maintains its own database.
The value "accounting_storage/mysql" indicates that accounting records
should be written to a MySQL database specified by the
\fStorageLoc\fR parameter.
The value "accounting_storage/pgsql" indicates that accounting records
should be written to a PostgreSQL database specified by the
\fBStorageLoc\fR parameter.
This value must be specified.</li>
<li><b>StorageUser</b>:
Define the name of the user we are going to connect to the database
with to store the job accounting data.</li>
</ul> </ul>
<h2>Database Configuration</h2> <h2>Database Configuration</h2>
...@@ -174,7 +254,18 @@ TBD.</li> ...@@ -174,7 +254,18 @@ TBD.</li>
to as an <i>Association</i>, which consists of four elements: to as an <i>Association</i>, which consists of four elements:
cluster name, partition name, user name, and account name. </p> cluster name, partition name, user name, and account name. </p>
<p>TBD</p> <p><b>TBD</b></p>
<h2>Tools</h2>
<p>There are two tools available to work with accounting data,
sacct and sacctmgr. Both of these tools will get or set data
through the SlurmDBD daemon.
Sacct is used to generate accounting report for both running and
completed jobs.
Sacctmgr is used to manage associations in the database:
add or remove clusters, add or remove users, etc.
See the man pages for each command for more information.</p>
<p style="text-align:center;">Last modified 12 March 2008</p> <p style="text-align:center;">Last modified 12 March 2008</p>
......
...@@ -15,6 +15,10 @@ The size of each line in the file is limited to 1024 characters. ...@@ -15,6 +15,10 @@ The size of each line in the file is limited to 1024 characters.
Changes to the configuration file take effect upon restart of Changes to the configuration file take effect upon restart of
SlurmDbd or daemon receipt of the SIGHUP signal unless otherwise noted. SlurmDbd or daemon receipt of the SIGHUP signal unless otherwise noted.
.LP .LP
This file should be only on the computer where SlurmDBD executes and
should only be readable by the user which executes SlurmDBD (e.g. "slurm").
This file should be protected from unauthorized access since it
contains a database password.
The overall configuration parameters available include: The overall configuration parameters available include:
.TP .TP
...@@ -86,12 +90,24 @@ Fully qualified pathname of a file into which the Slurm Database Daemon's ...@@ -86,12 +90,24 @@ Fully qualified pathname of a file into which the Slurm Database Daemon's
logs are written. logs are written.
The default value is none (performs logging via syslog). The default value is none (performs logging via syslog).
.TP
\fBMessageTimeout\fR
Time permitted for a round\-trip communication to complete
in seconds. Default value is 10 seconds.
.TP .TP
\fBPidFile\fR \fBPidFile\fR
Fully qualified pathname of a file into which the Slurm Database Daemon Fully qualified pathname of a file into which the Slurm Database Daemon
may write its process ID. This may be used for automated signal processing. may write its process ID. This may be used for automated signal processing.
The default value is "/var/run/slurmdbd.pid". The default value is "/var/run/slurmdbd.pid".
.TP
\fBPluginDir\fR
Identifies the places in which to look for SLURM plugins.
This is a colon\-separated list of directories, like the PATH
environment variable.
The default value is "/usr/local/lib/slurm".
.TP .TP
\fBSlurmUser\fR \fBSlurmUser\fR
The name of the user that the \fBslurmctld\fR daemon executes as. The name of the user that the \fBslurmctld\fR daemon executes as.
...@@ -107,7 +123,12 @@ to store the data. ...@@ -107,7 +123,12 @@ to store the data.
Ideally this should be the host on which slurmdbd executes. Ideally this should be the host on which slurmdbd executes.
.TP .TP
\fBStoragePassword\fR \fBStorageLoc\fR
Specifies the location of the database where accounting
records are written.
.TP
\fBStoragePass\fR
Define the password used to gain access to the database to store Define the password used to gain access to the database to store
the job accounting data. the job accounting data.
...@@ -118,8 +139,22 @@ the job accounting data. ...@@ -118,8 +139,22 @@ the job accounting data.
.TP .TP
\fBStorageType\fR \fBStorageType\fR
Define the port the database server is listening on where we are going Define the accounting storage mechanism type.
to store the data. Acceptable values at present include
"accounting_storage/gold", "accounting_storage/mysql", and
"accounting_storage/pgsql".
The value "accounting_storage/gold" indicates that account records
will be written to Gold
(http://www.clusterresources.com/pages/products/gold-allocation-manager.php),
which maintains its own database.
The value "accounting_storage/mysql" indicates that accounting records
should be written to a MySQL database specified by the
\fStorageLoc\fR parameter.
The value "accounting_storage/pgsql" indicates that accounting records
should be written to a PostgreSQL database specified by the
\fBStorageLoc\fR parameter.
This value must be specified.
.TP .TP
\fBStorageUser\fR \fBStorageUser\fR
Define the name of the user we are going to connect to the database Define the name of the user we are going to connect to the database
...@@ -137,6 +172,8 @@ AuthInfo=/var/run/munge/munge.socket.2 ...@@ -137,6 +172,8 @@ AuthInfo=/var/run/munge/munge.socket.2
.br .br
AuthType=auth/munge AuthType=auth/munge
.br .br
DbdHost=db_host
.br
DebugLevel=4 DebugLevel=4
.br .br
LogFile=/var/log/slurmdbd.log LogFile=/var/log/slurmdbd.log
...@@ -145,7 +182,9 @@ PidFile=/var/tmp/jette/slurmdbd.pid ...@@ -145,7 +182,9 @@ PidFile=/var/tmp/jette/slurmdbd.pid
.br .br
SlurmUser=slurm_mgr SlurmUser=slurm_mgr
.br .br
StoragePassword=shazaam StoragePass=shazaam
.br
StorageType=accounting_storage/mysql
.br .br
StorageUser=database_mgr StorageUser=database_mgr
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment