Add spank(8) manpage

doc/man/Makefile.am

@@ -85,7 +85,8 @@ man5_MANS = man5/bluegene.conf.5 \
 	man5/slurm.conf.5
 
 man8_MANS = man8/slurmctld.8 \
-	man8/slurmd.8
+	man8/slurmd.8 \
+	man8/spank.8
 
 dist-hook:
 	-rm -rf `find $(distdir) -name CVS`

doc/man/man5/slurm.conf.5

@@ -226,15 +226,13 @@ environment variable.
 The default value is "/usr/local/lib/slurm".
 .TP
 \fBPlugStackConfig\fR
-Location of the config file for SLURM stackable plugins that
-use the Stackable Plugin Architecture for Node job Kontrol
-(SPANK).
-This provides support for a highly configurable set of plugins 
-to be called before and/or after execution of each task 
-spawned as part of a user's job step. 
-More detailed documentation will be provided soon. 
-Default location is "plugstack.conf" in the same directory 
-as the system slurm.conf.
+Location of the config file for SLURM stackable plugins that use
+the Stackable Plugin Architecture for Node job (K)control (SPANK).
+This provides support for a highly configurable set of plugins to
+be called before and/or after execution of each task spawned as
+part of a user's job step.  Default location is "plugstack.conf"
+in the same directory as the system slurm.conf. For more information
+on SPANK plugins, see the \fBspank\fR(8) manual.
 .TP
 \fBProctrackType\fR
 Identifies the plugin to be used for process tracking. 
@@ -974,5 +972,5 @@ details.
 .SH "SEE ALSO"
 .LP
 \fBgethostbyname\fR(3), \fBgroup\fR(5), \fBhostname\fR(1), 
-\fBscontrol\fR(1), \fBslurmctld\fR(8), \fBslurmd\fR(8),
+\fBscontrol\fR(1), \fBslurmctld\fR(8), \fBslurmd\fR(8), \fBspank(8)\fR,
 \fBsyslog\fR(2)

doc/man/man8/spank.8

+.TH "SPANK" "5" "May 2006" "SPANK" "SLURM plug-in architecture for Node and job (K)control"
+.SH "NAME"
+\fBSPANK\fR \- SLURM Plug-in Architecture for Node and job (K)control 
+.SH "DESCRIPTION"
+This manual briefly describes the capabilities of the SLURM Plug-in
+architecture for Node and job Kontrol (\fBSPANK\fR) as well as the \fBSPANK\fR
+configuration file: (By default: \fBplugstack.conf\fP.)
+.LP
+\fBSPANK\fR provides a very generic interface for stackable plug-ins 
+which may be used to dynamically modify the job launch code in
+SLURM. \fBSPANK\fR plugins may be built without access to SLURM source
+code. They need only be compiled against SLURM's \fBspank.h\fR header file,
+added to the \fBSPANK\fR config file \fBplugstack.conf\fR,
+and they will be loaded at runtime during the next job launch. Thus,
+the \fBSPANK\fR infrastructure provides administrators and other developers
+a low cost, low effort ability to dynamically modify the runtime
+behavior of SLURM job launch.
+.LP
+\fBSPANK\fR plugins may be called from multiple points during the SLURM job
+launch. A plugin may define the following functions:
+.TP 2
+\fBslurm_spank_init\fR 
+Called just after job step is initialized.
+.TP
+\fBslurm_spank_user_init\fR 
+Called after privileges are temporarily dropped.
+.TP
+\fBslurm_spank_task_init\fR 
+Called for each task just before execve(2).
+.TP
+\fBslurm_spank_task_post_fork\fR 
+Called for each task from parent process after fork(2) is complete.
+Due to the fact that \fBslurmd\fR does not exec any tasks until all
+tasks have completed fork(2), this call is guaranteed to run before
+the user task is executed.
+.TP
+\fBslurm_spank_task_exit\fR
+Called for each task as its exit status is collected by SLURM.
+.TP
+\fBslurm_spank_exit\fR
+Called once just before \fBslurmstepd\fR exits.
+.LP
+All of these functions have the same prototype, for example:
+.nf
+
+   slurm_spank_init (spank_t spank, int ac, char *argv[])
+
+.fi
+.LP
+Where \fBspank\fR is the \fBSPANK\fR handle which must be passed back to
+SLURM when the plugin calls functions like \fBspank_get_item\fR and
+\fBspank_getenv\fR. Configured arguments (See \fBCONFIGURATION\fR
+below) are passed in the argument vector \fBargv\fR with argument
+count \fBac\fR.
+.LP
+\fBSPANK\fR plugins do not have direct access to internally definied SLURM 
+data structures. Instead, information about the currently executing
+job is obtained via the \fBspank_get_item\fR function call.
+.nf
+
+  spank_err_t spank_get_item (spank_t spank, spank_item_t item, ...);
+
+.fi
+The \fBspank_get_item\fR call must be passed the current \fBSPANK\fR
+handle as well as the item requested, which is defined by the
+passed \fBspank_item_t\fR. A variable number of pointer arguments are also
+passed, depending on which item was requested by the plugin. A
+list of the valid values for \fBitem\fR is kept in the \fBspank.h\fR header
+file. Some examples are:
+.TP 2
+\fBS_JOB_UID\fR
+User id for running job. (uid_t *) is third arg of \fBspank_get_item\fR
+.TP
+\fBS_JOB_STEPID\fR
+Job step id for running job. (uint32_t *) is third arg of \fBspank_get_item\fR
+.TP
+\fBS_TASK_EXIT_STATUS\fR
+Exit status for exited task. Only valid from \fBslurm_spank_task_exit\fR.
+(int *) is third arg of \fBspank_get_item\fR.
+.TP
+\fBS_JOB_ARGV\fR
+Complete job command line. Third and fourth args to \fBspank_get_item\fR
+are (int *, char ***).
+.LP
+See \fBspank.h\fR for more details, and \fBEXAMPLES\fR below for an example
+of \fBspank_get_item\fR usage.
+.LP
+\fBSPANK\fR plugins may also use the \fBspank_getenv\fR and
+\fBspank_setenv\fR functions to view and modify the job's
+environment. \fBspank_getenv\fR searches the job's environment for
+the environment variable \fIvar\fR and copies the current value
+into a buffer \fIbuf\fR of length \fIlen\fR.  \fBspank_setenv\fR
+allows a \fBSPANK\fR plugin to set or overwrite a variable in the job's
+environment. The prototypes are:
+.nf
+
+ spank_err_t spank_getenv (spank_t spank, const char *var, 
+		           char *buf, int len);
+ spank_err_t spank_setenv (spank_t spank, const char *var, 
+		           const char *val, int overwrite);
+.fi
+.LP
+See \fBspank.h\fR for more information, and \fBEXAMPLES\fR below for an example
+for \fBspank_getenv\fR usage.
+.SH "CONFIGURATION"
+.LP
+The default \fBSPANK\fR plug-in stack configuration file is
+\fBplugstack.conf\fR in the same directory as \fBslurm.conf\fR(5),
+though this may be changed via the SLURM config parameter
+\fIPlugStackConfig\fR. The config file lists \fBSPANK\fR plugins,
+one per line, along with whether the plugin is \fIrequired\fR or
+\fIoptional\fR, and any global arguments that are to be passed to
+the plugin for runtime configuration.  Comments are preceded with '#' 
+and extend to the end of the line.  If the configuration file
+is missing or empty, it will simply be ignored.
+.LP
+The format of each non-comment line in the configuration file is:
+\fB
+.nf
+
+  required/optional   plugin   arguments
+
+.fi
+\fR For example:
+.nf
+
+  optional /usr/lib/slurm/test.so
+
+.fi
+Tells \fBslurmd\fR to load the plugin \fBtest.so\fR passing no arguments.
+If a \fBSPANK\fR plugin is \fIrequired\fR, then failure of any of the
+plugin's functions will cause \fBslurmd\fR to terminate the job, while
+\fIoptional\fR plugins only cause a warning.
+.LP
+\fBSPANK\fR plugins are stackable, meaning that more than one plugin may
+be placed into the config file. The plugins will simply be called
+in order, one after the other, and appropriate action taken on
+failure given that state of the plugin's \fIoptional\fR flag.
+.LP
+The \fBSPANK\fR config file is re-read on each job launch, so editing
+the config file will not affect running jobs. However care should
+be taken so that a partially edited config file is not read by a
+launching job.  
+.SH "EXAMPLES"
+.LP
+Simple \fBSPANK\fR config file:
+.nf
+
+#
+# SPANK config file
+#
+# required?       plugin                     args
+#
+optional          /usr/lib/slurm/renice.so   verbose
+required          /usr/lib/slurm/test.so     
+
+.fi
+.LP
+Simple \fBSPANK\fR plugin to modify job nice value. 
+Allows priority to be set via SLURM_RENICE environment variable, 
+though system-wide default could be set using \fBSPANK\fR plugin arguments
+in the configuration file.
+.nf
+
+#include <sys/types.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <sys/resource.h>
+
+#include <slurm/spank.h>
+
+SPANK_PLUGIN(renice, 1);
+
+int slurm_spank_task_post_fork (spank_t sp, int ac, char **av)
+{
+        pid_t pid;
+        int taskid;
+        char buf [1024];
+        long int prio;
+        const char *var = "SLURM_RENICE";
+        char *p;
+
+        spank_get_item (sp, S_TASK_GLOBAL_ID, &taskid);
+        spank_get_item (sp, S_TASK_PID, &pid);
+
+        if (spank_getenv (sp, var, buf, 1024) != ESPANK_SUCCESS) {
+                return (0);
+        }
+
+        prio = strtol (buf, &p, 10);
+        if ((*p != '\0') || (prio < -20) || (prio > 20)) {
+                slurm_error ("Bad value for %s: \"%s\"\n", var, buf);
+                return (-1);
+        }
+
+        slurm_info ("re-nicing task%d pid %ld to %ld\n", 
+                    taskid, pid, prio);
+
+        if (setpriority (PRIO_PROCESS, (int) pid, (int) prio) < 0) {
+                slurm_error ("setpriority: %m");
+                return (-1);
+        }
+
+        return (0);
+}
+
+.fi
+
+.SH "COPYING"
+Copyright (C) 2002-2006 The Regents of the University of California.
+Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
+UCRL-CODE-217948.
+.LP
+This file is part of SLURM, a resource management program.
+For details, see <http://www.llnl.gov/linux/slurm/>.
+.LP
+SLURM is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 2 of the License, or (at your option)
+any later version.
+.LP
+SLURM is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
+details.
+.SH "FILES"
+\fB/etc/slurm/slurm.conf\fR - SLURM configuration file.
+.br
+\fB/etc/slurm/plugstack.conf\fR - SPANK configuration file.
+.br
+\fB/usr/include/slurm/spank.h\fR - SPANK header file.
+.SH "SEE ALSO"
+.LP
+\fBslurm.conf\fR(5)