job_submit_plugins.shtml

<!--#include virtual="header.txt"-->

<h1><a name="top">Slurm Job Submit Plugin API</a></h1>

<h2> Overview</h2>
<p> This document describes Slurm job submit plugins and the API that
defines them. It is intended as a resource to programmers wishing to write
their own Slurm job submit plugins. This is version 100 of the API.</p>

<p>Slurm job submit plugins must conform to the
Slurm Plugin API with the following specifications:</p>

<p><span class="commandline">const char
plugin_name[]="<i>full&nbsp;text&nbsp;name</i>"</span>
<p style="margin-left:.2in">
A free-formatted ASCII text string that identifies the plugin.

<p><span class="commandline">const char
plugin_type[]="<i>major/minor</i>"</span><br>
<p style="margin-left:.2in">
The major type must be &quot;job_submit.&quot;
The minor type can be any suitable name for the type of job submission package.
We include samples in the Slurm distribution for
<ul>
<li><b>all_partitions</b>&#151;Set default partition to all partitions on
the cluster.</li>
<li><b>defaults</b>&#151;Set default values for job submission or modify
requests.</li>
<li><b>logging</b>&#151;Log select job submission and modification
parameters.</li>
<li><b>lua</b>&#151;Interface to <a href="http://www.lua.org">Lua</a> scripts
implementing these functions (actually a slight variation of them). Sample Lua
scripts can be found with the Slurm distribution in the directory
<i>contribs/lua</i>. The default installation location of the Lua scripts is
the same location as the Slurm configuration file, <i>slurm.conf</i>.</li>
<li><b>partition</b>&#151;Sets a job's default partition based upon job
submission parameters and available partitions.</li>
</ul>
<p>Slurm can be configured to use multiple job_submit plugins if desired,
however the lua plugin will only execute one lua script named "job_submit.lua"
and located in default script directory (typically the subdirectory "etc" of
the installation directory).</p>

<p class="footer"><a href="#top">top</a>

<h2>API Functions</h2>
<p>All of the following functions are required. Functions which are not
implemented must be stubbed.

<p class="commandline"> int init (void)
<p style="margin-left:.2in"><b>Description</b>:<br>
  Called when the plugin is loaded, before any other functions are
  called. Put global initialization here.
<p style="margin-left:.2in"><b>Returns</b>: <br>
  <span class="commandline">SLURM_SUCCESS</span> on success, or<br>
  <span class="commandline">SLURM_ERROR</span> on failure.</p>

<p class="commandline"> void fini (void)
<p style="margin-left:.2in"><b>Description</b>:<br>
  Called when the plugin is removed. Clear any allocated storage here.
<p style="margin-left:.2in"><b>Returns</b>: None.</p>

<p><b>Note</b>: These init and fini functions are not the same as those
described in the <span class="commandline">dlopen (3)</span> system library.
The C run-time system co-opts those symbols for its own initialization.
The system <span class="commandline">_init()</span> is called before the Slurm
<span class="commandline">init()</span>, and the Slurm
<span class="commandline">fini()</span> is called before the system's
<span class="commandline">_fini()</span>.</p>

<p class="commandline">
int job_submit(struct job_descriptor *job_desc, uint32_t submit_uid, char **error_msg)
<p style="margin-left:.2in"><b>Description</b>:<br>
This function is called by the slurmctld daemon with job submission parameters
supplied by the salloc, sbatch or srun command. It can be used to log and/or
modify the job parameters supplied by the user as desired. Note that this
function has access to the slurmctld's global data structures, for example
to examine the available partitions, reservations, etc.
<p style="margin-left:.2in"><b>Arguments</b>: <br>
<span class="commandline">job_desc</span>
(input/output) the job allocation request specifications.<br>
<span class="commandline">submit_uid</span>
(input) user ID initiating the request.<br>
<span class="commandline">error_msg</span>
(output) If the argument is not null, then a plugin generated error message
can be stored here. The error message is expected to have allocated memory
which Slurm will release using the xfree function. The error message is
propagated to the caller only when the return code of this function is
not SLURM_SUCESS.<br>
<p style="margin-left:.2in"><b>Returns</b>: <br>
<span class="commandline">SLURM_SUCCESS</span> on success, or<br>
<span class="commandline">SLURM_ERROR</span> on failure.

<p class="commandline">
int job_modify(struct job_descriptor *job_desc, struct job_record *job_ptr,
uint32_t modify_uid)
<p style="margin-left:.2in"><b>Description</b>:<br>
This function is called by the slurmctld daemon with job modification parameters
supplied by the scontrol or sview command. It can be used to log and/or
modify the job parameters supplied by the user as desired. Note that this
function has access to the slurmctld's global data structures, for example to
examine the available partitions, reservations, etc.
<p style="margin-left:.2in"><b>Arguments</b>: <br>
<span class="commandline">job_desc</span>
(input/output) the job allocation request specifications.<br>
<span class="commandline">job_ptr</span>
(input/output) slurmctld daemon's current data structure for the job to
be modified.<br>
<span class="commandline">modify_uid</span>
(input) user ID initiating the request.<br>
<p style="margin-left:.2in"><b>Returns</b>: <br>
<span class="commandline">SLURM_SUCCESS</span> on success, or<br>
<span class="commandline">SLURM_ERROR</span> on failure.

<h2>Lua Functions</h2>
<p>The Lua functions differ slightly from those implemented in C for
better ease of use. Sample Lua scripts can be found with the Slurm distribution
in the directory <i>contribs/lua</i>. The default installation location of
the Lua scripts is the same location as the Slurm configuration file,
<i>slurm.conf</i>.</p>

<p class="commandline">
int job_submit(struct job_descriptor *job_desc, List part_list, uint32_t submit_uid)
<p style="margin-left:.2in"><b>Description</b>:<br>
This function is called by the slurmctld daemon with job submission parameters
supplied by the salloc, sbatch or srun command. It can be used to log and/or
modify the job parameters supplied by the user as desired. Note that this
function has access to the slurmctld's global data structures, for example
to examine the available partitions, reservations, etc.
<p style="margin-left:.2in"><b>Arguments</b>: <br>
<span class="commandline">job_desc</span>
(input/output) the job allocation request specifications.<br>
<span class="commandline">part_list</span>
(input) List of pointer to partitions which this user is authorized to use.<br>
<span class="commandline">submit_uid</span>
(input) user ID initiating the request.<br>
<p style="margin-left:.2in"><b>Returns</b>: <br>
<span class="commandline">0</span> on success, or an
errno on failure. Slurm specific error numbers from <i>slurm/slurm_errno.h</i>
may be used. On failure, the request will be rejected and the user will have an
appropriate error message printed for that errno. 

<p class="commandline">
int job_modify(struct job_descriptor *job_desc, struct job_record *job_ptr,
List part_list, int modify_uid)
<p style="margin-left:.2in"><b>Description</b>:<br>
This function is called by the slurmctld daemon with job modification parameters
supplied by the scontrol or sview command. It can be used to log and/or
modify the job parameters supplied by the user as desired. Note that this
function has access to the slurmctld's global data structures, for example to
examine the available partitions, reservations, etc.
<p style="margin-left:.2in"><b>Arguments</b>: <br>
<span class="commandline">job_desc</span>
(input/output) the job allocation request specifications.<br>
<span class="commandline">job_ptr</span>
(input/output) slurmctld daemon's current data structure for the job to
be modified.<br>
<span class="commandline">part_list</span>
(input) List of pointer to partitions which this user is authorized to use.<br>
<span class="commandline">modify_uid</span>
(input) user ID initiating the request.<br>
<span class="commandline">0</span> on success, or an
errno on failure. Slurm specific error numbers from <i>slurm/slurm_errno.h</i>
may be used. On failure, the request will be rejected and the user will have an
appropriate error message printed for that errno. 

<h2>Versioning</h2>
<p> This document describes version 110 of the Slurm Job Submission API. Future
releases of Slurm may revise this API.
<p class="footer"><a href="#top">top</a>

<h2>Building</h2>
<p>Generally using a LUA interface for a job submit plugin is best:
It is simple to write and maintain with minimal dependencies upon the Slurm
data structures.
However using C does provide a mechanism to get more information than available
using LUA including full access to all of the data structures and functions
in the slurmctld daemon.
The simplest way to build a C program would be to just replace one of the
job submit plugins included in the Slurm distribution with your own code
(i.e. use a patch with your own code).
Then just build and install Slurm with that new code.
Building a new plugin outside of the Slurm distribution is possible, but
far more complex.
It also requires access to a multitude of Slurm header files as shown in the
proceedure below.</p>

<ol>
<li>You will need to at least partly build Slurm first. The "configure" command
must be executed in order to build the "config.h" file in the build directory.</li>

<li>Create a local directory somewhere for your files to build with.
Also create subdirectories named ".libs" and ".deps".</li>

<li>Copy a ".deps/job_submit_*Plo" file from another job_submit plugin's ".deps"
directory (made as part of the build process) into your local ".deps" subdirectory.
Rename the file as appropriate to reflect your plugins name (e.g. rename
"job_submit_partition.Plo" to be something like "job_submit_mine.Plo").</li>

<li>Compile and link your plugin. Those options might differ depending
upon your build environment. Check the options used for building the
other job_submit plugins and modify the example below as required.</li>

<li>Install the plugin.</li>
</ol>

<pre>
# Example:
# The Slurm source is in ~/SLURM/slurm.git
# The Slurm build directory is ~/SLURM/slurm.build
# The plugin build is to take place in the directory
#   "~/SLURM/my_submit"
# The installation locaiton is "/usr/local"

# Build Slurm from ~/SLURM/slurm.build
# (or at least run "~/SLURM/slurm.git/configure")

# Set up your plugin files
cd ~/SLURM
mkdir my_submit
cd my_submit
mkdir .libs
mkdir .deps
# Create your plugin code
vi job_submit_mine.c

# Copy up a dependency file
cp ~/SLURM/slurm.build/src/plugins/job_submit/partition/.deps/job_submit_partition.Plo \
   .deps/job_submit_nine.Plo

# Compile
gcc -DHAVE_CONFIG_H -I~/SLURM/slurm.build -I~/slurm.git \
   -g -O2 -pthread -fno-gcse -Werror -Wall -g -O0       \
   -fno-strict-aliasing -MT job_submit_mine.lo          \
   -MD -MP -MF .deps/job_submit_mine.Tpo                \
   -c job_submit_mine.c -o job_submit_mine.o

# Some clean up
mv -f .deps/job_submit_mine.Tpo .deps/job_submit_mine.Plo
rm -fr .libs/job_submit_mine.a .libs/job_submit_mine.la \
   .libs/job_submit_mine.lai job_submit_mine.so

# Link
gcc -shared -fPIC -DPIC .libs/job_submit_mine.o -O2         \
   -pthread -O0 -pthread -Wl,-soname -Wl,job_submit_mine.so \
   -o job_submit_mine.so

# Install
cp job_submit_mine.so file \
   /usr/local/lib/slurm/job_submit_mine.so
</pre>

<p style="text-align:center;">Last modified 25 February 2015</p>

<!--#include virtual="footer.txt"-->