NEWS

This file describes changes in recent versions of SLURM. It primarily
documents those changes that are of interest to users and admins.

* Changes in SLURM 1.3.0-pre5
=============================
 -- Add select_g_reconfigure() function to node changes in slurmctld configuration
    that can impact node scheduling.
 -- scontrol to set/get partition's MaxTime and job's Timelimit in minutes plus
    new formats: min:sec, hr:min:sec, days-hr:min:sec, days-hr, etc.
 -- scontrol "notify" command added to send message to stdout of srun for 
    specified job id.
 -- For BlueGene, make alpha part of node location specification be case insensitive.
 -- Report scheduler-plugin specific configuration information with the 
    "scontrol show configuration" command on the SCHEDULER_CONF line. This
    information is not found in the "slurm.conf" file, but a scheduler plugin 
    specific configuration (e.g. "wiki.conf").
 -- sview partition information reported now includes partition priority.

* Changes in SLURM 1.3.0-pre4
=============================
 -- Job step launch in srun is now done from the slurm api's all further
    modifications to job launch should be done there.
 -- Add new partition configuration parameter Priority. Add job count to 
    Shared parameter.
 -- Add new configuration parameters DefMemPerTask, MaxMemPerTask, and 
    SchedulerTimeSlice.
 -- In sched/wiki2, return REJMESSAGE with details on why a job was 
    requeued (e.g. what node failed).

* Changes in SLURM 1.3.0-pre3
=============================
 -- Remove slaunch command
 -- Added srun option "--checkpoint=time" for job step to automatically be 
    checkpointed on a period basis.
 -- Change behavior of "scancel -s KILL <jobid>" to send SIGKILL to all job
    steps rather than cancelling the job. This now matches the behavior of
    all other signals. "scancel <jobid>" still cancels the job and all steps.
 -- Add support for new job step options --exclusive and --immediate. Permit
    job steps to be queued when resources are not available within an existing 
    job allocation to dedicate the resources to the job step. Useful for
    executing simultaneous job steps. Provides resource management both at 
    the level of jobs and job steps.
 -- Add support for feature count in job constraints, for example
    srun --nodes=16 --constraint=graphics*4 ...
    Based upon work by Kumar Krishna (HP, India).
 -- Add multi-core options to salloc and sbatch commands (sbatch.patch and
    cleanup.patch from Chris Holmes, HP).
 -- In select/cons_res properly release resources allocated to job being 
    suspended (rmbreak.patch, from Chris Holmes, HP).
 -- Removed database and jobacct plugin replaced with jobacct_storage 
    and jobacct_gather for easier hooks for further expansion of the
    jobacct plugin.

* Changes in SLURM 1.3.0-pre2
=============================
 -- Added new srun option --pty to start job with pseudo terminal attached 
    to task 0 (all other tasks have I/O discarded)
 -- Disable user specifying jobid when sched/wiki2 configured (needed for 
    Moab releases until early 2007).
 -- Report command, args and working directory for batch jobs with 
    "scontrol show job".

* Changes in SLURM 1.3.0-pre1
=============================
 -- !!! SRUN CHANGES !!!
    The srun options -A/--allocate, -b/--batch, and -a/--attach have been
    removed!  That functionality is now available in the separate commands
    salloc, sbatch, and sattach, respectively.
 -- Add new node state FAILING plus trigger for when node enters that state.
 -- Add new configuration paramter "PrivateData". This can be used to 
    prevent a user from seeing jobs or job steps belonging to other users.
 -- Added configuration parameters for node power save mode: ResumeProgram
    ResumeRate, SuspendExcNodes, SuspendExcParts, SuspendProgram and 
    SuspendRate.
 -- Slurmctld maintains the IP address (rather than hostname) for srun 
    communications. This fixes some possible network routing issues.
 -- Added global database plugin.  Job accounting and Job completion are the 
    first to use it.  Follow documentation to add more to the plugin.
 -- Removed no-longer-needed jobacct/common/common_slurmctld.c since that is
    replaced by the database plugin.
 -- Added new configuration parameter: CryptoType.
    Moved existing digital signature logic into new plugin: crypto/openssl.
    Added new support for crypto/munge (available with GPL license).

* Changes in SLURM 1.2.20
=========================
 -- In switch/federation, fix small memory leak effecting slurmd.
 -- Add PMI_FANOUT_OFF_HOST environment variable to control how message 
    forwarding is done for PMI (MPICH2). See "man srun" for details.
 -- From sbatch set SLURM_NTASKS_PER_NODE when --ntasks-per-node option is 
    specified.
 -- BLUEGENE: Documented the prefix should always be lower case and the 3
    digit suffix should be uppercase if any letters are used as digits. 
 -- In sched/wiki and sched/wiki2, add support for --cpus-per-task option.
    From Miguel Ros, BSC.
 -- In sched/wiki2, prevent invalid memory pointer (and likely seg fault) 
    for job associated with a partition that has since been deleted.
 -- In sched/wiki2 plus select/cons_res, prevent invalid memory pointer 
    (and likely seg fault) when a job is requeued.
 -- In sched/wiki, add support for job suspend, resume, and modify.
 -- In sched/wiki, add suppport for processor allocation (not just node allocation)
    with layout control.

* Changes in SLURM 1.2.19
=========================
*** NOTE IMPORTANT CHANGE IN RPM BUILD BELOW ****
 -- slurm.spec file (used to build RPMs) was updated in order to support Mock, a
    chroot build environment. See https://hosted.fedoraproject.org/projects/mock/
    for more information. The following RPMs are no longer build by default:
    aix-federation, auth_none, authd, bluegene, sgijob, and switch-elan. Change 
    the RPMs built using the following options in ~/rpmmacros: "%_with_authd 1", 
    "%_without_munge 1", etc. See the slurm.spec file for more details.
 -- Print warning if non-privileged user requests negative "--nice" value on
    job submission (srun, salloc, and sbatch commands).
 -- In sched/wiki and sched/wiki2, add support for srun's --ntasks-per-node 
    option.
 -- In select/bluegene with Groups defined for Images, fix possible memory 
    corruption. Other configurations are not affected. 
 -- BLUEGENE - Fix bug that prevented user specification of linux-image, 
    mloader-image, and ramdisk-image on job submission.
 -- BLUEGENE - filter Groups specified for image not just by submitting 
    user's current group, but all groups the user has access to.
 -- BLUEGENE - Add salloc options to specify images to be loaded (--blrts-image, 
    --linux-image, --mloader-image, and --ramdisk-image).
 -- BLUEGENE - In bluegene.conf, permit Groups to be comma separated in addition 
    to colon separators previously supported.
 -- sbatch will accept batch script containing "#SLURM" options and advise
    changed to "#SBATCH".
 -- If srun --output or --error specification contains a task number rather 
    than a file name, send stdout/err from specified task to srun's stdout/err
    rather than to a file by the same name as the task's number.
 -- For srun --multi-prog option, verify configuration file before attempting 
    to launch tasks, report clear explanation of any configuration file errors.
 -- For sched/wiki2, add optional timeout option to srun's --get-user-env
    parameter, change default timeout for "su - <user> env" from 3 to 8 seconds.
    On timeout, attempt to load env from file at StateSaveLocation/env_cache/<user>.
    The format of this file is the same as output of "env" command. If there
    is no env cache file, then abort the request.
 -- squeue modified for completing job to remove nodes that have already 
    completed the job before applying node filter logic.
 -- squeue formatted output option added for job comment, "%q" (the obvious 
    choices for letters are already in use).
 -- Added configure option --enable-load-env-no-login for use with Moab. If
    set then the user job runs with the environment built without a login
    ("su <user> env" rather than "su - <user> env").
 -- Fix output of "srun -o %C" (allocated CPU count) for running jobs. This was
    broken in 1.2.18 for handling requeue of Moab jobs.
 -- Added logic to mpiexec wrapper to read in the MPIEXEC_TIMEOUT var
 -- Updated qstat wrapper to display information for partitions (-Q) option
 -- NOTE: SLURM should now work directly with Globus using the PBS GRAM.

* Changes in SLURM 1.2.18
=========================
 -- BLUEGENE - bug fix for smap stating passthroughs are used when they aren't
 -- Fixed bug in sview to be able to edit partitions correctly
 -- Fixed bug so in slurm.conf files where SlurmdPort isn't defined things
    work correctly.
 -- In sched/wiki2 and sched/wiki add support for batch job being requeued
    in Slurm either when nodes fail or upon request.
 -- In sched/wiki2 and sched/wiki with FastSchedule=2 configured and nodes 
    configured with more CPUs than actually exist, return a value of TASKS 
    equal to the number of configured CPUs that are allocated to a job rather 
    than the number of physical CPUs allocated.
 -- For sched/wiki2, timeout "srun --get-user-env ..." command after 3 seconds 
    if unable to perform pseudo-login and get user environment variables.
 -- Add contribs/time_login.c program to test how long pseudo-login takes
    for specific users or all users. This can identify users for which Moab 
    job submissions are unable to set the proper environment variables.
 -- Fix problem in parallel make of Slurm.
 -- Fixed bug in consumable resources when CR_Core_Memory is enabled
 -- Add delay in slurmctld for "scontrol shutdown" RPC to get propagated 
    to slurmd daemons.

* Changes in SLURM 1.2.17
=========================
 -- In select/cons_res properly release resources allocated to job being 
    suspended (rmbreak.patch, from Chris Holmes, HP).
 -- Fix AIX linking problem for PMI (mpich2) support.
 -- Improve PMI logic for greater scalability (up to 16k tasks run).
 -- Add srun support for SLURM_THREADS and PMI_FANOUT environment variables.
 -- Fix support in squeue for output format with left justification of 
    reason (%r) and reason/node_list (%R) output.
 -- Automatically requeue a batch job when a node allocated to it fails
    or the prolog fails (unless --no-requeue or --no-kill option used).
 -- In sched/wiki, enable use of wiki.conf parameter ExcludePartitions to
    directly schedule selected partitions without Maui control.
 -- In sched/backfill, if a job requires specific nodes, schedule other jobs
    ahead of it rather than completely stopping backfill scheduling for that
    partition.
 -- BLUEGENE - corrected logic making block allocation work in a circular 
    fashion instead of linear.
 
* Changes in SLURM 1.2.16
=========================
 -- Add --overcommit option to the salloc command.
 -- Run task epilog from job's working directory rather than directory
    where slurmd daemon started from.
 -- Log errors running task prolog or task epilog to srun's output.
 -- In sched/wiki2, fix bug processing condensed hostlist expressions.
 -- Release contribs/mpich1.slurm.patch without GPL license.