NEWS

This file describes changes in recent versions of SLURM. It primarily
documents those changes that are of interest to users and admins.

* Changes in SLURM 2.1.0-pre3
=============================
 -- Removed sched/gang plugin and moved the logic directly into the slurmctld
    daemon so that job preemption and gang scheduling can be used with the
    sched/backfill plugin. Added configuration parameter:
    PreemptMode=off|suspend|cancel|requeue|checkpoint 
    to enable/disable the job preemption logic (disabled by default).
    (NOTE: There are some problems with memory management which could prevent a
    job from starting when memory would be freed by a job being requeued or 
    otherwise removed, these are being worked on)
 -- If the --partition option is used with the sinfo or squeue command then
    print information about even hidden partitions.
 -- Replaced misc cpu allocation members in job_info_t with select_job_res_t
    which will only be populated when requested (show_flags & SHOW_DETAIL)
 -- Added a --detail option to "scontrol show job" to display the cpu/mem
    allocation info on a node-by-node basis.
 -- Added logic to give correct request uid for individual steps that 
    were cancelled.
 -- Created a spank_get_item() option (S_JOB_ALLOC_MEM) that conveys the memory
    that the select/cons_res plugin has allocated to a job.
 -- BLUEGENE - blocks in error state are now handled correctly in accounting.
 -- Modify squeue to print job step information about a specific job ID using
    the following syntax: "squeue -j <job_id> -s".
 -- BLUEGENE - scontrol delete block and update block can now remove blocks 
    on dynamic laid out systems.
 -- BLUEGENE - Vastly improve Dynamic layout mode algorithm.

* Changes in SLURM 2.1.0-pre2
=============================
 -- Added support for smap to query off node name for display.
 -- Slurmdbd modified to set user ID and group ID to SlurmUser if started as 
    user root.
 -- Configuration parameter ControlMachine changed to  accept multiple comma-
    separated hostnames for support of some high-availability architectures.
 -- ALTERED API CALL slurm_get_job_steps 0 has been changed to NO_VAL for both
    job and step id to recieve all jobs/steps.  Please make adjustments to
    your code.
 -- salloc's --wait=<secs> option deprecated by --immediate=<secs> option to 
    match the srun command.
 -- Add new slurmctld list for node features with node bitmaps for simplified
    scheduling logic.
 -- Multiple features can be specified when creating a reservation. Use "&" 
    (AND) or "|" (OR) separators between the feature names.
 -- Changed internal node name caching so that front-end mode would work with
    multiple lines of node name definitions. 
 -- Add node state flag for power-up/configuring. Represented by "#" suffix
    on the node state name (e.g. "ALLOCATED#") for command output.
 -- Add CONFIGURING/CF job state flag for node power-up/configuring.
 -- Modify job step cancel logic for scancel and srun (on reciept of SIGTERM 
    or three SIGINT) to immediately send SIGKILL to spawned tasks.  Previous 
    logic would send SIGCONT, SIGTERM, wait KillWait seconds, SIGKILL.
 -- Created a spank_get_item() option (S_JOB_ALLOC_CORES) that conveys the cpus
    that the select/cons_res plugin has allocated to a job.
 -- Improve sview performance (outrageously) on very large machines.
 -- Add support for licenses in resource reservation.
 -- BLUEGENE - Jobs waiting for a block to boot will now be in Configuring
    state. 
 -- bit_fmt now does not return brackets surrounding any set of data.

* Changes in SLURM 2.1.0-pre1
=============================
 -- Slurmd notifies slurmctld of node boot time to better clean up after node
    reboots.
 -- Slurmd sends node registration information repeatedly until successful
    transmit.
 -- Change job_state in job structure to dedicate 8-bits to state flags. 
    Added macros to get state information (IS_JOB_RUNNING(job_ptr), etc.)
 -- Added macros to get node state information (IS_NODE_DOWN(node_ptr), etc).
 -- Added support for Solaris. Patch from David Hoppner.
 -- Rename "slurm-aix-federation-<version>.rpm" to just 
    "slurm-aix-<version>.rpm" (federation switch plugin may not be present).
 -- Eliminated the redundant squeue output format and sort options of 
    "%o" and "%b". Use "%D" and "%S" formats respectively. Also eliminated 
    "%X" and "%Y" and "%Z" formats. Use "%z" instead.
 -- Added mechanism for SPANK plugins to set environment variables for
    Prolog, Epilog, PrologSLurmctld and EpilogSlurmctld programs using
    the functions spank_get_job_env, spank_set_job_env, and 
    spank_unset_job_env. See "man spank" for more information.
 -- Completed the work to begun in 2.0.0 to standardize on using '-Q' as the
    --quiet flag for all the commands.
 -- BLUEGENE - sinfo and sview now display correct cpu counts for partitions
 -- Cleaned up the cons_res plugin.  It now uses a ptr to a part_record
    instead of having to do strcmp's to find the correct one.
 -- Pushed most all the plugin specific info in src/common/node_select.c 
    into the respected plugin.
 -- BLUEGENE - closed some corner cases where a block could had been removed 
    while a job was waiting for it to become ready because an underlying 
    part of the block was put into an error state.
 -- Modify sbcast logic to prevent a user from moving files to nodes they
    have not been allocated (this would be possible in previous versions
    only by hacking the sbcast code).
 -- Add contribs/sjstat script (Perl tool to report job state information).
    Put into new RPM: sjstat.
 -- Add sched/wiki2 (Moab) JOBMODIFY command support for VARIABLELIST option
    to set supplemental environment variables for pending batch jobs.
 -- BLUEGENE - add support for scontrol show blocks.
 -- Added support for job step time limits.

* Changes in SLURM 2.0.5
========================
 -- BLUEGENE - Added support for emulating systems with a X-dimension of 4.
 -- BLUEGENE - When a nodecard goes down on a non-Dynamic system SLURM will 
    now only drain blocks under 1 midplane, if no such block exists then SLURM 
    will drain the entire midplane and not mark any block in error state.  
    Previously SLURM would drain every overlapping block of the nodecard 
    making it possible for a large block to make other blocks not work since 
    they overlap some other part of the block that really isn't bad.
 -- BLUEGENE - Handle L3 errors on boot better.
 -- Don't revoke a pending batch launch request from the slurmctld if the
    job is immediately suspended (a normal event with gang scheduling).
 -- BLUEGENE - Fixed issue with restart of slurmctld would allow error block 
    nodes to be considered for building new blocks when testing if a job would 
    run.  This is a visual bug only, jobs would never run on new block, but 
    the block would appear in slurm tools.
 -- Better responsiveness when starting new allocations when running with the 
    slurmdbd.

* Changes in SLURM 2.0.4
========================
 -- Permit node suspend/resume logic to be enabled through "scontrol reconfig"
    given appropriate changes to slurm configuration file.
 -- Check for return codes on functions with warn_unused_result set.
 -- Fix memory leak in getting step information (as used by squeue -s).
 -- Better logging for when job's request bad output file locations.
 -- Fix issue where if user specified non-existant file to write to slurmstepd
    will regain privileges before sending batch script ended to the controller.
 -- Fix bug when using the priority_multifactor plugin with no associations 
    yet.
 -- BLUEGENE - we no longer check for the images to sync state.  This was 
    needed long ago when rebooting blocks wasn't a possibility and should 
    had been removed when that functionality was available.
 -- Added message about no connection with the database for sacctmgr.
 -- On BlueGene, let srun or salloc exit on SIGINT if slurmctld dies while
    booting its block.
 -- In select/cons_res fix bug that could result in invalid memory pointer
    if node configurations in slurm.conf contains 8 or more distinct 
    socket/core/thread counts.
 -- Modify select/cons_res to recognize updated memory size upon node startup
    if FastSchedule=0.
 -- Fixed bug if not enforcing associations, but running with them and the 
    priority/multifactor, the slurmctld will not core dump on processing usage.
 -- QOS will not be reset to the default when added back a previously deleted
    association.
 -- Do not set a job's virtual memory limit based upon the job's specified
    memory limit (which should be a real memory limit, not virtual).
 -- In sched/wiki2 - Strip single and double quotes out of a node's reason 
    string to avoid confusing Moab's parser.
 -- BLUEGENE - fix for sinfo/sview for displaying proper node count for nodes 
    in draining state.
 -- Fix for sview when viewing a certain part of a group (like 1 job) so it 
    doesn't core when the part is gone.
 -- BLUEGENE - Changed order of SYNC's to be on the front of the list to
    avoid having a job terminated with a TERM before the SYNC of the 
    job happens.
 -- Validate configured PluginDir value is a valid directory before trying to
    use it.
 -- Fix to resolve agent_queue_request symbol from some checkpoint plugins.
 -- Fix possible execve error for sbatch script read from stdin.
 -- Modify handling of user ID/name and group ID/name in the slurm.conf file
    to properly handle user names that contain all digits. Return error code 
    from uid_from_string() and gid_from_string() functions rather than a uid of
    -1, which might be a valid uid or gid on some systems.
 -- Fix in re-calcuation of job priorities due to DOWN or DRAINED nodes.

* Changes in SLURM 2.0.3
========================
 -- Add reservation creation/update flag of Ignore_Jobs to enable the creation
    of a reservation that overlaps jobs expected to still be running when
    the reservation starts. This would be especially useful to reserve all 
    nodes for system maintenence without adjusting time limits of running
    jobs before creating the reservation. Without this flag, nodes allocated
    jobs expected to running when the reservation begins can not be placed 
    into a reservation.
 -- In task/affinity plugin, add layer of abstraction to logic translating
    block masks to physical machine masks. Patch from Matthieu Hautreux, CEA.
 -- Fix for setting the node_bitmap in a job to NULL if the job does not 
    start correctly when expected to start.
 -- Fixed bug in srun --pty logic. Output from the task was split up 
    arbitrarily into stdout and stderr streams, and sometimes was printed 
    out of order.
 -- If job requests minimum and maximum node count range with select/cons_res,
    try to satisfy the higher value (formerly only allocated the minimum).
 -- Fix for checking for a non-existant job when querying steps.
 -- For job steps with the --exclusive option, base initial wait time in 
    partly upon the process ID for better performance with many job steps 
    started at the same time. Maintain exponential back-off as needed.
 -- Fix for correct step ordering in sview.
 -- Support optional argument to srun and salloc --immediate option. Specify
    timeout value in seconds for job or step to be allocated resources.

* Changes in SLURM 2.0.2
========================
 -- Fix, don't remove job details when a job is cancelled while pending.
 -- Do correct type for mktime so garbage isn't returned on 64bit systems 
    for accounting archival.
 -- Better checking in sacctmgr to avoid infinite loops.
 -- Fix minor memory leak in fake_slurm_step_layout_create()