NEWS

This file describes changes in recent versions of Slurm. It primarily
documents those changes that are of interest to users and administrators.

* Changes in Slurm 17.02.8
==========================

* Changes in Slurm 17.02.7
==========================
 -- Fix deadlock if requesting to create more than 10000 reservations.
 -- Fix potential memory leak when creating partition name.
 -- Execute the HealthCheckProgram once when the slurmd daemon starts rather
    than executing repeatedly until an exit code of 0 is returned.
 -- Set job/step start and end times to 0 when using --truncate and start > end.
 -- Make srun --pty option ignore EINTR allowing windows to resize.
 -- When resuming node only send one message to the slurmdbd.
 -- Modify srun --pty option to use configured SrunPortRange range.
 -- Fix issue with whole gres not being printed out with Slurm tools.
 -- Fix issue with multiple jobs from an array are prevented from starting.
 -- Fix for possible slurmctld abort with use of salloc/sbatch/srun
    --gres-flags=enforce-binding option.
 -- Fix race condition when using jobacct_gather/cgroup where the memory of the
    step wasn't always gathered correctly.
 -- Better debug when slurmdbd queue is filling up in the slurmctld.
 -- Fixed truncation on scontrol show config output.
 -- Serialize updates from from the dbd to the slurmctld.
 -- Fix memory leak in slurmctld when agent queue to the DBD has filled up.
 -- CRAY - Throttle step creation if trying to create too many steps at once.
 -- If failing after switch_g_job_init happened make sure switch_g_job_fini is
    called.
 -- Fix minor memory leak if launch fails in the slurmstepd.
 -- Fix issue where UnkillableStepProgram if step was in an ending state.
 -- Fix bug when tracking multiple simultaneous spawned ping cycles.
 -- jobcomp/elasticsearch plugin now saves state of pending requests on
    slurmctld daemon shutdown so then can be recovered on restart.
 -- Fix issue when an alternate munge key when communicating on a persistent
    connection.
 -- Document inconsistent behavior of GroupUpdateForce option.
 -- Fix bug in selection of GRES bound to specific CPUs where the GRES count
    is 2 or more. Previous logic could allocate CPUs not available to the job.
 -- Increase buffer to handle long /proc/<pid>/stat output so that Slurm can
    read correct RSS value and take action on jobs using more memory than
    requested.
 -- Fix srun job jobs that can run immediately to run in the highest priority
    partion when multiple partitions are listed. scontrol show jobs can
    potentially show the partition list in priority order.
 -- Fix starting controller if StateSaveLocation path didn't exist.
 -- Fix inherited association 'max' TRES limits combining multiple limits in
    the tree.
 -- Sort TRES id's on limits when getting them from the database.
 -- Fix issue with pmi[2|x] when TreeWidth=1.
 -- Correct buffer size used in determining specialized cores to avoid possible
    truncation of core specification and not reserving the specified cores.
 -- Close race condition on Slurm structures when setting DebugFlags.
 -- Make it so the cray/switch plugin grabs new DebugFlags on a reconfigure.
 -- Fix incorrect lock levels when creating or updating a reservation.
 -- Fix overlapping reservation resize.
 -- Add logic to help support Dell KNL systems where syscfg is different than
    the normal Intel syscfg.
 -- CRAY - Fix BB to handle type= correctly, regression in 17.02.6.

* Changes in Slurm 17.02.6
==========================
 -- Fix configurator.easy.html to output the SelectTypeParameters line.
 -- If a job requests a specific memory requirement then gets something else
    from the slurmctld make sure the step allocation is made aware of it.
 -- Fix missing initialization in slurmd.
 -- Fix potential degradation when running HTC (> 100 jobs a sec) like
    workflows through the slurmd.
 -- Fix race condition which could leave a stepd hung on shutdown.
 -- CRAY - Add configuration for ATP to the ansible play script.
 -- Fix potential to corrupt DBD message.
 -- burst_buffer logic modified to support sizes in both SI and EIC size units
    (e.g. M/MiB for powers of 1024, MB for powers of 1000).

* Changes in Slurm 17.02.5
==========================
 -- Prevent segfault if a job was blocked from running by a QOS that is then
    deleted.
 -- Improve selection of jobs to preempt when there are multiple partitions
    with jobs subject to preemption.
 -- Only set kmem limit when ConstrainKmemSpace=yes is set in cgroup.conf.
 -- Fix bug in task/affinity that could result in slurmd fatal error.
 -- Increase number of jobs that are tracked in the slurmd as finishing at one
    time.
 -- Note when a job finishes in the slurmd to avoid a race when launching a
    batch job takes longer than it takes to finish.
 -- Improve slurmd startup on large systems (> 10000 nodes)
 -- Add LaunchParameters option of cray_net_exclusive to control whether all
    jobs on the cluster have exclusive access to their assigned nodes.
 -- Make sure srun inside an allocation gets --ntasks-per-[core|socket]
    set correctly.
 -- Only make the extern step at job creation.
 -- Fix for job step task layout with --cpus-per-task option.
 -- Fix --ntasks-per-core option/environment variable parsing to set
    the requested value, instead of always setting one (srun).
 -- Correct error message when ClusterName in configuration files does not match
    the name in the slurmctld daemon's state save file.
 -- Better checking when a job is finishing to avoid underflow on job's
    submitted to a QOS/association.
 -- Handle partition QOS submit limits correctly when a job is submitted to
    more than 1 partition or when the partition is changed with scontrol.
 -- Performance boost for when Slurm is dealing with credentials.
 -- Fix race condition which could leave a stepd hung on shutdown.
 -- Add lua support for opensuse.

* Changes in Slurm 17.02.4
==========================
 -- Do not attempt to schedule jobs after changing the power cap if there are
    already many active threads.
 -- Job expansion example in FAQ enhanced to demonstrate operation in
    heterogeneous environments.
 -- Prevent scontrol crash when operating on array and no-array jobs at once.
 -- knl_cray plugin: Log incomplete capmc output for a node.
 -- knl_cray plugin: Change capmc parsing of mcdram_pct from string to number.
 -- Remove log files from test20.12.
 -- When rebooting a node and using the PrologFlags=alloc make sure the
    prolog is ran after the reboot.
 -- node_features/knl_generic - If a node is rebooted for a pending job, but
    fails to enter the desired NUMA and/or MCDRAM mode then drain the node and
    requeue the job.
 -- node_features/knl_generic disable mode change unless RebootProgram
    configured.
 -- Add new burst_buffer function bb_g_job_revoke_alloc() to be executed
    if there was a failure after the initial resource allocation. Does not
    release previously allocated resources.
 -- Test if the node_bitmap on a job is NULL when testing if the job's nodes
    are ready.  This will be NULL is a job was revoked while beginning.
 -- Fix incorrect lock levels when testing when job will run or updating a job.
 -- Add missing locks to job_submit/pbs plugin when updating a jobs
    dependencies.
 -- Add support for lua5.3
 -- Add min_memory_per_node|cpu to the job_submit/lua plugin to deal with lua
    not being able to deal with pn_min_memory being a uint64_t.  Scripts are
    urged to change to these new variables avoid issue.  If not set the
    variables will be 'nil'.
 -- Calculate priority correctly when 'nice' is given.
 -- Fix minor typos in the documentation.
 -- node_features/knl_cray: Preserve non-KNL active features if slurmctld
    reconfigured while node boot in progress.
 -- node_features/knl_generic: Do not repeatedly log errors when trying to read
    KNL modes if not KNL system.
 -- Add missing QOS read lock to backfill scheduler.
 -- When doing a dlopen on liblua only attempt the version compiled against.
 -- Fix null-dereference in sreport cluster ulitization when configured with
    memory-leak-debug.
 -- Fix Partition info in 'scontrol show node'. Previously duplicate partition
    names, or Partitions the node did not belong to could be displayed.
 -- Fix it so the backup slurmdbd will take control correctly.
 -- Fix unsafe use of MAX() macro, which could result in problems cleaning up
    accounting plugins in slurmd, or repeat job cancellation attempts in
    scancel.
 -- Fix 'scontrol update reservation duration=unlimited' to set the duration
    to 365-days (as is done elsewhere), rather than 49710 days.
 -- Check if variable given to scontrol show job is a valid jobid.
 -- Fix WithSubAccounts option to not include WithDeleted unless requested.
 -- Prevent a job tested on multiple partitions from being marked
    WHOLE_NODE_USER.
 -- Prevent a race between completing jobs on a user-exclusive node from
    leaving the node owned.
 -- When scheduling take the nodes in completing jobs out of the mix to reduce
    fragmentation.  SchedulerParameters=reduce_completing_frag
 -- For jobs submited to multiple partitions, report the job's earliest start
    time for any partition.
 -- Backfill partitions that use QOS Grp limits to "float" better.
 -- node_features/knl_cray: don't clear configured GRES from non-KNL node.
 -- sacctmgr - prevent segfault in command when a request is denied due
    to a insufficient priviledges.
 -- Add warning about libcurl-devel not being installed during configure.
 -- Streamline job purge by handling file deletion on a separate thread.
 -- Always set RLIMIT_CORE to the maximum permitted for slurmd, to ensure
    core files are created even on non-developer builds.
 -- Fix --ntasks-per-core option/environment variable parsing to set
    the requested value, instead of always setting one.
 -- If trying to cancel a step that hasn't started yet for some reason return
    a good return code.
 -- Fix issue with sacctmgr show where user=''

* Changes in Slurm 17.02.3
==========================
 -- Increase --cpu_bind and --mem_bind field length limits.
 -- Fix segfault when using AdminComment field with job arrays.
 -- Clear Dependency field when all dependencies are satisfied.
 -- Add --array-unique to squeue which will display one unique pending job
    array element per line.
 -- Reset backfill timers correctly without skipping over them in certain
    circumstances.
 -- When running the "scontrol top" command, make sure that all of the user's
    jobs have a priority that is lower than the selected job. Previous logic
    would permit other jobs with equal priority (no jobs with higher priority).
 -- Fix perl api so we always get an allocation when calling Slurm::new().
 -- Fix issue with cleaning up cpuset and devices cgroups when multiple steps
    end at the same time.
 -- Document that PriorityFlags option of DEPTH_OBLIVIOUS precludes the use of
    FAIR_TREE.
 -- Fix issue if an invalid message came in a Slurm daemon/command may abort.
 -- Make it impossible to use CR_CPU* along with CR_ONE_TASK_PER_CORE. The
    options are mutually exclusive.
 -- ALPS - Fix scheduling when ALPS doesn't agree with Slurm on what nodes
    are free.
 -- When removing a partition make sure it isn't part of a reservation.