NEWS

 This file describes changes in recent versions of Slurm. It primarily
documents those changes that are of interest to users and administrators.

* Changes in Slurm 14.11.6
==========================
 -- If SchedulerParameters value of bf_min_age_reserve is configured, then
    a newly submitted job can start immediately even if there is a higher
    priority non-runnable job which has been waiting for less time than
    bf_min_age_reserve.
 -- qsub wrapper modified to export "all" with -V option
 -- RequeueExit and RequeueExitHold configuration parameters modified to accept
    numeric ranges. For example "RequeueExit=1,2,3,4" and "RequeueExit=1-4" are
    equivalent.
 -- Correct the job array specification parser to accept brackets in job array
    expression (e.g. "123_[4,7-9]").
 -- Fix for misleading job submit failure errors sent to users. Previous error
    could indicate why specific nodes could not be used (e.g. too small memory)
    when other nodes could be used, but were not for another reason.
 -- Fix squeue --array to display correctly the array elements when the
    % separator is specified at the array submission time.
 -- Fix priority from not being calculated correctly due to memory issues.
 -- Fix a transient pending reason 'JobId=job_id has invalid QOS'.
 -- A non-administrator change to job priority will not be persistent except
    for holding the job. User's wanting to change a job priority on a persistent
    basis should reset it's "nice" value.
 -- Print buffer sizes as unsigned values when failed to pack messages.
 -- Fix race condition where sprio would print factors without weights applied.
 -- Document the sacct option JobIDRaw which for arrays prints the jobid instead
    of the arrayTaskId.
 -- Allow users to modify MinCPUsNode, MinMemoryNode and MinTmpDiskNode of
    their own jobs.
 -- Increase the jobid print field in SQUEUE_FORMAT in
    opt_modulefiles_slurm.in.
 -- Enable compiling without optimizations and with debugging symbols by
    default. Disable this by configuring with --disable-debug.
 -- job_submit/lua plugin: Add mail_type and mail_user fields.
 -- Correct output message from sshare.
 -- Use standard statvfs(2) syscall if available, in preference to
    non-standard statfs.
 -- Add a new option -U/--Users to sshare to display only users
    information, parent and ancestors are not printed.
 -- Purge 50000 records at a time so that locks can released periodically.
 -- Fix potentially uninitialized variables
 -- ALPS - Fix issue where a frontend node could become unresponsive and never
    added back into the system.
 -- Gate epilog complete messages as done with other messages
 -- If we have more than a certain number of agents (50) wait longer when gating
    rpcs.
 -- FrontEnd - ping non-responding or down nodes.
 -- switch/cray: If CR_PACK_NODES is configured, then set the environment
    variable "PMI_CRAY_NO_SMP_ENV=1"
 -- Fix invalid memory reference in SlurmDBD when putting a node up.
 -- Allow opening of plugstack.conf even when a symlink.
 -- Fix scontrol reboot so that rebooted nodes will not be set down with reason
    'Node xyz unexpectedly rebooted' but will be correctly put back to service.
 -- CRAY - Throttle the post NHC operations as to not hog the job write lock
    if many steps/jobs finish at once.
 -- Disable changes to GRES count while jobs are running on the node.
 -- CRAY - Fix issue with scontrol reconfig.
 -- slurmd: Remove wrong reporting of "Error reading step  ... memory limit".
    The logic was treating success as an error.
 -- Eliminate "Node ping apparently hung" error messages.
 -- Fix average CPU frequency calculation.
 -- When allocating resources with resolution of sockets, charge the job for all
    CPUs on allocated sockets rather than just the CPUs on used cores.
 -- Prevent slurmdbd error if cluster added or removed while rollup in progress.
    Removing a cluster can cause slurmdbd to abort. Adding a cluster can cause
    the slurmdbd rollup to hang.
 -- sview - When right clicking on a tab make sure we don't display the page
    list, but only the column list.
 -- FRONTEND - If doing a clean start make sure the nodes are brought up in the
    database.
 -- MySQL - Fix issue when using the TrackSlurmctldDown and nodes are down at
    the same time, don't double bill the down time.
 -- MySQL - Various memory leak fixes.
 -- sreport - Fix Energy displays
 -- Fix node manager logic to keep unexpectedly rebooted node in state
    NODE_STATE_DOWN even if already down when rebooted.
 -- Fix for array jobs submitted to multiple partitions not starting.
 -- CRAY - Enable ALPs mpp compatibility code in sbatch for native Slurm.

* Changes in Slurm 14.11.5
==========================
 -- Correct the squeue command taking into account that a node can
    have NULL name if it is not in DNS but still in slurm.conf.
 -- Fix slurmdbd regression which would cause a segfault when a node is set
    down with no reason.
 -- BGQ - Fix issue with job arrays not being handled correctly
    in the runjob_mux plugin.
 -- Print FAIR_TREE, if configured, in "scontrol show config" output for
    PriorityFlags.
 -- Add SLURM_JOB_GPUS environment variable to those available in the Prolog.
 -- Load lua-5.2 library if using lua5.2 for lua job submit plugin.
 -- GRES logic: Prevent bad node_offset due to not preserving no_consume flag.
 -- Fix wrong variables used in the wrapper functions needed for systems that
    don't support strong_alias
 -- Fix code for apple computers SOL_TCP is not defined
 -- Cray/BASIL - Check for mysql credentials in /root/.my.cnf.
 -- Fix sprio showing wrong priority for job arrays until priority is
    recalculated.
 -- Account to batch step all CPUs that are allocated to a job not
    just one since the batch step has access to all CPUs like other steps.
 -- Fix job getting EligibleTime set before meeting dependency requirements.
 -- Correct the initialization of QOS MinCPUs per job limit.
 -- Set the debug level of information messages in cgroup plugin to debug2.
 -- For job running under a debugger, if the exec of the task fails, then
    cancel its I/O and abort immediately rather than waiting 60 seconds for
    I/O timeout.
 -- Fix associations not getting default qos set until after a restart.
 -- Set the value of total_cpus not to be zero before invoking
    acct_policy_job_runnable_post_select.
 -- MySQL - When requesting cluster resources, only return resources for the
    cluster(s) requested.
 -- Add TaskPluginParam=autobind=threads option to set a default binding in the
    case that "auto binding" doesn't find a match.
 -- Introduce a new SchedulerParameters variable nohold_on_prolog_fail.
    If configured don't requeue jobs on hold is a Prolog fails.
 -- Make it so sched_params isn't read over and over when an epilog complete
    message comes in
 -- Fix squeue -L <licenses> not filtering out jobs with licenses.
 -- Changed the implementation of xcpuinfo_abs_to_mac() be identical
    _abs_to_mac() to fix CPUs allocation using cpuset cgroup.
 -- Improve the explanation of the unbuffered feature in the
    srun man page.
 -- Make taskplugin=cgroup work for core spec.  needed to have task/cgroup
    before.
 -- Fix reports not using the month usage table.
 -- BGQ - Sanity check given for translating small blocks into slurm bg_records.
 -- Fix bug preventing the requeue/hold or requeue/special_exit of job from the
    completing state.
 -- Cray - Fix for launching batch step within an existing job allocation.
 -- Cray - Add ALPS_APP_ID_ENV environment variable.
 -- Increase maximum MaxArraySize configuration parameter value from 1,000,001
    to 4,000,001.
 -- Added new SchedulerParameters value of bf_min_age_reserve. The backfill
    scheduler will not reserve resources for pending jobs until they have
    been pending for at least the specified number of seconds. This can be
    valuable if jobs lack time limits or all time limits have the same value.
 -- Fix support for --mem=0 (all memory of a node) with select/cons_res plugin.
 -- Fix bug that can permit someone to kill job array belonging to another user.
 -- Don't set the default partition on a license only reservation.
 -- Show a NodeCnt=0, instead of NO_VAL, in "scontrol show res" for a license
    only reservation.
 -- BGQ - When using static small blocks make sure when clearing the job the
    block is set up to it's original state.
 -- Start job allocation using lowest numbered sockets for block task
    distribution for consistency with cyclic distribution.

* Changes in Slurm 14.11.4
==========================
 -- Make sure assoc_mgr locks are initialized correctly.
 -- Correct check of enforcement when filling in an association.
 -- Make sacctmgr print out classification correctly for clusters.
 -- Add array_task_str to the perlapi job info.
 -- Fix for slurmctld abort with GRES types configured and no CPU binding.
 -- Fix for GRES scheduling where count > 1 per topology type (or GRES types).
 -- Make CR_ONE_TASK_PER_CORE work correctly with task/affinity.
 -- job_submit/pbs - Fix possible deadlock.
 -- job_submit/lua - Add "alloc_node" to job information available.
 -- Fix memory leak in mysql accounting when usage rollup happens.
 -- If users specify ALL together with other variables using the
    --export sbatch/srun command line option, propagate the users'
    environ to the execution side.
 -- Fix job array scheduling anomaly that can stop scheduling of valid tasks.
 -- Fix perl api tests for libslurmdb to work correctly.
 -- Remove some misleading logs related to non-consumable GRES.
 -- Allow --ignore-pbs to take effect when read as an #SBATCH argument.
 -- Fix Slurmdb::clusters_get() in perl api from not returning information.
 -- Fix TaskPluginParam=Cpusets from logging error message about not being able
    to remove cpuset dir which was already removed by the release_agent.
 -- Fix sorting by time left in squeue.
 -- Fix the file name substitution for job stderr when %A, %a %j and %u
    are specified.
 -- Remove minor warning when compiling slurmstepd.
 -- Fix database resources so they can add new clusters to them after they have
    initially been added.
 -- Use the slurm_getpwuid_r wrapper of getpwuid_r to handle possible
    interrupts.
 -- Correct the scontrol man page and command listing which node states can
    be set by the command.
 -- Stop sacct from printing non-existent stat information for
    Front End systems.
 -- Correct srun and acct_gather.conf man pages, mention Filesystem instead
    of Lustre.
 -- When a job using multiple partition starts send to slurmdbd only
    the partition in which the job runs.
 -- ALPS - Fix depth for MemoryAllocation in BASIL with CLE 5.2.3.
 -- Fix assoc_mgr hash to deal with users that don't have a uid yet when making
    reservations.
 -- When a job uses multiple partition set the environment variable
    SLURM_JOB_PARTITION to be the one in which the job started.
 -- Print spurious message about the absence of cgroup.conf at log level debug2
    instead of info.
 -- Enable CUDA v7.0+ use with a Slurm configuration of TaskPlugin=task/cgroup
    ConstrainDevices=yes (in cgroup.conf). With that configuration
    CUDA_VISIBLE_DEVICES will start at 0 rather than the device number.
 -- Fix job array logic that can cause slurmctld to abort.
 -- Report job "shared" field properly in scontrol, squeue, and sview.
 -- If a job is requeued because of RequeueExit or RequeueExitHold sent event
    REQUEUED to slurmdbd.