Newer
Older
This file describes changes in recent versions of SLURM. It primarily
documents those changes that are of interest to users and admins.
Danny Auble
committed
* Changes in SLURM 1.4.0-pre12
==============================
-- Added support for hard time limit by associations with added configuration
option PriorityUsageResetPeriod. This specifies the interval at which to
clear the record of time used. This is currently only available with the
priority/multifactor plugin.
-- Added SLURM_SUBMIT_DIR to sbatch's output environment variables.
-- Backup slurmdbd support implemented.
-- Update to checkpoint/xlch logic from Hongjia Cao, NUDT.
Danny Auble
committed
* Changes in SLURM 1.4.0-pre10
==============================
-- Critical bug fix in task/affinity when the CoresPerSocket is greater
than the ThreadsPerCore (invalid memory reference).
-- Add DebugFlag parameter of "Wiki" to log sched/wiki and wiki2
communications in greater detail.
-- Add "-d <slurmstepd_path>" as an option to the slurmd daemon to
specifying a non-stardard slurmstepd file, used for testing purposes.
-- Minor cleanup to crypto/munge plugin.
- Restrict uid allowed to decode job credentials in crypto/munge
- Get slurm user id early in crypto/munge
- Remove buggy error code handling in crypto/munge
-- Added sprio command - works only with the priority/multifactor plugin
-- Add real topology plugin infrastructure (it was initially added
directly into slurmctld code). To specify topology information,
set TopologyType=topology/tree and add configuration information
to a new file called topology.conf. See "man topology.conf" or
topology.html web page for details.
-- Set "/proc/self/oom_adj" for slurmd and slurmstepd daemons based upon
the values of SLURMD_OOM_ADJ and SLURMSTEPD_OOM_ADJ environment
variables. This can be used to prevent daemons being killed when
a node's memory is exhausted. Based upon patch by Hongjia Cao, NUDT.
-- Fix several bugs in task/affinity: cpuset logic was broken and
--cpus-per-task option not properly handled.
-- Ensure slurmctld adopts SlurmUser GID as well as UID on startup.
* Changes in SLURM 1.4.0-pre9
=============================
-- OpenMPI users only: Add srun logic to automatically recreate and
re-launch a job step if the step fails with a reserved port conflict.
-- Added TopologyPlugin configuration parameter.
-- Added switch topology data structure to slurmctld (for use by select
plugin) add load it based upon new slurm.conf parameters: SwitchName,
Nodes, Switches and LinkSpeed.
-- Modify select/linear and select/cons_res plugins to optimize resource
allocation with respect to network topology.
-- Added support for new configuration parameter EpilogSlurmctld (executed
by slurmctld daemon).
-- Added checkpoint/blcr plugin, SLURM now support job checkpoint/restart
using BLCR. Patch from Hongjia Cao, NUDT, China.
-- Made a variety of new environment variables available to PrologSlurmctld
and EpilogSlurmctld. See the "Prolog and Epilog Scripts" section of the
slurm.conf man page for details.
-- NOTE: Cold-start (without preserving state) required for upgrade from
version 1.4.0-pre8.
* Changes in SLURM 1.4.0-pre8
=============================
-- In order to create a new partition using the scontrol command, use
the "create" option rather than "update" (which will only operate
upon partitions that already exist).
-- Added environment variable SLURM_RESTART_COUNT to batch jobs to
indicated the count of job restarts made.
-- Added sacctmgr command "show config".
-- Added the scancel option --nodelist to cancel any jobs running on a
given list of nodes.
-- Add partition-specific DefaultTime (default time limit for jobs,
if not specified use MaxTime for the partition. Patch from Par
Andersson, National Supercomputer Centre, Sweden.
-- Add support for the scontrol command to be able change the Weight
associated with nodes. Patch from Krishnakumar Ravi[KK] (HP).
-- Add DebugFlag configuration option of "CPU_Bind" for detailed CPU
binding information to be logged.
-- Fix some significant bugs in task binding logic (possible infinite loops
and memory corruption).
-- Add new node state flag of NODE_STATE_MAINT indicating the node is in
a reservation of type MAINT.
-- Modified task/affinity plugin to automatically bind tasks to sockets,
cores, or threads as appropriated based upon resource allocation and
task count. User can override with srun's --cpu_bind option.
-- Fix bug in backfill logic for select/cons_res plugin, resulted in
error "cons_res:_rm_job_from_res: node_state mis-count".
-- Add logic go bind a batch job to the resources allocated to that job.
-- Add configuration parameter MpiParams for (future) OpenMPI port
management. Add resv_port_cnt and resv_ports fields to the job step
data structures. Add environment variable SLURM_STEP_RESV_PORTS to
show what ports are reserved for a job step.
-- Add support for SchedulerParameters=interval=<sec> to control the time
interval between executions of the backfill scheduler logic.
-- Preserve record of last job ID in use even when doing a cold-start unless
there is no job state file or there is a change in its format (which only
happens when there is a change in SLURM's major or minor version number:
v1.3 -> v1.4).
-- Added new configuration parameter KillOnBadExit to kill a job step as soon
as any task of a job step exits with a non-zero exit code. Patch based
on work from Eric Lin, Bull.
-- Add spank plugin calls for use by salloc and sbatch command, see
"man spank" for details.
-- NOTE: Cold-start (without preserving state) required for upgrade from
* Changes in SLURM 1.4.0-pre7
=============================
-- Bug fix for preemption with select/cons_res when there are no idle nodes.
-- Bug fix for use of srun options --exclusive and --cpus-per-task together
for job step resource allocation (tracking of cpus in use was bad).
-- Added the srun option --preserve-env to pass the current values of
environment variables SLURM_NNODES and SLURM_NPROCS through to the
executable, rather than computing them from commandline parameters.
-- For select/cons_res or sched/gang only: Validate a job's resource
allocation socket and core count on each allocated node. If the node's
configuration has been changed, then abort the job.
-- For select/cons_res or sched/gang only: Disable updating a node's
processor count if FastSchedule=0. Administrators must set a valid
processor count although the memory and disk space configuration can
be loaded from the compute node when it starts.
-- Add configure option "--disable-iso8601" to disable SLURM use of ISO 8601
time format at the time of SLURM build. Default output for all commands
is now ISO 8601 (yyyy-mm-ddThh:mm:ss).
-- Add support for scontrol to explicity power a node up or down using the
configured SuspendProg and ResumeProg programs.
-- Fix book select/cons_res logic for tracking the number of allocated
CPUs on a node when a partition's Shared value is YES or FORCE.
-- Added configure options "--enable-cray-xt" and "--with-apbasil=PATH" for
eventual support of Cray-XT systems.
* Changes in SLURM 1.4.0-pre6
=============================
-- Fix job preemption when sched/gang and select/linear are configured with
non-sharing partitions.
-- In select/cons_res insure that required nodes have available resources.
* Changes in SLURM 1.4.0-pre5
=============================
-- Correction in setting of SLURM_CPU_BIND environment variable.
-- Rebuild slurmctld's job select_jobinfo->node_bitmap on restart/reconfigure
of the daemon rather than restoring the bitmap since the nodes in a system
can change (be added or removed).
-- Add configuration option "--with-cpusetdir=PATH" for non-standard
locations.
-- Get new multi-core data structures working on BlueGene systems.
-- Modify PMI_Get_clique_ranks() to return an array of integers rather
than a char * to satisfy PMI standard. Correct logic in
PMI_Get_clique_size() for when srun --overcommit option is used.
-- Fix bug in select/cons_res, allocated a job all of the processors on a
node when the --exclusive option is specified as a job submit option.
-- Add NUMA cpu_bind support to the task affinity plugin. Binds tasks to
a set of CPUs that belong NUMA locality domain with the appropriate
--cpu-bind option (ldoms, rank_ldom, map_ldom, and mask_ldom), see
"man srun" for more information.
* Changes in SLURM 1.4.0-pre4
=============================
-- For task/affinity, force jobs to use a particular task binding by setting
the TaskPluginParam configuration parameter rather than slurmd's
SLURM_ENFORCED_CPU_BIND environment variable.
-- Enable full preemption of jobs by partition with select/cons_res
(cons_res_preempt.patch from Chris Holmes, HP).
-- Add configuration parameter DebugFlags to provide detailed logging for
specific subsystems (steps and triggers so far).
-- srun's --no-kill option is passed to slurmctld so that a job step is
killed even if the node where srun executes goes down (unless the
--no-kill option is used, previous termination logic would fail if
srun was not responding).
-- Transfer a job step's core bitmap from the slurmctld to the slurmd
within the job step credential.
-- Add cpu_bind, cpu_bind_type, mem_bind and mem_bind_type to job allocation
request and job_details structure in slurmctld. Add support to --cpu_bind
and --mem_bind options from salloc and sbatch commands.
Moe Jette
committed
* Changes in SLURM 1.4.0-pre3
=============================
-- Internal changes: CPUs per node changed from 32-bit to 16-bit size.
Node count fields changed from 16-bit to 32-bit size in some structures.
-- Remove select plugin functions select_p_get_extra_jobinfo(),
select_p_step_begin() and select_p_step_fini().
-- Remove the following slurmctld job structure fields: num_cpu_groups,
cpus_per_node, cpu_count_reps, alloc_lps_cnt, alloc_lps, and used_lps.
Use equivalent fields in new "select_job" structure, which is filled
in by the select plugins.
-- Modify mem_per_task in job step request from 16-bit to 32-bit size.
Use new "select_job" structure for the job step's memory management.
-- Add core_bitmap_job to slurmctld's job step structure to identify
Moe Jette
committed
-- Add new configuration option OverTimeLimit to permit jobs to exceed
their (soft) time limit by a configurable amount. Backfill scheduling
will be based upon the soft time limit.
Moe Jette
committed
-- Remove select_g_get_job_cores(). That data is now within the slurmctld's
job structure.
* Changes in SLURM 1.4.0-pre2
=============================
-- Remove srun's --ctrl-comm-ifhn-addr option (for PMI/MPICH2). It is no
Loading
Loading full blame...