See bugzilla bug 132

When using select/cons_res and CR_Core_Memory, hyperthreaded nodes may be
overcommitted on memory when CPU counts are scaled. I've tested 2.4.2 and HEAD
(2.5.0-pre3).

Conditions:
-----------
* SelectType=select/cons_res
* SelectTypeParameters=CR_Core_Memory
* Using threads
  - Ex. "NodeName=linux0 Sockets=1 CoresPerSocket=4 ThreadsPerCore=2
RealMemory=400"

Description:
------------
In the cons_res plugin, _verify_node_state() in job_test.c checks if a node has
sufficient memory for a job. However, the per-CPU memory limits appear to be
scaled by the number of threads. This new value may exceed the available memory
on the node. And, once a node is overcommitted on memory, future memory checks
in _verify_node_state() will always succeed.

Scenario to reproduce:
----------------------
With the example node linux0, we run a single-core job with 250MB/core
    srun --mem-per-cpu=250 sleep 60

cons_res checks that it will fit: ((real - alloc) >= job mem)
    ((400 - 0) >= 250) and the job starts

Then, the memory requirement is doubled:
    "slurmctld: error: cons_res: node linux0 memory is overallocated (500) for
job X"
    "slurmd: scaling CPU count by factor of 2"

This job should not have started

While the first job is still running, we submit a second, identical job
    srun --mem-per-cpu=250 sleep 60

cons_res checks that it will fit:
    ((400 - 500) >= 250), the unsigned int wraps, the test passes, and the job
starts

This second job also should not have started

purged from the system if its front-end node goes down.

database, and the job is running on a small block make sure we free up
the correct node count.

This addresses bug 130

with a job running or trying to run on it.

are planning on using the block.  Previously it would fail those jobs
erroneously.

no big deal, but now we get just a bit more memory allocated (4152 over
instead of the 4100 we were previously looking for)

they are a no-opt when they get to the DBD and we would favor having
job completion messages instead of step completion messages if possible.

or previous piecemeal method.

Adapted from a patch from Stephen Trofinoff <trofinoff@cscs.ch>

Only set memory.memsw.limit_in_bytes to the computed amount of allowed
RAM+Swap when ConstrainSwapSpace=yes in cgroup.conf.
When ConstrainSwapSpace=yes and ConstrainRAMSpace=no, automatically
set AllowedRAMSpace to 100% in order to compute the memsw.limit based
on the allocated memory plus the allowed swap percent. Then use that
limit for both mem and mem+sw limits.