Improve preemption documentation

Improve documentation with respect to preemption rules, namely
PreemptMode=suspend is incompatible with PreemptType=preempt/qos.
Patch from Bill Brophy, Bull.

doc/html/preempt.shtml

@@ -102,7 +102,7 @@ parameters are not critical.
 The PreemptMode can be specified on a system-wide basis or on a per-partition
 basis when <I>PreemptType=preempt/partition_prio</I>. Note that when specified 
 on a partition, a compatible mode must also be specified system-wide;
-specifically if a PreemptMode is set to SUSPEND for any partition or QOS, then
+specifically if a PreemptMode is set to SUSPEND for any partition, then
 the system-wide PreemptMode must include the GANG parameter so the module
 responsible for resuming jobs executes.
 Configure to <I>CANCEL</I>, <I>CHECKPOINT</I>,
@@ -139,7 +139,9 @@ can preempt jobs from lower priority partitions.</LI>
 can preempt jobs from a lower QOS. These jobs can be in the same partition
 or different partitions. PreemptMode must be set to CANCEL, CHECKPOINT,
 or REQUEUE. This option requires the use of a database identifying
-available QOS and their preemption rules. </LI>
+available QOS and their preemption rules. This option is not compatible with
+PreemptMode=OFF or PreemptMode=SUSPEND (i.e. preempted jobs must be removed
+from the resources).</LI>
 </UL>
 </LI>
 <LI>

doc/man/man1/sacctmgr.1

@@ -1018,7 +1018,9 @@ Mechanism used to preempt jobs of this QOS if the clusters \fIPreemptType\fP
 is configured to \fIpreempt/qos\fP.  The default preemption mechanism
 is specified by the cluster\-wide \fIPreemptMode\fP configuration parameter.
 Possible values are "Cluster" (meaning use cluster default), "Cancel",
-"Checkpoint", "Requeue" and "Suspend".
+"Checkpoint" and "Requeue".  This option is not compatible with
+PreemptMode=OFF or PreemptMode=SUSPEND (i.e. preempted jobs must be removed
+from the resources).
 
 .TP
 \fIPriority\fP
@@ -1170,7 +1172,9 @@ Mechanism used to preempt jobs of this QOS if the clusters \fIPreemptType\fP
 is configured to \fIpreempt/qos\fP.  The default preemption mechanism
 is specified by the cluster\-wide \fIPreemptMode\fP configuration parameter.
 Possible values are "Cluster" (meaning use cluster default), "Cancel",
-"Checkpoint", "Requeue" and "Suspend".
+"Checkpoint" and "Requeue".  This option is not compatible with
+PreemptMode=OFF or PreemptMode=SUSPEND (i.e. preempted jobs must be removed
+from the resources).
 
 .TP
 \fIPriority\fP

src/sacctmgr/qos_functions.c

@@ -205,6 +205,10 @@ static int _set_cond(int *start, int argc, char *argv[],
 					" Bad Preempt Mode given: %s\n",
 					argv[i]);
 				exit_code = 1;
+			} else if (qos_cond->preempt_mode == PREEMPT_MODE_SUSPEND) {
+				printf("PreemptType and PreemptMode "
+					"values incompatible\n");
+				exit_code = 1;
 			} else
 				set = 1;
 		} else {
@@ -411,6 +415,10 @@ static int _set_rec(int *start, int argc, char *argv[],
 					" Bad Preempt Mode given: %s\n",
 					argv[i]);
 				exit_code = 1;
+			} else if (qos->preempt_mode == PREEMPT_MODE_SUSPEND) {
+				printf("PreemptType and PreemptMode "
+					"values incompatible\n");
+				exit_code = 1;
 			} else
 				set = 1;
 		/* Preempt needs to follow PreemptMode */