From b1fda7c60bb99ff9dbc0dd7f822615d0f0a901b3 Mon Sep 17 00:00:00 2001 From: Morris Jette <jette@schedmd.com> Date: Wed, 23 Dec 2015 15:52:22 -0800 Subject: [PATCH] Add cray-specific port config info Add cray-specific port configuration information which considers Realm-Specific IP Addressing (RSIP). bug 2277 --- doc/html/cray.shtml | 10 +++++++++- doc/man/man5/slurm.conf.5 | 30 ++++++++++++++++++++++++++++-- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/doc/html/cray.shtml b/doc/html/cray.shtml index 9258760b14f..d6ef40a6450 100644 --- a/doc/html/cray.shtml +++ b/doc/html/cray.shtml @@ -136,6 +136,14 @@ page for details. </p> +<li>SlurmctldPort, SlurmdPort, SrunPortRange</li> +<p> + Realm-Specific IP Addressing (RSIP) will automatically try to interact with + anything opened on ports 8192 to 60000. Configure SlurmctldPort, SlurmdPort, + and SrunPortRange to use ports above 60000. In the case of SrunPortRange, + making 1000 or more ports available is recommended. +</p> + <li>Switch</li> <p> Set <b><i>SwitchType=switch/cray</i></b> to use. @@ -201,6 +209,6 @@ is down and prevents any new jobs from being started. <p class="footer"><a href="#top">top</a></p> -<p style="text-align:center;">Last modified 26 February 2015</p> +<p style="text-align:center;">Last modified 23 December 2015</p> <!--#include virtual="footer.txt"--> diff --git a/doc/man/man5/slurm.conf.5 b/doc/man/man5/slurm.conf.5 index fae6a155c19..6e99034eaab 100644 --- a/doc/man/man5/slurm.conf.5 +++ b/doc/man/man5/slurm.conf.5 @@ -2960,6 +2960,11 @@ NOTE: Either \fBslurmctld\fR and \fBslurmd\fR daemons must not execute on the same nodes or the values of \fBSlurmctldPort\fR and \fBSlurmdPort\fR must be different. +\fBNote\fR: On Cray systems, Realm-Specific IP Addressing (RSIP) will +automatically try to interact with anything opened on ports 8192\-60000. +Configure SlurmctldPort to use a port outside of the configured SrunPortRange +and RSIP's port range. + .TP \fBSlurmctldTimeout\fR The interval, in seconds, that the backup controller waits for the @@ -3041,6 +3046,11 @@ NOTE: Either slurmctld and slurmd daemons must not execute on the same nodes or the values of \fBSlurmctldPort\fR and \fBSlurmdPort\fR must be different. +\fBNote\fR: On Cray systems, Realm-Specific IP Addressing (RSIP) will +automatically try to interact with anything opened on ports 8192\-60000. +Configure SlurmdPort to use a port outside of the configured SrunPortRange +and RSIP's port range. + .TP \fBSlurmdSpoolDir\fR Fully qualified pathname of a directory into which the \fBslurmd\fR @@ -3105,7 +3115,13 @@ By default these ports are ephemeral meaning the port numbers are selected by the kernel. Using this parameter allow sites to configure a range of ports from which srun ports will be selected. This is useful if sites want to allow only certain port range on their network. -\fBNote\fR. A sufficient number of ports must be configured based on the + +\fBNote\fR: On Cray systems, Realm-Specific IP Addressing (RSIP) will +automatically try to interact with anything opened on ports 8192\-60000. +Configure SrunPortRange to use a range of ports above those used by RSIP, +ideally 1000 or more ports, for example "SrunPortRange=60001\-63000". + +\fBNote\fR: A sufficient number of ports must be configured based on the estimated number of srun on the submission nodes considering that srun opens 3 listening ports plus 2 more for every 48 hosts. Example: .RS @@ -3726,7 +3742,12 @@ to for work on this particular node. By default there is a single port number for all \fBslurmd\fR daemons on all compute nodes as defined by the \fBSlurmdPort\fR configuration parameter. Use of this option is not generally recommended except for development or testing purposes. If multiple -\fBslurmd\fR daemons execute on a node this can specify a range of ports +\fBslurmd\fR daemons execute on a node this can specify a range of ports. + +\fBNote\fR: On Cray systems, Realm-Specific IP Addressing (RSIP) will +automatically try to interact with anything opened on ports 8192\-60000. +Configure Port to use a port outside of the configured SrunPortRange and +RSIP's port range. .TP \fBProcs\fR @@ -3998,6 +4019,11 @@ number for all \fBslurmd\fR daemons on all frontend nodes as defined by the \fBSlurmdPort\fR configuration parameter. Use of this option is not generally recommended except for development or testing purposes. +\fBNote\fR: On Cray systems, Realm-Specific IP Addressing (RSIP) will +automatically try to interact with anything opened on ports 8192\-60000. +Configure Port to use a port outside of the configured SrunPortRange and +RSIP's port range. + .TP \fBReason\fR Identifies the reason for a frontend node being in state "DOWN", "DRAINED" -- GitLab