From dc71ee532a7212e4b2cae90ba794911cff46af86 Mon Sep 17 00:00:00 2001
From: Morris Jette <jette@schedmd.com>
Date: Tue, 16 Aug 2011 14:24:36 -0700
Subject: [PATCH] BlueGene/Q: bluegene.conf man page update
Large-scale update of bluegene.conf man page to incorporate BlueGene/Q
information plus some general updates.
---
doc/man/man5/bluegene.conf.5 | 151 +++++++++++++++++++----------------
1 file changed, 83 insertions(+), 68 deletions(-)
diff --git a/doc/man/man5/bluegene.conf.5 b/doc/man/man5/bluegene.conf.5
index dc713da6177..5bdb8e2a0c0 100644
--- a/doc/man/man5/bluegene.conf.5
+++ b/doc/man/man5/bluegene.conf.5
@@ -1,8 +1,10 @@
-.TH "bluegene.conf" "5" "April 2008" "bluegene.conf 2.0" "Slurm configuration file"
+.TH "bluegene.conf" "5" "August 2011" "bluegene.conf 2.3" "SLURM configuration file"
+
.SH "NAME"
-bluegene.conf \- Slurm configuration file for BlueGene systems
+bluegene.conf \- SLURM configuration file for BlueGene systems
+
.SH "DESCRIPTION"
-\fB/etc/bluegene.conf\fP is an ASCII file which describes BlueGene specific
+\fB/etc/bluegene.conf\fP is an ASCII file which describes IBM BlueGene specific
SLURM configuration information. This includes specifications for bgblock
layout, configuration, logging, etc.
The file location can be modified at system build time using the
@@ -17,33 +19,36 @@ Changes to the configuration file take only effect upon restart of
the slurmctld daemon. "scontrol reconfigure" does nothing with this file.
Changes will only take place after a restart of the controller.
.LP
+There are some differences between BlueGene/L, BlueGene/P and BlueGene/Q
+systems with respects to the contents of the bluegene.conf file.
-There are some differences between Bluegene/L and Bluegene/P in respects to the contents of the bluegene.conf file.
-
-.SH "The Bluegene/L specific options are:"
+.SH "The BlueGene/L specific options are:"
.TP
\fBAltBlrtsImage\fR
-Alternative BlrtsImage. This is an optional field only used for
-mulitple images on a system and should be followed by a Groups= with
-the user groups allowed to use this image (i.e. Groups=da,jette) if
-Groups= is not stated then this image will be able to be used by all
-groups. You can put as many alternative images as you want in the conf file.
+Alternative BlrtsImage. This is an optional field only used for multiple
+images on a system and should be followed by a Groups option indicating
+the user groups allowed to use this image (i.e. Groups=da,jette). If
+Groups is not specified then this image will be usable by all
+groups. You can put as many alternative images as you want in the
+bluegene.conf file.
.TP
\fBAltLinuxImage\fR
-Alternative LinuxImage. This is an optional field only used for
-mulitple images on a system and should be followed by a Groups= with
-the user groups allowed to use this image (i.e. Groups=da,jette) if
-Groups= is not stated then this image will be able to be used by all
-groups. You can put as many alternative images as you want in the conf file.
+Alternative LinuxImage. This is an optional field only used for multiple
+images on a system and should be followed by a Groups option indicating
+the user groups allowed to use this image (i.e. Groups=da,jette). If
+Groups is not specified then this image will be usable by all
+groups. You can put as many alternative images as you want in the
+bluegene.conf file.
.TP
\fBAltRamDiskImage\fR
-Alternative RamDiskImage. This is an optional field only used for
-mulitple images on a system and should be followed by a Groups= with
-the user groups allowed to use this image (i.e. Groups=da,jette) if
-Groups= is not stated then this image will be able to be used by all
-groups. You can put as many alternative images as you want in the conf file.
+Alternative RamDiskImage. This is an optional field only used for multiple
+images on a system and should be followed by a Groups option indicating
+the user groups allowed to use this image (i.e. Groups=da,jette). If
+Groups is not specified then this image will be usable by all
+groups. You can put as many alternative images as you want in the
+bluegene.conf file.
.TP
\fBBlrtsImage\fR
@@ -60,21 +65,21 @@ There is no default value and this must be specified.
RamDiskImage used for creation of all bgblocks.
There is no default value and this must be specified.
-.SH "The Bluegene/P specific options are:"
+.SH "The BlueGene/P specific options are:"
.TP
\fBAltCnloadImage\fR
-Alternative CnloadImage. This is an optional field only used for
-mulitple images on a system and should be followed by a Groups= with
-the user groups allowed to use this image (i.e. Groups=da,jette) if
-Groups= is not stated then this image will be able to be used by all
+Alternative CnloadImage. This is an optional field only used for multiple
+images on a system and should be followed by a Groups option indicating
+the user groups allowed to use this image (i.e. Groups=da,jette). If
+Groups is not specified then this image will be usable by all
groups. You can put as many alternative images as you want in the conf file.
.TP
\fBAltIoloadImage\fR
-Alternative IoloadImage. This is an optional field only used for
-mulitple images on a system and should be followed by a Groups= with
-the user groups allowed to use this image (i.e. Groups=da,jette) if
-Groups= is not stated then this image will be able to be used by all
+Alternative IoloadImage. This is an optional field only used for multiple
+images on a system and should be followed by a Groups option indicating
+the user groups allowed to use this image (i.e. Groups=da,jette). If
+Groups is not specified then this image will be usable by all
groups. You can put as many alternative images as you want in the conf file.
.TP
@@ -87,20 +92,19 @@ There is no default value and this must be specified.
IoloadImage used for creation of all bgblocks.
There is no default value and this must be specified.
-.SH "All options below are common on all Bluegene systems:"
+.SH "All options below are common on all BlueGene systems:"
.TP
\fBAltMloaderImage\fR
-Alternative MloaderImage. This is an optional field only used for
-mulitple images on a system and should be followed by a Groups= with
-the user groups allowed to use this image (i.e. Groups=da,jette) if
-Groups= is not stated then this image will be able to be used by all
+Alternative MloaderImage. This is an optional field only used for multiple
+images on a system and should be followed by a Groups option indicating
+the user groups allowed to use this image (i.e. Groups=da,jette). If
+Groups is not specified then this image will be usable by all
groups. You can put as many alternative images as you want in the conf file.
.TP
\fBBasePartitionNodeCount\fR
-The number of c\-nodes per base partition.
-There is no default value and this must be specified. (For bgl systems this
-is usually 512)
+The number of c\-nodes (compute nodes) per base partition.
+There is no default value and this must be specified (usually 512).
.TP
\fBBridgeAPILogFile\fR
@@ -116,33 +120,39 @@ The default value is 0.
.TP
\fB0\fR: Log only error and warning messages
.TP
-\fB1\fR: Log level 0 and information messasges
+\fB1\fR: Log level 0 plus information messages
.TP
-\fB2\fR: Log level 1 and basic debug messages
+\fB2\fR: Log level 1 plus basic debug messages
.TP
-\fB3\fR: Log level 2 and more debug message
+\fB3\fR: Log level 2 plus more debug message
.TP
\fB4\fR: Log all messages
.RE
.TP
\fBDenyPassthrough\fR
-Specify which dimsions you do not want to allow pass throughs. Valid options are X, Y, Z or all.
-example: If you don't want to allow passthroughs in the X and Y diminsions you would specify DenyPassthrough=X,Y
+Specify which dimensions you do not want to allow pass\-throughs.
+Valid options are A, X, Y, Z or all ("A" applies only to BlueGene/Q systems).
+For example, to prevent pass\-throughs in the X and Y dimensions you would
+specify "DenyPassthrough=X,Y".
+By default, pass\-throughs are enabled in every dimension.
.TP
\fBLayoutMode\fR
Describes how SLURM should create bgblocks.
.RS
+.TP 10
+\fBSTATIC\fR:
+Create and use the defined non\-overlapping bgblocks.
.TP
-\fBSTATIC\fR: Create and use the defined non\-overlapping bgblocks.
-.TP
-\fBOVERLAP\fR: Create and use the defined bgblocks, which may overlap.
+\fBOVERLAP\fR:
+Create and use the defined bgblocks, which may overlap.
It is highly recommended that none of the bgblocks have any passthroughs
in the X\-dimension.
\fBUse this mode with extreme caution.\fR
.TP
-\fBDYNAMIC\fR: Create and use bglblocks as needed for each job.
+\fBDYNAMIC\fR:
+Create and use bgblocks as needed for each job.
Bgblocks will not be defined in the bluegene.conf file.
Dynamic partitioning may introduce fragmentation of resources
and starvation of larger jobs.
@@ -157,25 +167,26 @@ There is no default value and this must be specified.
.TP
\fBNodeCardNodeCount\fR
Number of c\-nodes per node card.
-There is no default value and this must be specified. (For bgl systems this
-is usually 32)
+There is no default value and this must be specified. For BlueGene/L systems
+this is usually 32.
.TP
\fBNumPsets\fR
The Numpsets used for creation of all bgblocks. This value really means the
-number of IOnodes on a base partition. This number must be the smallest
+number of IO nodes on a base partition. This number must be the smallest
number if you have a heterogeneous system.
There is no default value and this must be specified. The typical settings
-for bgl systems goes as follows... For IO rich systems 64 is the value that
+for BlueGene/L systems are as follows: For IO rich systems, 64 is the value that
should be used to create small blocks. For systems that are not IO rich, or
-you do not wish to create small blocks, 8 is usually the number to use.
-For bgp IO rich systems 32 is the value that should be used to create small
-blocks since you can only have 2 ionodes per nodecard instead of 4 like on bgl.
+for which small blocks are not desirable, 8 is usually the number to use.
+For BlueGene/P IO rich systems, 32 is the value that should be used to create
+small blocks since there are only 2 IO nodes per nodecard instead of 4 as on
+BlueGene/L.
.LP
Each bgblock is defined by the base partitions used to construct it.
-Ordering is very important for laying out switch wires. Please create
-blocks with smap, and once done don't change the order of blocks created.
+Ordering is very important for laying out switch wires. Please use the smap
+tool to define blocks and do not change the order of blocks created.
A bgblock is implicitly created containing all resources on the system.
Bgblocks must not overlap in static mode (except for implicitly
created bgblock). This will be the case when smap is used to create
@@ -186,23 +197,27 @@ will be based upon the NodeName defined in slurm.conf
.TP
\fBBPs\fR
-Define the XYZ coordinates of the bgblock end points.
+Define the coordinates of the bgblock end points.
+For BlueGene/L and BlueGene/P systems there will be three coordinates (X, Y, and Z).
+For BlueGene/Q systems there will be for coordinates (A, X, Y, and Z).
.TP
\fBType\fR
Define the network connection type for the bgblock.
The default value is TORUS.
.RS
+.TP 8
+\fBMESH\fR:
+Communication occur over a mesh.
.TP
-\fBMESH\fR: Communication occur over a mesh.
-.TP
-\fBSMALL\fR: The base partition is divided into more than one bgblock.
+\fBSMALL\fR:
+The base partition is divided into more than one bgblock.
The administrator should define the number of single node cards and
quarter base partition blocks using the options \fB32CNBlocks\fR and
-\fB128CNBlocks\fR respectively for a Bluegene L system. \fB16CNBlocks\fR,
+\fB128CNBlocks\fR respectively for a BlueGene/L system. \fB16CNBlocks\fR,
\fB64CNBlocks\fR, and \fB256CNBlocks\fR are also available for
-Bluegene P systems. Keep in mind you
-must have enough ionodes to make all these configurations possible.
+BlueGene/P systems. Keep in mind you
+must have enough IO nodes to make all these configurations possible.
The total number of c\-nodes in defined blocks must not exceed
\fBBasePartitionNodeCnt\fR.
@@ -210,8 +225,8 @@ If not specified, the base partition will be divided into four
blocks.
See example below.
.TP
-\fBTORUS\fR: Communications occur over a torus (end\-points of network
-directly connect.
+\fBTORUS\fR:
+Communications occur over a torus (end\-points of network directly connect.
.RE
.SH "EXAMPLE"
@@ -247,7 +262,7 @@ LayoutMode=STATIC
.br
##################################################################
.br
-# LEAVE AS COMMENT, Full\-system bglblock, implicitly created
+# LEAVE AS COMMENT, Full\-system bgblock, implicitly created
.br
# BPs=[000x333] Type=TORUS # 4x4x4 = 64 midplanes
.br
@@ -268,7 +283,7 @@ BPs=[332] Type=TORUS # 1x1x1 = 1
BPs=[333] Type=SMALL 32CNBlocks=4 128CNBlocks=3 # 1/16 * 4 + 1/4 * 3
.SH "COPYING"
-Copyright (C) 2006 The Regents of the University of California.
+Copyright (C) 2006-2010 The Regents of the University of California.
Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
CODE\-OCEC\-09\-009. All rights reserved.
.LP
@@ -288,4 +303,4 @@ details.
/etc/bluegene.conf
.SH "SEE ALSO"
.LP
-\fBslurm.conf\fR(5)
+\fBsmap\fR(1), \fBslurm.conf\fR(5)
--
GitLab