From ed45d6efef9dc407974d0a06ae374d3af2dbceb2 Mon Sep 17 00:00:00 2001 From: Danny Auble <da@llnl.gov> Date: Wed, 15 Nov 2006 20:09:04 +0000 Subject: [PATCH] documentation added for image support for bluegene systems. --- NEWS | 1 - doc/html/bluegene.shtml | 70 ++++++++++++++++++++++++++++++--------- doc/man/man1/sbatch.1 | 15 +++++++++ doc/man/man1/srun.1 | 8 +++++ etc/bluegene.conf.example | 49 ++++++++++++++++++++++----- 5 files changed, 119 insertions(+), 24 deletions(-) diff --git a/NEWS b/NEWS index 269dcf2dc73..7ff5658c9a6 100644 --- a/NEWS +++ b/NEWS @@ -4,7 +4,6 @@ documents those changes that are of interest to users and admins. * Changes in SLURM 1.2.0-pre7 ============================= -- BLUEGENE - added configurable images for bluegene block creation. - (No documentation out side of srun and sbatch just yet) -- Plug a bunch of memory leaks. * Changes in SLURM 1.2.0-pre6 diff --git a/doc/html/bluegene.shtml b/doc/html/bluegene.shtml index 475667eea26..9f985261730 100644 --- a/doc/html/bluegene.shtml +++ b/doc/html/bluegene.shtml @@ -36,16 +36,22 @@ to represent multiples of 1024 (e.g. "2k" is equivalent to "2048").</p> <h2>User Tools</h2> -<p>The normal set of SLURM user tools: srun, scancel, sinfo, squeue and scontrol +<p>The normal set of SLURM user tools: srun, sbatch, scancel, sinfo, squeue, +and scontrol provide all of the expected services except support for job steps. SLURM performs resource allocation for the job, but initiation of tasks is performed using the <i>mpirun</i> command. SLURM has no concept of a job step on BlueGene. -Three new srun options are available: +Seven new srun, sbatch options are available: <i>--geometry</i> (specify job size in each dimension), <i>--no-rotate</i> (disable rotation of geometry), <i>--conn-type</i> (specify interconnect type between base partitions, mesh or torus). +<i>--blrts-image</i> (specify alternative blrts image for bluegene block. Default if not set). +<i>--linux-image</i> (specify alternative linux image for bluegene block. Default if not set). +<i>--mloader-image</i> (specify alternative mloader image for bluegene block. Default if not set). +<i>--ramdisk-image</i> (specify alternative ramdisk image for bluegene block. Default if not set). The <i>--nodes</i> option with a minimum and (optionally) maximum node count continues to be available. + Note that this is a c-node count.</p> <p>To reiterate: srun is used to submit a job script, but mpirun is used to launch the parallel tasks. @@ -91,7 +97,8 @@ For example, "bg[620x731]" is used to represent the eight base partitions enclos with endpoints bg620 and bg731 (bg620, bg621, bg630, bg631, bg720, bg721, bg730 and bg731).</p></a> -<p>One new tool provided is <i>smap</i>. +<p>One new tool provided is <i>smap</i> as of 1.2 <i>sview</i> is +another new tool offering even more viewing and configuring options. Smap is aware of system topography and provides a map of what base partitions are allocated to jobs, partitions, etc. See the smap man page for details. @@ -363,13 +370,13 @@ This is done using the keywords <i>BasePartitionNodeCnt=NODE_COUNT</i> and <i>NodeCardNodeCnt=NODE_COUNT</i> respectively in the <i>bluegene.conf</i> file (i.e. <i>BasePartitionNodeCnt=512</i> and <i>NodeCardNodeCnt=32</i>).</p> -<p>Note that the <i>Image</i> and <i>Numpsets</i> values defined in +<p>Note that the <i>Numpsets</i> values defined in <i>bluegene.conf</i> are used only when SLURM creates bgblocks. -If previously defined bgblocks are used by SLURM, their configurations -are not altered. +<p>The <i>Images</i> can change during job start based on input from +the user. If you change the bgblock layout, then slurmctld and slurmd should both be cold-started (e.g. <b>/etc/init.d/slurm startclean</b>). -If you which to modify the <i>Image</i> and <i>Numpsets</i> values +If you which to modify the <i>Numpsets</i> values for existing bgblocks, either modify them manually or destroy the bgblocks and let SLURM recreate them. Note that in addition to the bgblocks defined in <i>bluegene.conf</i>, an @@ -382,10 +389,21 @@ A sample <i>bluegene.conf</i> file is shown below. ############################################################################### # Global specifications for BlueGene system # -# BlrtsImage: BlrtsImage used for creation of all bgblocks. -# LinuxImage: LinuxImage used for creation of all bgblocks. -# MloaderImage: MloaderImage used for creation of all bgblocks. -# RamDiskImage: RamDiskImage used for creation of all bgblocks. +# DefaultBlrtsImage: BlrtsImage used for creation of all bgblocks. +# DefaultLinuxImage: LinuxImage used for creation of all bgblocks. +# DefaultMloaderImage: MloaderImage used for creation of all bgblocks. +# DefaultRamDiskImage: RamDiskImage used for creation of all bgblocks. +# +# You may add extra images which a user can specify from the srun +# command line (see man srun). When adding these images you may also add +# a Groups= at the end of the image path to specify which groups can +# use the image. +# +# BlrtsImage: Alternative BlrtsImage(s). +# LinuxImage: Alternative LinuxImage(s). +# MloaderImage: Alternative MloaderImage(s). +# RamDiskImage: Alternative RamDiskImage(s). +# # LayoutMode: Mode in which slurm will create blocks: # STATIC: Use defined non-overlapping bgblocks # OVERLAP: Use defined bgblocks, which may overlap @@ -408,10 +426,32 @@ A sample <i>bluegene.conf</i> file is shown below. # NOTE: The bgl_serial value is set at configuration time using the # "--with-bgl-serial=" option. Its default value is "BGL". ############################################################################### -BlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw.rts -LinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage.elf -MloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader.rts -RamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk.elf +# These are the default images with are used if the user doesn't specify +# which image they want +DefaultBlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw.rts +DefaultLinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage.elf +DefaultMloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader.rts +DefaultRamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk.elf + +#Only group jette can use these images +BlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw2.rts Groups=jette +LinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage2.elf Groups=jette +MloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader2.rts Groups=jette +RamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk2.elf Groups=jette + +# Since no groups are specified here any user can use them +BlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw3.rts +LinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage3.elf +MloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader3.rts +RamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk3.elf + +# Another option for images would be a "You can use anything you like image" * +# This allows the user to use any image entered with no security checking +BlrtsImage=* Groups=da:adamb +LinuxImage=* Groups=da:adamb +MloaderImage=* Groups=da:adamb +RamDiskImage=* Groups=da:adamb + LayoutMode=STATIC BasePartitionNodeCnt=512 NodeCardNodeCnt=32 diff --git a/doc/man/man1/sbatch.1 b/doc/man/man1/sbatch.1 index 99821359979..1e5cf702655 100644 --- a/doc/man/man1/sbatch.1 +++ b/doc/man/man1/sbatch.1 @@ -347,6 +347,21 @@ Disables rotation of the job's requested geometry in order to fit an appropriate partition. By default the specified geometry can rotate in three dimensions. +.TP +\fB\-\-blrts\-image\fR[=]<\fIpath\fR> +Path to blrts image for bluegene block. Default if not set + +.TP +\fB\-\-linux\-image\fR[=]<\fIpath\fR> +Path to linux image for bluegene block. Default if not set + +.TP +\fB\-\-mloader\-image\fR[=]<\fIpath\fR> +Path to mloader image for bluegene block. Default if not set + +.TP +\fB\-\-ramdisk\-image\fR[=]<\fIpath\fR> +Path to ramdisk image for bluegene block. Default if not set .SH "INPUT ENVIRONMENT VARIABLES" .PP diff --git a/doc/man/man1/srun.1 b/doc/man/man1/srun.1 index 9c56ac61747..675dd6c8d9f 100644 --- a/doc/man/man1/srun.1 +++ b/doc/man/man1/srun.1 @@ -795,6 +795,14 @@ SLURM will normally allocate a TORUS if possible for a given geometry. Disables rotation of the job's requested geometry in order to fit an appropriate partition. By default the specified geometry can rotate in three dimensions. +\fB\-\-blrts\-image\fR=\fIpath\fR +Path to blrts image for bluegene block. Default if not set +\fB\-\-linux\-image\fR=\fIpath\fR +Path to linux image for bluegene block. Default if not set +\fB\-\-mloader\-image\fR=\fIpath\fR +Path to mloader image for bluegene block. Default if not set +\fB\-\-ramdisk\-image\fR=\fIpath\fR +Path to ramdisk image for bluegene block. Default if not set .PP Help options diff --git a/etc/bluegene.conf.example b/etc/bluegene.conf.example index 123b5dacae7..f3f89c1ddc4 100644 --- a/etc/bluegene.conf.example +++ b/etc/bluegene.conf.example @@ -1,10 +1,21 @@ ############################################################################### # Global specifications for BlueGene system # -# BlrtsImage: BlrtsImage used for creation of all bgblocks. -# LinuxImage: LinuxImage used for creation of all bgblocks. -# MloaderImage: MloaderImage used for creation of all bgblocks. -# RamDiskImage: RamDiskImage used for creation of all bgblocks. +# DefaultBlrtsImage: BlrtsImage used for creation of all bgblocks. +# DefaultLinuxImage: LinuxImage used for creation of all bgblocks. +# DefaultMloaderImage: MloaderImage used for creation of all bgblocks. +# DefaultRamDiskImage: RamDiskImage used for creation of all bgblocks. +# +# You may add extra images which a user can specify from the srun +# command line (see man srun). When adding these images you may also add +# a Groups= at the end of the image path to specify which groups can +# use the image. +# +# BlrtsImage: Alternative BlrtsImage(s). +# LinuxImage: Alternative LinuxImage(s). +# MloaderImage: Alternative MloaderImage(s). +# RamDiskImage: Alternative RamDiskImage(s). +# # LayoutMode: Mode in which slurm will create blocks: # STATIC: Use defined non-overlapping bgblocks # OVERLAP: Use defined bgblocks, which may overlap @@ -27,10 +38,32 @@ # NOTE: The bgl_serial value is set at configuration time using the # "--with-bgl-serial=" option. Its default value is "BGL". ############################################################################### -BlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw.rts -LinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage.elf -MloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader.rts -RamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk.elf +# These are the default images with are used if the user doesn't specify +# which image they want +DefaultBlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw.rts +DefaultLinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage.elf +DefaultMloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader.rts +DefaultRamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk.elf + +#Only group jette can use these images +BlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw2.rts Groups=jette +LinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage2.elf Groups=jette +MloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader2.rts Groups=jette +RamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk2.elf Groups=jette + +# Since no groups are specified here any user can use them +BlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw3.rts +LinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage3.elf +MloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs-mloader3.rts +RamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk3.elf + +# Another option for images would be a "You can use anything you like image" * +# This allows the user to use any image entered with no security checking +BlrtsImage=* Groups=da:adamb +LinuxImage=* Groups=da:adamb +MloaderImage=* Groups=da:adamb +RamDiskImage=* Groups=da:adamb + LayoutMode=STATIC BasePartitionNodeCnt=512 NodeCardNodeCnt=32 -- GitLab