From d27e244b21b45ec614b560e7a05322e9ada05e34 Mon Sep 17 00:00:00 2001 From: Moe Jette <jette1@llnl.gov> Date: Thu, 26 Feb 2009 19:42:31 +0000 Subject: [PATCH] Move mpi_guide to its own web page. Add description of future open mpi mode fo operation. --- doc/html/Makefile.am | 1 + doc/html/Makefile.in | 1 + doc/html/documentation.shtml | 2 +- doc/html/mpi_guide.shtml | 296 +++++++++++++++++++++++++++++++++++ doc/html/quickstart.shtml | 225 ++------------------------ doc/html/review_release.html | 3 +- 6 files changed, 317 insertions(+), 211 deletions(-) create mode 100644 doc/html/mpi_guide.shtml diff --git a/doc/html/Makefile.am b/doc/html/Makefile.am index 977fd669b91..f8eed6c5d0b 100644 --- a/doc/html/Makefile.am +++ b/doc/html/Makefile.am @@ -28,6 +28,7 @@ generated_html = \ maui.html \ mc_support.html \ moab.html \ + mpi_guide.html \ mpiplugins.html \ news.html \ overview.html \ diff --git a/doc/html/Makefile.in b/doc/html/Makefile.in index decb37b77f9..b147af00fd6 100644 --- a/doc/html/Makefile.in +++ b/doc/html/Makefile.in @@ -277,6 +277,7 @@ generated_html = \ maui.html \ mc_support.html \ moab.html \ + mpi_guide.html \ mpiplugins.html \ news.html \ overview.html \ diff --git a/doc/html/documentation.shtml b/doc/html/documentation.shtml index ba75b23a036..479ba1ea420 100644 --- a/doc/html/documentation.shtml +++ b/doc/html/documentation.shtml @@ -8,7 +8,7 @@ Also see <a href="publications.html">Publications and Presentations</a>. <ul> <li><a href="quickstart.shtml">Quick Start User Guide</a></li> <li><a href="mc_support.shtml">Support for Multi-core/Multi-threaded Architectures</a></li> -<li><a href="quickstart.shtml#mpi">Guide to MPI Use</a></li> +<li><a href="mpi_guide.shtml">MPI Use Guide</a></li> <li>Specific Systems</li> <ul> <li><a href="bluegene.shtml">Blue Gene User and Administrator Guide</a></li> diff --git a/doc/html/mpi_guide.shtml b/doc/html/mpi_guide.shtml new file mode 100644 index 00000000000..0793b24e82d --- /dev/null +++ b/doc/html/mpi_guide.shtml @@ -0,0 +1,296 @@ +<!--#include virtual="header.txt"--> + +<h1>MPI Use Guide</h1> + +<p>MPI use depends upon the type of MPI being used. +There are three fundamentally different modes of operation used +by these various MPI implementation. +<ol> +<li>SLURM directly launches the tasks and performs initialization +of communications (Quadrics MPI, MPICH2, MPICH-GM, MPICH-MX, +MVAPICH, MVAPICH2 and some MPICH1 modes).</li> +<li>SLURM creates a resource allocation for the job and then +mpirun launches tasks using SLURM's infrastructure (OpenMPI, +LAM/MPI and HP-MPI).</li> +<li>SLURM creates a resource allocation for the job and then +mpirun launches tasks using some mechanism other than SLURM, +such as SSH or RSH (BlueGene MPI and some MPICH1 modes). +These tasks initiated outside of SLURM's monitoring +or control. SLURM's epilog should be configured to purge +these tasks when the job's allocation is relinquished. </li> +</ol> +<p>Links to instructions for using several varieties of MPI +with SLURM are provided below. +<ul> +<li><a href="#bluegene_mpi">BlueGene MPI</a></li> +<li><a href="#hp_mpi">HP-MPI</a></li> +<li><a href="#lam_mpi">LAM/MPI</a></li> +<li><a href="#mpich1">MPICH1</a></li> +<li><a href="#mpich2">MPICH2</a></li> +<li><a href="#mpich_gm">MPICH-GM</a></li> +<li><a href="#mpich_mx">MPICH-MX</a></li> +<li><a href="#mvapich">MVAPICH</a></li> +<li><a href="#mvapich2">MVAPICH2</a></li> +<li><a href="#open_mpi">Open MPI</a></li> +<li><a href="#quadrics_mpi">Quadrics MPI</a></li> +</ul></p> +<hr size=4 width="100%"> + + +<h2><a name="open_mpi" href="http://www.open-mpi.org/"><b>Open MPI</b></a></h2> + +<p>Open MPI relies upon +SLURM to allocate resources for the job and then mpirun to initiate the +tasks. When using <span class="commandline">salloc</span> command, +<span class="commandline">mpirun</span>'s -nolocal option is recommended. +For example: +<pre> +$ salloc -n4 sh # allocates 4 processors + # and spawns shell for job +> mpirun -np 4 -nolocal a.out +> exit # exits shell spawned by + # initial srun command +</pre> +<p>Note that any direct use of <span class="commandline">srun</span> +will only launch one task per node when the LAM/MPI plugin is used. +To launch more than one task per node using the +<span class="commandline">srun</span> command, the <i>--mpi=none</i> +option will be required to explicitly disable the LAM/MPI plugin.</p> + +<h2>Future Use</h2> +There is work underway in both SLURM and Open MPI to support task launch +using the <span class="commandline">srun</span> command. +We expect this mode of operation to be supported late in 2009. +It may differ slightly from the description below. +It relies upon SLURM version 2.0 (or higher) managing +reservations of communication ports for the Open MPI's use. +Specify the range of ports to be reserved in the <i>slurm.conf</i> +file using the <i>MpiParams</i> parameter. +For example: <i>MpiParams=ports:12000-12999</i>. +Then launch tasks using the <span class="commandline">srun</span> command +plus the option <i>--resv-ports</i>. +The ports reserved on every allocated node will be identified in an +environment variable as shown here: +<i>SLURM_STEP_RESV_PORTS=12000-12015</i></p> +<hr size=4 width="100%"> + + +<h2><a name="quadrics_mpi" href="http://www.quadrics.com/"><b>Quadrics MPI</b></a></h2> + +<p>Quadrics MPI relies upon SLURM to +allocate resources for the job and <span class="commandline">srun</span> +to initiate the tasks. One would build the MPI program in the normal manner +then initiate it using a command line of this sort:</p> +<pre> +$ srun [options] <program> [program args] +</pre> +<hr size=4 width="100%"> + + +<h2><a name="lam_mpi" href="http://www.lam-mpi.org/"><b>LAM/MPI</b></a></h2> + +<p>LAM/MPI relies upon the SLURM +<span class="commandline">salloc</span> or <span class="commandline">sbatch</span> +command to allocate. In either case, specify +the maximum number of tasks required for the job. Then execute the +<span class="commandline">lamboot</span> command to start lamd daemons. +<span class="commandline">lamboot</span> utilizes SLURM's +<span class="commandline">srun</span> command to launch these daemons. +Do not directly execute the <span class="commandline">srun</span> command +to launch LAM/MPI tasks. For example: +<pre> +$ salloc -n16 sh # allocates 16 processors + # and spawns shell for job +> lamboot +> mpirun -np 16 foo args +1234 foo running on adev0 (o) +2345 foo running on adev1 +etc. +> lamclean +> lamhalt +> exit # exits shell spawned by + # initial srun command +</pre> +<p>Note that any direct use of <span class="commandline">srun</span> +will only launch one task per node when the LAM/MPI plugin is configured +as the default plugin. To launch more than one task per node using the +<span class="commandline">srun</span> command, the <i>--mpi=none</i> +option would be required to explicitly disable the LAM/MPI plugin +if that is the system default.</p> +<hr size=4 width="100%"> + + +<h2><a name="hp_mpi" href="http://www.hp.com/go/mpi"><b>HP-MPI</b></a></h2> + +<p>HP-MPI uses the +<span class="commandline">mpirun</span> command with the <b>-srun</b> +option to launch jobs. For example: +<pre> +$MPI_ROOT/bin/mpirun -TCP -srun -N8 ./a.out +</pre></p> +<hr size=4 width="100%"> + + +<h2><a name="mpich2" href="http://www.mcs.anl.gov/research/projects/mpich2/"><b>MPICH2</b></a></h2> + +<p>MPICH2 jobs are launched using the <b>srun</b> command. Just link your program with +SLURM's implementation of the PMI library so that tasks can communicate +host and port information at startup. (The system administrator can add +these option to the mpicc and mpif77 commands directly, so the user will not +need to bother). For example: +<pre> +$ mpicc -L<path_to_slurm_lib> -lpmi ... +$ srun -n20 a.out +</pre> +<b>NOTES:</b> +<ul> +<li>Some MPICH2 functions are not currently supported by the PMI +library integrated with SLURM</li> +<li>Set the environment variable <b>PMI_DEBUG</b> to a numeric value +of 1 or higher for the PMI library to print debugging information</li> +</ul></p> +<hr size=4 width="100%"> + + +<h2><a name="mpich_gm" href="http://www.myri.com/scs/download-mpichgm.html"><b>MPICH-GM</b></a></h2> + +<p>MPICH-GM jobs can be launched directly by <b>srun</b> command. +SLURM's <i>mpichgm</i> MPI plugin must be used to establish communications +between the launched tasks. This can be accomplished either using the SLURM +configuration parameter <i>MpiDefault=mpichgm</i> in <b>slurm.conf</b> +or srun's <i>--mpi=mpichgm</i> option. +<pre> +$ mpicc ... +$ srun -n16 --mpi=mpichgm a.out +</pre> +<hr size=4 width="100%"> + + +<h2><a name="mpich_mx" href="http://www.myri.com/scs/download-mpichmx.html"><b>MPICH-MX</b></a></h2> + +<p>MPICH-MX jobs can be launched directly by <b>srun</b> command. +SLURM's <i>mpichmx</i> MPI plugin must be used to establish communications +between the launched tasks. This can be accomplished either using the SLURM +configuration parameter <i>MpiDefault=mpichmx</i> in <b>slurm.conf</b> +or srun's <i>--mpi=mpichmx</i> option. +<pre> +$ mpicc ... +$ srun -n16 --mpi=mpichmx a.out +</pre> +<hr size=4 width="100%"> + + +<h2><a name="mvapich" href="http://mvapich.cse.ohio-state.edu/"><b>MVAPICH</b></a></h2> + +<p>MVAPICH jobs can be launched directly by <b>srun</b> command. +SLURM's <i>mvapich</i> MPI plugin must be used to establish communications +between the launched tasks. This can be accomplished either using the SLURM +configuration parameter <i>MpiDefault=mvapich</i> in <b>slurm.conf</b> +or srun's <i>--mpi=mvapich</i> option. +<pre> +$ mpicc ... +$ srun -n16 --mpi=mvapich a.out +</pre> +<b>NOTE:</b> If MVAPICH is used in the shared memory model, with all tasks +running on a single node, then use the <i>mpich1_shmem</i> MPI plugin instead.<br> +<b>NOTE (for system administrators):</b> Configure +<i>PropagateResourceLimitsExcept=MEMLOCK</i> in <b>slurm.conf</b> and +start the <i>slurmd</i> daemons with an unlimited locked memory limit. +For more details, see +<a href="http://mvapich.cse.ohio-state.edu/support/mvapich_user_guide.html#x1-420007.2.3">MVAPICH</a> +documentation for "CQ or QP Creation failure".</p> +<hr size=4 width="100%"> + + +<h2><a name="mvapich2" href="http://nowlab.cse.ohio-state.edu/projects/mpi-iba"><b>MVAPICH2</b></a></h2> + +<p>MVAPICH2 jobs can be launched directly by <b>srun</b> command. +SLURM's <i>none</i> MPI plugin must be used to establish communications +between the launched tasks. This can be accomplished either using the SLURM +configuration parameter <i>MpiDefault=none</i> in <b>slurm.conf</b> +or srun's <i>--mpi=none</i> option. The program must also be linked with +SLURM's implementation of the PMI library so that tasks can communicate +host and port information at startup. (The system administrator can add +these option to the mpicc and mpif77 commands directly, so the user will not +need to bother). <b>Do not use SLURM's MVAPICH plugin for MVAPICH2.</b> +<pre> +$ mpicc -L<path_to_slurm_lib> -lpmi ... +$ srun -n16 --mpi=none a.out +</pre> +<hr size=4 width="100%"> + + +<h2><a name="bluegene_mpi" href="http://www.research.ibm.com/bluegene/"><b>BlueGene MPI</b></a></h2> + +<p>BlueGene MPI relies upon SLURM to create the resource allocation and then +uses the native <span class="commandline">mpirun</span> command to launch tasks. +Build a job script containing one or more invocations of the +<span class="commandline">mpirun</span> command. Then submit +the script to SLURM using <span class="commandline">sbatch</span>. +For example:</p> +<pre> +$ sbatch -N512 my.script +</pre> +<p>Note that the node count specified with the <i>-N</i> option indicates +the base partition count. +See <a href="bluegene.html">BlueGene User and Administrator Guide</a> +for more information.</p> +<hr size=4 width="100%"> + + +<h2><a name="mpich1" href="http://www-unix.mcs.anl.gov/mpi/mpich1/"><b>MPICH1</b></a></h2> + +<p>MPICH1 development ceased in 2005. It is recommended that you convert to +MPICH2 or some other MPI implementation. +If you still want to use MPICH1, note that it has several different +programming models. If you are using the shared memory model +(<i>DEFAULT_DEVICE=ch_shmem</i> in the mpirun script), then initiate +the tasks using the <span class="commandline">srun</span> command +with the <i>--mpi=mpich1_shmem</i> option.</p> +<pre> +$ srun -n16 --mpi=mpich1_shmem a.out +</pre> + +<p>If you are using MPICH P4 (<i>DEFAULT_DEVICE=ch_p4</i> in +the mpirun script) and SLURM version 1.2.11 or newer, +then it is recommended that you apply the patch in the SLURM +distribution's file <i>contribs/mpich1.slurm.patch</i>. +Follow directions within the file to rebuild MPICH. +Applications must be relinked with the new library. +Initiate tasks using the +<span class="commandline">srun</span> command with the +<i>--mpi=mpich1_p4</i> option.</p> +<pre> +$ srun -n16 --mpi=mpich1_p4 a.out +</pre> +<p>Note that SLURM launches one task per node and the MPICH +library linked within your applications launches the other +tasks with shared memory used for communications between them. +The only real anomaly is that all output from all spawned tasks +on a node appear to SLURM as coming from the one task that it +launched. If the srun --label option is used, the task ID labels +will be misleading.</p> + +<p>Other MPICH1 programming models current rely upon the SLURM +<span class="commandline">salloc</span> or +<span class="commandline">sbatch</span> command to allocate resources. +In either case, specify the maximum number of tasks required for the job. +You may then need to build a list of hosts to be used and use that +as an argument to the mpirun command. +For example: +<pre> +$ cat mpich.sh +#!/bin/bash +srun hostname -s | sort -u >slurm.hosts +mpirun [options] -machinefile slurm.hosts a.out +rm -f slurm.hosts +$ sbatch -n16 mpich.sh +sbatch: Submitted batch job 1234 +</pre> +<p>Note that in this example, mpirun uses the rsh command to launch +tasks. These tasks are not managed by SLURM since they are launched +outside of its control.</p> + +<p style="text-align:center;">Last modified 26 February 2009</p> + +<!--#include virtual="footer.txt"--> diff --git a/doc/html/quickstart.shtml b/doc/html/quickstart.shtml index e0bbdc28b12..a09677b3e70 100644 --- a/doc/html/quickstart.shtml +++ b/doc/html/quickstart.shtml @@ -346,11 +346,11 @@ adev0: scancel 473 adev0: squeue JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) </pre> -<p class="footer"><a href="#top">top</a></p> - +<p class="footer"><a href="#top">top</a></p> <h2><a name="mpi">MPI</a></h2> + <p>MPI use depends upon the type of MPI being used. There are three fundamentally different modes of operation used by these various MPI implementation. @@ -368,215 +368,22 @@ These tasks initiated outside of SLURM's monitoring or control. SLURM's epilog should be configured to purge these tasks when the job's allocation is relinquished. </li> </ol> -<p>Instructions for using several varieties of MPI with SLURM are -provided below.</p> - -<p> <a href="http://www.open-mpi.org/"><b>Open MPI</b></a> relies upon -SLURM to allocate resources for the job and then mpirun to initiate the -tasks. When using <span class="commandline">salloc</span> command, -<span class="commandline">mpirun</span>'s -nolocal option is recommended. -For example: -<pre> -$ salloc -n4 sh # allocates 4 processors - # and spawns shell for job -> mpirun -np 4 -nolocal a.out -> exit # exits shell spawned by - # initial srun command -</pre> -<p>Note that any direct use of <span class="commandline">srun</span> -will only launch one task per node when the LAM/MPI plugin is used. -To launch more than one task per node using the -<span class="commandline">srun</span> command, the <i>--mpi=none</i> -option will be required to explicitly disable the LAM/MPI plugin.</p> - -<p> <a href="http://www.quadrics.com/"><b>Quadrics MPI</b></a> relies upon SLURM to -allocate resources for the job and <span class="commandline">srun</span> -to initiate the tasks. One would build the MPI program in the normal manner -then initiate it using a command line of this sort:</p> -<pre> -$ srun [options] <program> [program args] -</pre> - -<p> <a href="http://www.lam-mpi.org/"><b>LAM/MPI</b></a> relies upon the SLURM -<span class="commandline">salloc</span> or <span class="commandline">sbatch</span> -command to allocate. In either case, specify -the maximum number of tasks required for the job. Then execute the -<span class="commandline">lamboot</span> command to start lamd daemons. -<span class="commandline">lamboot</span> utilizes SLURM's -<span class="commandline">srun</span> command to launch these daemons. -Do not directly execute the <span class="commandline">srun</span> command -to launch LAM/MPI tasks. For example: -<pre> -$ salloc -n16 sh # allocates 16 processors - # and spawns shell for job -> lamboot -> mpirun -np 16 foo args -1234 foo running on adev0 (o) -2345 foo running on adev1 -etc. -> lamclean -> lamhalt -> exit # exits shell spawned by - # initial srun command -</pre> -<p>Note that any direct use of <span class="commandline">srun</span> -will only launch one task per node when the LAM/MPI plugin is configured -as the default plugin. To launch more than one task per node using the -<span class="commandline">srun</span> command, the <i>--mpi=none</i> -option would be required to explicitly disable the LAM/MPI plugin -if that is the system default.</p> - -<p class="footer"><a href="#top">top</a></p> - -<p><a href="http://www.hp.com/go/mpi"><b>HP-MPI</b></a> uses the -<span class="commandline">mpirun</span> command with the <b>-srun</b> -option to launch jobs. For example: -<pre> -$MPI_ROOT/bin/mpirun -TCP -srun -N8 ./a.out -</pre></p> - -<p><a href="http://www.mcs.anl.gov/research/projects/mpich2/"><b> -MPICH2</b></a> jobs -are launched using the <b>srun</b> command. Just link your program with -SLURM's implementation of the PMI library so that tasks can communicate -host and port information at startup. (The system administrator can add -these option to the mpicc and mpif77 commands directly, so the user will not -need to bother). For example: -<pre> -$ mpicc -L<path_to_slurm_lib> -lpmi ... -$ srun -n20 a.out -</pre> -<b>NOTES:</b> +<p>Links to instructions for using several varieties of MPI +with SLURM are provided below. <ul> -<li>Some MPICH2 functions are not currently supported by the PMI -library integrated with SLURM</li> -<li>Set the environment variable <b>PMI_DEBUG</b> to a numeric value -of 1 or higher for the PMI library to print debugging information</li> +<li><a href="mpi_guide.shtml#bluegene_mpi">BlueGene MPI</a></li> +<li><a href="mpi_guide.shtml#hp_mpi">HP-MPI</a></li> +<li><a href="mpi_guide.shtml#lam_mpi">LAM/MPI</a></li> +<li><a href="mpi_guide.shtml#mpich1">MPICH1</a></li> +<li><a href="mpi_guide.shtml#mpich2">MPICH2</a></li> +<li><a href="mpi_guide.shtml#mpich_gm">MPICH-GM</a></li> +<li><a href="mpi_guide.shtml#mpich_mx">MPICH-MX</a></li> +<li><a href="mpi_guide.shtml#mvapich">MVAPICH</a></li> +<li><a href="mpi_guide.shtml#mvapich2">MVAPICH2</a></li> +<li><a href="mpi_guide.shtml#open_mpi">Open MPI</a></li> +<li><a href="mpi_guide.shtml#quadrics_mpi">Quadrics MPI</a></li> </ul></p> -<p><a href="http://www.myri.com/scs/download-mpichgm.html"><b>MPICH-GM</b></a> -jobs can be launched directly by <b>srun</b> command. -SLURM's <i>mpichgm</i> MPI plugin must be used to establish communications -between the launched tasks. This can be accomplished either using the SLURM -configuration parameter <i>MpiDefault=mpichgm</i> in <b>slurm.conf</b> -or srun's <i>--mpi=mpichgm</i> option. -<pre> -$ mpicc ... -$ srun -n16 --mpi=mpichgm a.out -</pre> - -<p><a href="http://www.myri.com/scs/download-mpichmx.html"><b>MPICH-MX</b></a> -jobs can be launched directly by <b>srun</b> command. -SLURM's <i>mpichmx</i> MPI plugin must be used to establish communications -between the launched tasks. This can be accomplished either using the SLURM -configuration parameter <i>MpiDefault=mpichmx</i> in <b>slurm.conf</b> -or srun's <i>--mpi=mpichmx</i> option. -<pre> -$ mpicc ... -$ srun -n16 --mpi=mpichmx a.out -</pre> - -<p><a href="http://mvapich.cse.ohio-state.edu/"><b>MVAPICH</b></a> -jobs can be launched directly by <b>srun</b> command. -SLURM's <i>mvapich</i> MPI plugin must be used to establish communications -between the launched tasks. This can be accomplished either using the SLURM -configuration parameter <i>MpiDefault=mvapich</i> in <b>slurm.conf</b> -or srun's <i>--mpi=mvapich</i> option. -<pre> -$ mpicc ... -$ srun -n16 --mpi=mvapich a.out -</pre> -<b>NOTE:</b> If MVAPICH is used in the shared memory model, with all tasks -running on a single node, then use the <i>mpich1_shmem</i> MPI plugin instead.<br> -<b>NOTE (for system administrators):</b> Configure -<i>PropagateResourceLimitsExcept=MEMLOCK</i> in <b>slurm.conf</b> and -start the <i>slurmd</i> daemons with an unlimited locked memory limit. -For more details, see -<a href="http://mvapich.cse.ohio-state.edu/support/mvapich_user_guide.html#x1-420007.2.3">MVAPICH</a> -documentation for "CQ or QP Creation failure".</p> - -<p><a href="http://nowlab.cse.ohio-state.edu/projects/mpi-iba"><b>MVAPICH2</b></a> -jobs can be launched directly by <b>srun</b> command. -SLURM's <i>none</i> MPI plugin must be used to establish communications -between the launched tasks. This can be accomplished either using the SLURM -configuration parameter <i>MpiDefault=none</i> in <b>slurm.conf</b> -or srun's <i>--mpi=none</i> option. The program must also be linked with -SLURM's implementation of the PMI library so that tasks can communicate -host and port information at startup. (The system administrator can add -these option to the mpicc and mpif77 commands directly, so the user will not -need to bother). <b>Do not use SLURM's MVAPICH plugin for MVAPICH2.</b> -<pre> -$ mpicc -L<path_to_slurm_lib> -lpmi ... -$ srun -n16 --mpi=none a.out -</pre> - -<p><a href="http://www.research.ibm.com/bluegene/"><b>BlueGene MPI</b></a> relies -upon SLURM to create the resource allocation and then uses the native -<span class="commandline">mpirun</span> command to launch tasks. -Build a job script containing one or more invocations of the -<span class="commandline">mpirun</span> command. Then submit -the script to SLURM using <span class="commandline">sbatch</span>. -For example:</p> -<pre> -$ sbatch -N512 my.script -</pre> -<p>Note that the node count specified with the <i>-N</i> option indicates -the base partition count. -See <a href="bluegene.html">BlueGene User and Administrator Guide</a> -for more information.</p> - -<p><a href="http://www-unix.mcs.anl.gov/mpi/mpich1/"><b>MPICH1</b></a> -development ceased in 2005. It is recommended that you convert to -MPICH2 or some other MPI implementation. -If you still want to use MPICH1, note that it has several different -programming models. If you are using the shared memory model -(<i>DEFAULT_DEVICE=ch_shmem</i> in the mpirun script), then initiate -the tasks using the <span class="commandline">srun</span> command -with the <i>--mpi=mpich1_shmem</i> option.</p> -<pre> -$ srun -n16 --mpi=mpich1_shmem a.out -</pre> - -<p>If you are using MPICH P4 (<i>DEFAULT_DEVICE=ch_p4</i> in -the mpirun script) and SLURM version 1.2.11 or newer, -then it is recommended that you apply the patch in the SLURM -distribution's file <i>contribs/mpich1.slurm.patch</i>. -Follow directions within the file to rebuild MPICH. -Applications must be relinked with the new library. -Initiate tasks using the -<span class="commandline">srun</span> command with the -<i>--mpi=mpich1_p4</i> option.</p> -<pre> -$ srun -n16 --mpi=mpich1_p4 a.out -</pre> -<p>Note that SLURM launches one task per node and the MPICH -library linked within your applications launches the other -tasks with shared memory used for communications between them. -The only real anomaly is that all output from all spawned tasks -on a node appear to SLURM as coming from the one task that it -launched. If the srun --label option is used, the task ID labels -will be misleading.</p> - -<p>Other MPICH1 programming models current rely upon the SLURM -<span class="commandline">salloc</span> or -<span class="commandline">sbatch</span> command to allocate resources. -In either case, specify the maximum number of tasks required for the job. -You may then need to build a list of hosts to be used and use that -as an argument to the mpirun command. -For example: -<pre> -$ cat mpich.sh -#!/bin/bash -srun hostname -s | sort -u >slurm.hosts -mpirun [options] -machinefile slurm.hosts a.out -rm -f slurm.hosts -$ sbatch -n16 mpich.sh -sbatch: Submitted batch job 1234 -</pre> -<p>Note that in this example, mpirun uses the rsh command to launch -tasks. These tasks are not managed by SLURM since they are launched -outside of its control.</p> - -<p style="text-align:center;">Last modified 16 July 2008</p> +<p style="text-align:center;">Last modified 26 February 2009</p> <!--#include virtual="footer.txt"--> diff --git a/doc/html/review_release.html b/doc/html/review_release.html index 3f51ded643f..4a2922bfc6f 100644 --- a/doc/html/review_release.html +++ b/doc/html/review_release.html @@ -2,7 +2,7 @@ <head> <title>SLURM Web pages for Review and Release</title> -<!-- Updated 25 February 2009 --> +<!-- Updated 26 February 2009 --> </head> <body> @@ -34,6 +34,7 @@ <li><a href="https://computing-pre.llnl.gov/linux/slurm/maui.html">maui.html</a></li> <li><a href="https://computing-pre.llnl.gov/linux/slurm/mc_support.html">mc_support.html</a></li> <li><a href="https://computing-pre.llnl.gov/linux/slurm/moab.html">moab.html</a></li> +<li><a href="https://computing-pre.llnl.gov/linux/slurm/mpi_guide.html">mpi_guide.html</a></li> <li><a href="https://computing-pre.llnl.gov/linux/slurm/mpiplugins.html">mpiplugins.html</a></li> <li><a href="https://computing-pre.llnl.gov/linux/slurm/news.html">news.html</a></li> <li><a href="https://computing-pre.llnl.gov/linux/slurm/overview.html">overview.html</a></li> -- GitLab