From 042d23a802ef555b9acc0f7b9f8b2588d08987e0 Mon Sep 17 00:00:00 2001 From: Morris Jette <jette@schedmd.com> Date: Mon, 20 Apr 2015 15:39:34 -0700 Subject: [PATCH] Update MVAPICH2 configuration info --- doc/html/mpi_guide.shtml | 37 +++++++++++++------------------------ 1 file changed, 13 insertions(+), 24 deletions(-) diff --git a/doc/html/mpi_guide.shtml b/doc/html/mpi_guide.shtml index 1e817f5ad9d..cce20654c05 100644 --- a/doc/html/mpi_guide.shtml +++ b/doc/html/mpi_guide.shtml @@ -368,39 +368,28 @@ documentation for "CQ or QP Creation failure".</p> <hr size=4 width="100%"> -<h2><a name="mvapich2" href="http://nowlab.cse.ohio-state.edu/projects/mpi-iba"><b>MVAPICH2</b></a></h2> +<h2><a name="mvapich2" href="http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.0-userguide.html"><b>MVAPICH2</b></a></h2> <p>MVAPICH2 supports launching multithreaded programs by Slurm as well as mpirun_rsh. Please note that if you intend to use use srun, you need to build MVAPICH2 -with Slurm support. -Please refer to the -<a href="http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.0-userguide.html#x1-100004.3.2"> -MVAPICH2 User Guide</a> for details. -You can also use Slurm for only resource allocation (using the salloc or sbatch -command) and launch the jobs using mpirun_rsh. -However, please note that mpirun_rsh does not accept "-env" parameter, so -the command would be something like this:</p> +with Slurm support with a command line of this sort:</p> <pre> -$ mpirun_rsh -np 2 -hostfile <path_to_hostfile> \ - MV2_USE_CUDA=1 MV2_ENABLE_AFFINITY=0 ./mpi <application> +$ ./configure --with-pmi=pmi2 --with-pm=slurm </pre> -<p>Slurm must be configured to use the <i>none</i> MPI plugin to establish -communications between the launched tasks. -This can be accomplished either using the Slurm configuration parameter -<i>MpiDefault=none</i> in <b>slurm.conf</b> or srun's <i>--mpi=none</i> option. -<b>Do not use Slurm's MVAPICH plugin for MVAPICH2.</b> -The program must also be linked with -Slurm's implementation of the PMI library so that tasks can communicate -host and port information at startup. (The system administrator can add -these option to the mpicc and mpif77 commands directly, so the user will not -need to bother). +<p>Use of Slurm's <i>pmi2</i> plugin provides substantially higher performance and +scalability than Slurm's <i>pmi</i> plugin. +If <i>pmi2</i> is not configured to be Slurm's default MPI plugin at your site, +this can be specified using the srun command's "--mpi-pmi2" option as shown +below or with the environment variable setting of "SLURM_MPI_TYPTE=mpi2".</p> <pre> -$ mpicc -L<path_to_slurm_lib> -lpmi ... -$ srun -n16 --mpi=none a.out +$ srun -n16 --mpi=pmi2 a.out </pre> +<p>For more information, please see the +<a href="http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.0-userguide.html">MVAPICH2 User Guide</a></p> + <hr size=4 width="100%"> @@ -523,6 +512,6 @@ $ srun -N4 -n16 a.out <hr size=4 width="100%"> -<p style="text-align:center;">Last modified 15 April 2015</p> +<p style="text-align:center;">Last modified 20 April 2015</p> <!--#include virtual="footer.txt"--> -- GitLab