QsNet
Overview
QsNet MPI is a version of the Argonne National
Laboratory's MPI implementation, MPICH, with
functionality added to enable it to run natively over the Quadrics
QsNet, a very high-bandwidth, ultra low-latency network interconnect.
Setup
There is no setup needed for this software package.
Usage
There are two things to take into account in using the
QsNet MPI. The first is how to compile an MPI
program against it. The second is how to run the resulting
executable.
To compile against the QsNet MPI you should use
one of the compilation wrappers included with it. These wrappers
automatically add any necessary compilation arguments to your
compiler command to include the required QsNet MPI
headers and libraries. Their syntax is as follows:
mpif77 -fc=f77_compiler -config=f77_compiler ...
mpif90 -f90=f90_compiler -config=f90_compiler ...
mpicc -cc=C_compiler -config=C_compiler ...
where f77_compiler,
f90_compiler, and C_compiler
are the names of any of the installed Fortran77, Fortran90, and C compilers
respectively. The "..." is the rest of the normal compilation line,
such as in:
mpif77 -fc=ifort -config=ifort foo.f -o foo
|
A list of available compilers can be found in the 'Compilers and
Programming Tools' section of the HPC software pages. Examples of
the above commands can be found in the Examples section of this
page.
Important Note: the Lahey compilers do not work
with the QsNet MPI at the moment. This problem is being worked on.
To run code that was compiled against the QsNet MPI you need to use
the command /usr/global/bin/qmpirun in your PBS job
submission script. Its syntax is:
/usr/global/bin/qmpirun executable_name
where executable_name is the name of your
QsNet MPI executable. It is also necessary to specify
in the node request line of your PBS script that you want to run on
nodes that have the QsNet interconnect by adding the
quadrics keyword:
#PBS -l nodes=M:quadrics:ppn=N
where M is the number of nodes to run the job
across and N is the number of processors to use
on each node. An example PBS script can be found in the Examples
section of this page. Information on PBS scripts can be found in the
Users Guide for the system intended to be used.
Examples
For a Fortran77 MPI file foo.f that is intended to
be compiled with the Portland Group Fortran77 compiler to an executable
called foo, the following command would be used:
mpif77 -fc=pgf77 -config=pgf77 foo.f -o foo
|
For a C MPI file foo.c that is intended to be
compiled with the Intel C Compiler to an executable called
foo, the following command would be used:
mpicc -cc=icc -config=icc foo.c -o foo
|
An example PBS job submission script to run one of the above
foo executables over 4 nodes that have the QsNet
interconnect using 2 processors per node for a maximum of 2 hours
would look like this:
#PBS -l nodes=4:quadrics:ppn=2
#PBS -l walltime=2:00:00
#PBS -j oe
# change the current working directory to the directory where
# the executable file 'foo' can be found
cd /home/user/mpi
# run the executable file 'foo' using the qmpirun script
/usr/global/bin/qmpirun ./foo
|
Further information on PBS scripts and submitting jobs can be found in
Users Guide for the system that the job is intended to be run on.
Documentation
More information on the QsNet MPI can be found in
/usr/lib/mpi on LION-XL.
Please send questions or suggestions about this web page to beatnic@aset.psu.edu
ASET | ITS | Penn State
|
