5. Parallel applications

MPI based Fortran, C or C++ codes

The compilation of Fortran,  C or C++ codes parallelized with MPI, the form is quite similar:

mpif90 [options] file1 [file2 ...]         - for Fortran when using OpenMPI
mpicc [options] file1 [file2 ...]          - for C when using OpenMPI
mpiCC [options] file1 [file2 ...]          - For C++ when using OpenMPI

  mpif90, mpicc and mpiCC are using the Intel compilers, they are just wrappers that invokes all the necessary MPI stuff automatically for you. Therefore, everything else is the same for compiling MPI codes as for compiling plain Fortran/C/C++ codes. Please check also the section about MPI Libraries for more information about using MPI.

To see exactly what the wrappers do, you can use the "--showme" option, for example:

$ mpif90 --showme
ifort -I/global/apps/openmpi/1.3.3/include -I/global/apps/openmpi/1.3.3/lib -L/opt/torque/lib64 -Wl,--rpath -Wl,/opt/torque/lib64 -L/global/apps/openmpi/1.3.3/lib -lmpi_f90 -lmpi_f77 -lmpi -lopen-rte -lopen-pal -lrdmacm -libverbs -ltorque -ldl -lnsl -lutil

To start the MPI application:

mpirun MyApplication

 This will use all the CPUs available

OpenMP based codes

You only have to select the necessary compiler options for OpenMP. For intel compilers:

ifort -openmp MyApplication.f90
icc -openmp MyApplication.c

and for hybrid MPI/OpenMP:

mpif90 -openmp MyApplication.f90
mpicc -openmp MyApplication.c

 Launch hybrid MPI/OpenMP applications:

export OMP_NUM_THREADS=8
mpirun -np 12 -npernode 1 MyHybridApplication

OMP_NUM_THREADS=8 is the default, so it would normally not be necessary to specify it.

Note that you must ensure that you have reserved 12 nodes (96 cores) for this application:

$PBS -lnodes=12:ppn=8