Prerequisites
This lab follows the module
Basics of MPI Programming.
You should complete
the
Tutorial on the Velocity Scheduler Batch System: Running a Parallel Job
before starting this lab. For reference on creating projects in Microsoft
Visual Studio .NET see
Compiling and Porting Lab Solution.
These prerequisites cover the following material, which is necessary for doing this
lab:
- Compiling using the MPI libraries
- Using the mpirun command
- Creating a machines file
- Understanding the mechanics of running an MPI job on two nodes
Overview
The first two exercises in this lab ask you to modify a very simple MPI program,
using only the six basic MPI calls.
The third exercise provides a simple serial program, called karp, which calculates
PI using a DO loop to calculate an integral approximation. You are asked to convert
this to a parallel program using the SPMD (Single Program Multiple Data) implementation.
For someone relatively new to message passing, these exercises will probably take
the following time to complete:
- Exercise 1: 20-30 minutes
- Exercise 2: 20-30 minutes
- Exercise 3: 45-60 minutes
To view a list of all MPI calls, with syntax and descriptions, go to any machine
with MPI/Pro installed (such as a login node), navigate to
Start | All Programs
| Verari Systems | MPIPro | MPI Library Reference
In case the
MPI documentation is not installed with MPI/Pro, see the Documentation for MPI/PRO
subroutines at
\\tc.cornell.edu\tc\CTC Tools\SoftwareDocs\MPIPro\.
Exercise
Prior to Running Your First MPI Job
Before you run your first mpi job, you must use the mpipasswd command to
register your password with mpi. This is different from what you did when
using vsched -pa before running batch jobs. Issue the mpipasswd command at
a command prompt on one of the login nodes. It will put an encrypted file .mpipass
in your home directory. Once you do this, all applications using MPI/Pro should
be ok.
Before You Begin
- Create an empty subdirectory to work in so that the files from this exercise do
not get mixed up with your other files. In our example, we call this subdirectory
"Lab". Directions are given for the command line, but you can do all of these
tasks in Windows with the GUI and Drag and Drop.
- H:
- mkdir H:\users\%USERNAME%\Lab
- cd H:\users\%USERNAME%\Lab
- Copy all lab files found in
- H:\VWlabs\MPI\Basics\
to your home directory (or subdirectory) on H:, e.g.
- copy H:\VWlabs\MPI\Basics\* H:\Users\%USERNAME%\Lab\
- Edit and compile your programs on one of the login nodes using MicroSoft Visual
Studio or the command line. For the fortran code only use Visual Studio.
Note: The fortran solution requires the statement use mpi,
rather than an include statement. Add the file C:\Program Files\MPIPro\include\mpif.f90
to the project. It is required to create the module MPI.mod that is invoked
by use mpi.
- Create a .xml file using your favorite editor with <type>batch</type>
and with the number of nodes and processors in the specific directions for each
exercise. Use <affiliation>development</affiliation> for faster turn
around. These nodes have a 20 minute time limit, but this will be more than enough
for all of your jobs.
- Submit this .xml file with vsched -s. Remember NOT to run your executables on the
login nodes!
Exercise 1: Add messages sent from workers to master
C lab file: hello.c
C solution file: hello.ex1.c
Fortran lab file: hello.f
Fortran solution file: hello.ex1.f
Hello is a SPMD (Single Program Multiple Data) program, i.e., the same program runs
as a master process and as worker processes. The program determines whether it is
the master (rank 0) or a worker (rank 1 or higher) by using an IF statement, and
then branches to the appropriate segment of the program.
The master sends a message ("Hello world") to all the workers, and then prints that
message to stdout. Each worker receives its message, then prints it to stdout.
Your mission, should you accept it, is to modify the hello program so that each
worker, instead of printing the acknowledgement, sends a message back to the master,
containing the worker's rank. The master should receive these messages and print
out the message and the corresponding ranks to stdout.
Run this program on 2 processors on one node. One-node.bat is a template for the
batch file you will need, but try to create your own first. Important:
make sure that
ROOTDIR is set to the directory in which your executable is located. Using
the instructions in this exercise, with a project name HelloC, will put the executable
in H:\Users\%USERNAME%\Lab\HelloC\Debug.
Exercise 2: Match messages using tags
C lab file: hello.ex1.c
C solution file: hello.ex2.c
Fortran lab file: hello.ex1.f
Fortran solution file: hello.ex2.f
An application can use the tag parameter in the send and receive calls to discriminate
among messages. Modify the program you created in exercise 1 so that the master
sends two messages to each worker, using two different tags. Have each worker receive
the messages in reverse order, using the tags, and then reply to the master as in
exercise 1. Again, have the master receive and print out each message and the corresponding
ranks to stdout.
Note the comment in the solution codes provided (hello.ex2.*) regarding a limitation
to this solution.
Run this program with 4 processors. Multi-node.bat (along with setup.bat and cleanup.bat)
is a template for the batch file you will need. Important: in all
the .bat files, make sure that ROOTDIR is set to the directory in which your executable
is located. Using the instructions in this exercise, with a project name HelloC,
will put the executable in H:\Users\%USERNAME%\Lab\HelloC\Debug.
Exercise 3: Convert serial code to parallel
C lab file: karp.c
C solution file: karp.soln.c
Fortran lab file: karp.f
Fortran solution file: karp.soln.f
Input data file: values
Program karp calculates PI using an integral approximation. You are provided with
a serial version of karp, and asked to modify it into a parallel version in SPMD
(Single Program, Multiple Data) form.
a. Familiarize yourself with the way the serial program works.
Copy and run the serial program, then ask yourself these questions:
- How does it calculate PI? (Hint: look at the program comments)
- How does the precision of the calculation depend on N, the number of approximation
steps? (Hint: edit values to have various input values from 10 to 10000)
- What do you think will happen to the precision with which we calculate PI when we
split up the work among the nodes?
b. Add MPI calls to create a SPMD karp program.
In this section, you will split up the work among parallel processes.
Objective
- To obtain a real, working, SPMD program, implemented in MPI.
Steps
- Edit karp to divide up work among the processes. Use only the six basic MPI calls.
Hint: the master needs to let all the workers know the total number
of iterations, and then each worker calculates its loop indices so it does its share
of the work. When done, each worker sends its partial sum back to the master, which
receives them and calculates the final sum.
- Compile it on one of the login nodes
- Run it with 4 processors and with 8 processors. Modify the batch files you
used in Exercise 2. Although the calculations are very different, the mechanism
for running the jobs in batch is very similar. You must copy the input data file
"values" to the directory in which the executable will run. If the data file
will be used on more than one node, you can add a line to the setup file to copy
it to each node.
Questions
- Did it run?
- Did it give the right answer?
- For a given N, will the calculated value of PI always be exactly the same?
- How might a "broadcast" operation, in which one task sends the same message to all
other tasks, have helped you? (MPI's broadcast operation is covered in a later talk.)
Solution
The source code with the solution for the parallel program is referenced within
each individual exercise.
Contained herein are additional notes on using Visual Studio to create solutions.
These go beyond the steps for serial programs.
- Open the project properties.
- First, add the the include path for MPIPro:
C:\Program Files\MPIPro\include
in the C\C++(or Fortran) | General | Additional Include Directories
field.
- Second, add the library directory
C:\Program Files\MPIPro\lib
to the Linker | General | Additional Library Directories field.
-
Then, go either to Code Generation under the C/C++
tab or Libraries under the Fortran tab to choose
your runtime library. Select Multithreaded libraries.
-
Last you should type
MPIPro.lib
into Linker | Input | Additional Dependencies.
- For Intel Fortran projects with MPI
there are some additional details.
- Add the file C:\Program Files\MPIPro\include\mpif.f90 to the project.
- Under Fortran | External Procedures, the four settings should
be
C,REFERENCE or Default
Lower Case(Upper Case is the default.)
After Individual String Argument or After All
Arguments
No
- Add mpipro_cdec.lib to Linker | Input | Additional Dependencies
- Add libc to Linker | Input | Ignore Specific Library
Cleanup
Delete your \Lab\ folder on H:.
