Open MPI logo

FAQ:
VampirTrace Integration

  |   Home   |   Support   |   FAQ   |   all just the FAQ
This FAQ is for Open MPI v4.x and earlier.
If you are looking for documentation for Open MPI v5.x and later, please visit docs.open-mpi.org.

Table of contents:

  1. What is VampirTrace?
  2. Where can I find the complete documentation of VampirTrace?
  3. How to instrument my MPI application with VampirTrace?
  4. Does VampirTrace cause overhead to my application?
  5. How can I change the underlying compiler of the mpi*-vt wrappers?
  6. How can I pass VampirTrace related configure options through the Open MPI configure?
  7. How to disable the integrated VampirTrace, completely?


1. What is VampirTrace?

NOTE: VampirTrace was only included in Open MPI from v1.3.x through v1.10.x. It was removed in the v2.0.0 release of Open MPI. This FAQ question pertains to the versions of Open MPI that contained VampirTrace.

VampirTrace is a program tracing package that can collect a very fine grained event trace of your sequential or parallel program. The traces can be visualized by the Vampir tool and a number of other tools that read the Open Trace Format (OTF).

Tracing is interesting for performance analysis and optimization of parallel and HPC (High Performance Computing) applications in general and MPI programs in particular. In fact, that's where the letters 'mpi' in "Vampir" come from. Therefore, it is integrated into Open MPI for convenience.

VampirTrace is included in Open MPI v1.3 and later.

VampirTrace consists of two main components: First, the instrumentation part which slightly modifies the target program in order to be notified about run-time events of interest. Simply replace the compiler wrappers to activate it: mpicc to mpicc-vt, mpicxx to mpicxx-vt and so on (note that the *-vt variants of the wrapper compilers are unavailable before Open MPI v1.3). Second, the run-time measurement part is responsible for data collection. This can only be effective when the first part was performed — otherwise there will be no effect on your program at all.

VampirTrace has been developed at ZIH, TU Dresden in collaboration with the KOJAK project from JSC/FZ Juelich and is available as open source software under the BSD license; see ompi/contrib/vt/vt/COPYING.

The software is also available as a stand-alone source code package. The latest version can always be found at http://www.tu-dresden.de/zih/vampirtrace/.


2. Where can I find the complete documentation of VampirTrace?

NOTE: VampirTrace was only included in Open MPI from v1.3.x through v1.10.x. It was removed in the v2.0.0 release of Open MPI. This FAQ question pertains to the versions of Open MPI that contained VampirTrace.

A complete documentation of VampirTrace comes with the Open MPI software package as PDF and HTML. You can find it in the Open MPI source tree at ompi/contrib/vt/vt/doc/ or after installing Open MPI in $(install-prefix)/share/vampirtrace/doc/.


3. How to instrument my MPI application with VampirTrace?

NOTE: VampirTrace was only included in Open MPI from v1.3.x through v1.10.x. It was removed in the v2.0.0 release of Open MPI. This FAQ question pertains to the versions of Open MPI that contained VampirTrace.

All the necessary instrumentation of user functions as well as MPI and OpenMP events is handled by special compiler wrappers ( mpicc-vt, mpicxx-vt, mpif77-vt, mpif90-vt ). Unlike the normal wrappers ( mpicc and friends) these wrappers call VampirTrace's compiler wrappers ( vtcc, vtcxx, vtf77, vtf90 ) instead of the native compilers. The vt* wrappers use underlying platform compilers to perform the necessary instrumentation of the program and link the suitable VampirTrace library.

Original:

1
shell$ mpicc -c hello.c -o hello

With instrumentation:

1
shell$ mpicc-vt -c hello.c -o hello

For your application, simply change the compiler definitions in your Makefile(s):

1
2
3
4
5
6
7
8
9
# original definitions in Makefile
## CC=mpicc
## CXX=mpicxx
## F90=mpif90
 
# replace with
CC=mpicc-vt
CXX=mpicxx-vt
F90=mpif90-vt


4. Does VampirTrace cause overhead to my application?

NOTE: VampirTrace was only included in Open MPI from v1.3.x through v1.10.x. It was removed in the v2.0.0 release of Open MPI. This FAQ question pertains to the versions of Open MPI that contained VampirTrace.

By using the default MPI compiler wrappers ( mpicc, etc.) your application will be run without any changes at all. The VampirTrace compiler wrappers ( mpicc-vt etc.) link the VampirTrace library which intercepts MPI calls and some user level function/subroutine calls. This causes a certain amount of run-time overhead to applications. Usually, the overhead is reasonably small (0.x% - 5%) and VampirTrace by default enables precautions to avoid excessive overhead. However, it can be configured to produce very substantial overhead using non-default settings.


5. How can I change the underlying compiler of the mpi*-vt wrappers?

NOTE: VampirTrace was only included in Open MPI from v1.3.x through v1.10.x. It was removed in the v2.0.0 release of Open MPI. This FAQ question pertains to the versions of Open MPI that contained VampirTrace.

Unlike the standard MPI compiler wrappers ( mpicc etc.) the environment variables OMPI_CC, OMPI_CXX, OMPI_F77, OMPI_F90 do not affect the VampirTrace compiler wrappers. Please, use the environment variables VT_CC, VT_CXX, VT_F77, VT_F90 instead. In addition, you can set the compiler with the wrapper's option -vt:[cc|cxx|f77|f90]

The following two are equivalent, setting the underlying compiler to gcc:

1
2
shell$ VT_CC=gcc mpicc-vt -c hello.c -o hello
shell$ mpicc-vt -vt:cc gcc -c hello.c -o hello

Futhermore, you can modify the default settings in /share/openmpi/mpi*-wrapper-data.txt.


6. How can I pass VampirTrace related configure options through the Open MPI configure?

NOTE: VampirTrace was only included in Open MPI from v1.3.x through v1.10.x. It was removed in the v2.0.0 release of Open MPI. This FAQ question pertains to the versions of Open MPI that contained VampirTrace.

To give options to the VampirTrace configure script you can add this to the configure option: --with-contrib-vt-flags.

The following example passes the options --with-papi-lib-dir and --with-papi-lib to the VampirTrace configure script to specify the location and name of the PAPI library:

1
shell$ ./configure --with-contrib-vt-flags='--with-papi-lib-dir=/usr/lib64 --with-papi-lib=-lpapi64' ...


7. How to disable the integrated VampirTrace, completely?

NOTE: VampirTrace was only included in Open MPI from v1.3.x through v1.10.x. It was removed in the v2.0.0 release of Open MPI. This FAQ question pertains to the versions of Open MPI that contained VampirTrace.

By default, the VampirTrace part of Open MPI will be built and installed. If you would like to disable building and installing of VampirTrace, add the value vt to the configure option --enable-contrib-no-build.

1
shell$ ./configure --enable-contrib-no-build=vt ...