Understanding and managing hardware affinities with Hardware Locality (hwloc)
This tutorial will walk the audience across the complexity of modern computing servers.
We will detail why those characteristics are important to HPC application developers
and why they are difficult to manage manually.
We will then introduce the Hardware Locality software (hwloc)
which is developed to make developers' life easier by abstracting
hardware topologies in a portable way.
The tutorial is designed for HPC developers that want to optimize their application or library according to hardware
affinity. Some basic knowledge of hardware architecture may be required but many notions will be reexplained along
the examples given in the tutorial.
No knowledge of hwloc is required.
- Introduction about modern computing platform complexity
- Many levels of hierarchy (processors, cores, shared caches, NUMA, I/O, etc)
- Nothing is portable
- Examples of why hardware affinity matters in HPC
- MPI process placement according to communication patterns
- OpenMP thread placement according to nested parallel sections
- Adapting communication strategies to I/O affinities
- Consulting and managing affinities with hwloc tools
- Understanding the hardware topology
- Checking (and modifying) your tasks' placement
- Integrating hwloc help into your application
- Locating hardware ressources and consulting their attributes
- Binding tasks
Portable Hardware Locality (hwloc). https://www.open-mpi.org/projects/hwloc/.
Brice Goglin, Jeff Squyres, and Samuel Thibault. Hardware Locality: Peering under the hood of your server. Linux
Pro Magazine, 128:28-33, July 2011.
Brice Goglin and Stéphanie Moreaud. Dodging Non-Uniform I/O Access in Hierarchical Collective Operations for
Multicore Clusters. In the CASS workshop (IPDPS 2011), Anchorage, AK, May 2011. IEEE Computer Society Press.
Stéphanie Moreaud, Brice Goglin, and Raymond Namyst. Adaptive MPI Multirail Tuning for Non-Uniform Input/Output
Access. In EuroMPI 2010, volume 6305 of Lecture Notes in Computer Science, pages 239-248, Stuttgart, Germany,
September 2010. Springer-Verlag. Best paper award.
François Broquedis, Jérôme Clet-Ortega, Stéphanie Moreaud, Nathalie Furmento, Brice Goglin, Guillaume Mercier,
Samuel Thibault, and Raymond Namyst. hwloc: a Generic Framework for Managing Hardware Affinities in HPC Applications.
In PDP2010, Pisa, Italia, February 2010. IEEE Computer Society Press.
Brice Goglin (Inria Bordeaux – Sud-Ouest, France)
earned his PhD in Computer Science at Ecole normale supérieure in Lyon (France) in 2005.
He then spent one year as a software architect at Myricom, Inc.
where he developed the Myrinet Express HPC networking stack.
He is now a permanent computer scientist at Inria (Bordeaux, France)
where he works on high performance computing on multicore clusters.
He is the main developer of the Hardware Locality project (hwloc)
in collaboration with the Open MPI community.
He is also involved in the development MPI communication subsystems
such as Open-MX,
as well as in the integration of hardware affinity constraints in the Open MPI implementation.
Updated on July 12th, 2012