Standard Article

Parallel Programming Tools

  1. Jeffrey J. Evans

Published Online: 16 MAR 2009

DOI: 10.1002/9780470050118.ecse298

Wiley Encyclopedia of Computer Science and Engineering

Wiley Encyclopedia of Computer Science and Engineering

How to Cite

Evans, J. J. 2009. Parallel Programming Tools. Wiley Encyclopedia of Computer Science and Engineering. 1–11.

Author Information

  1. Purdue University, West Lafayette, Indiana

Publication History

  1. Published Online: 16 MAR 2009


High performance computing (HPC) hardware has evolved from (but not eliminated) vector supercomputer platforms to PC cluster and other scalable parallel systems. This architectural change is exciting because large-scale parallelism can be more easily realized and expanded as hardware component performance evolves and component costs decrease. Moreover, the notion of large scale has grown from describing hundreds to now meaning up to hundreds of thousands of processors. This evolution also implies that hardware and software interactions have grown in number and complexity, which dramatically increased debugging and performance tuning problems for parallel application developers.

Today's HPC systems are also exposing natural tensions between the operations and user domains. Complex hardware and systems software interactions are often dependent on subtle timing conditions that can be difficult and time consuming to reproduce. Add to this one or more parallel applications that simultaneously execute on a HPC system it becomes clear that locating and eliminating undesired interactions or software bugs can be extraordinarily tedious, and it may also require expertise from across domains that have different interests and objectives. In these situations, system administrators may tend to suspect the parallel application, whereas the application developer often suspects some performance degradation of the underlying system.

This article discusses key perspectives for parallel programming tools: basics, which includes a classification of tool types, challenges and opportunities for parallel programming tools, libraries, like NetLib, PetSc, and PAPI, and performance analysis environments, including Jumpshot, Pablo, Tau, and Paradyn. Because the trend over the past 5 years has been toward clusters the discussion and examples will primarily focus on the single-program–multiple-data (SPMD) parallel programming model using message passing to exchange data and provide program synchronization.


  • parallel programming tools;
  • monitoring;
  • debugging;
  • optimization