ARSC HPC Users' Newsletter 384, April 4, 2008

The -mpitrace Option on Midnight

[By: Oralee Nudson]

The MPI compilers on midnight include the "-mpitrace" flag. This can be a quick and easy option for debugging MPI applications. Adding this flag to program compilation directs each MPI task to print an announcement to stdout every time an MPI call is made. Reviewing this "trace" of MPI calls may be helpful in determining the cause of a program deadlock, segmentation fault, or unexplainable errors in program output.

Here's an example compile statement:

mg56 % mpicc mpi_example.c -mpitrace -o mpi_example

The following output gives an example of this "verbose" processor output:

mt350 % mpirun -np 2 ./mpi_example
Starting MPI_Init...
[0] Ending MPI_Init
[1] Ending MPI_Init
[1] Starting MPI_Comm_size...
[1] Ending MPI_Comm_size
[1] Starting MPI_Comm_rank...
[1] Ending MPI_Comm_rank
[1] Starting MPI_Alltoall..
[0] Starting MPI_Comm_size...
[0] Ending MPI_Comm_size
[0] Starting MPI_Comm_rank...
[0] Ending MPI_Comm_rank
[0] Starting MPI_Alltoall...
[1] Ending MPI_Alltoall
[0] Ending MPI_Alltoall
[1] Starting MPI_Finalize...
[0] Starting MPI_Finalize...
[0] Ending MPI_Finalize
[1] Ending MPI_Finalize
Cleaning up all processes ...

The "-mpitrace" option is available for all MPI compilers on midnight (i.e. C, C++ and Fortran).

PathScale Compiler Environment Variables

The PathScale compilers are aware of several environment variables that are used at compile time. These environment variables allow you to pass flags to a compile, and include:

  • PSC_CFLAGS : used by pathcc during compilation and linking
  • PSC_CXXFLAGS : used by pathCC during compilation and linking
  • PSC_FFLAGS : used by pathf90 and pathf95 during compilation and linking
  • PSC_GENFLAGS : used by pathcc, pathCC, path90, and pathf95 during compilation and linking

Here's an example which will tell the compiler to generate C intermediate code:

mg56 % export PSC_CFLAGS="-CLIST:=ON:show=ON:emit_pfetch=ON"
mg56 % mpicc hello.c -o hello.exe
/import/local/pkg/pathscale/pathscale-3.0/lib/3.0/be translates ...

Since these environment variables are interpreted by the compiler they can be convenient when a configure script isn't doing what you expect, or when you don't want to change a Makefile to try different options.

If you have a code which uses more than one programming language, you can set the PSC_GENFLAGS variable to ensure each compiler gets the same options (e.g. include and library directories).

ARSC Technology Panel Talks

There will be two talks by members of the ARSC Technology Panel next week.

Presenter: Becky Verastegui, CIO Oak Ridge National Laboratory
Title: Building a World Class IT Infrastructure to enhance R&D at Oak Ridge NationalLaboratory
Time: Friday, April 11th, 2008 -- 1:00 PM
Location: West Ridge Research Building Room 010
Abstract: Over the last 12 months IT at ORNL has rapidly deployed many new technologies to enhance thework of the staff at ORNL. These technologies include: portals, blogs, Instant Messaging, VoiceOver IP, desktop video conferencing, 3GB inboxes, executive dashboards, expertise locators, visualsearch engines, mapping software, and a lot of chaos. The presentation will give an overview of theIT modernization initiative at ORNL. It will also include live demonstrations of how the integratedinformation technology is being used to transform the R&D at ORNL.
Presenter: Keith Bergeron, USAF Academy Instructor and ARSC User
Time: Friday, April 11th 2008 -- 1:45 PM
Location: West Ridge Research Building Room 010
Title: Modeling and Simulation Research at the U.S. Air Force Academy
Abstract: The USAF Academy's (USAFA) Modeling and Simulation Research Center (M&SRC) provides thecomputational foundation in expertise, equipment, and personnel to facilitate M&S and HighPerformance Computing (HPC) research and education. Researchers bring M&S tools into theclassroom and laboratories for students to learn the capabilities, and limitations, of moderncomputational tools. Working with the DoD HPC Modernization Program, the M&SRC enriches cadetexperience and improves support of the Air Force Research Laboratory and other USAF organizationsin completing their missions. Descriptions of how M&S is performed at the Academy are included,as well as details about the computational resources being used. In addition, descriptions of themethods used to bring these tools to Department of Aeronautics classes are described indetail.

Quick-Tip Q & A

A:[[ Here's one proposed a long time ago by friend and former 
  [[ co-editor, Guy Robinson.
  [[ There are some interesting collective nouns in the English 
  [[ language.
  [[ E.g., a "pride" of lions, a "knot" of toads.
  [[ The question is, what collectives would you propose for terms in 
  [[ the supercomputing vernacular?  For instance, we want collectives 
  [[ for terms such as cores, nodes, infinite loops, bugs, compiler 
  [[ options, defunct HPC vendors, ARSC consultants, irrelevent 
  [[ benchmarks, etc.,
  [[ etc.  Where there is a known collective, maybe you can improve 
  [[ on it.  This should be interesting... maybe even fun.

# Here's a response from Derek Bastille

   A cluster of cores
   A stack of nodes
   A tangle of infinite loops
   A gaggle of consultants
   A swarm of bugs
   A (hungry) pack of HPC vendors
   A morass of compiler options
   A brace of compilers
   A thicket of libraries
   An assemblage of engineers
   A camp of users
   A bestiary of benchmarks

# Thanks to Rick Roberts for this response.
   suggestion             what it actually refers to
   ====================   ===========================================
   Clowder of cores       clowder is a set of cats
   Nodule of nodes
   Murder of Crays        set of HPC vendors; murder is a set of crows
   YiMP                   set of defunct HPC vendors
   Scalin-marks           a set of irrelevent benchmarks
   Do-Mobius              infinite loop
   Blasphemous Bill       cost of new system
   Hypo-thermiants        ARSC consultants
   ice-worms              ARSC consultants (sorry, I couldn't help myself!)
   Guy de Part            fondly remembered ARSC consultants.

# Here's are several suggestions from Lee Higbie.

   A blyght of programmers.

   Maybe we also have flops of supercomputers?

   And, of course, it's the blyght of programmers using the flop of
   supercomputers that HPCMP has.

Q: I just generated a bunch of image files that are sequentially named:
   image1.png, image2.png, ..., image100.png.  Unfortunately when I 
   do an "ls" on the directory, the files aren't sorted the way I 
   would like.

    mg56 % ls -1
   How can rename the files with padded zeros so the files show up
   in numerical order?  For example, I would like the file image1.png
   to become image001.png and image33.jpg to become image033.png, etc.
   Show me how to do this shell gurus!

Answers, Questions, and Tips Graciously Accepted

Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top