ARSC HPC Users' Newsletter 384, April 4, 2008
The -mpitrace Option on Midnight
[By: Oralee Nudson]
The MPI compilers on midnight include the "-mpitrace" flag. This can be a quick and easy option for debugging MPI applications. Adding this flag to program compilation directs each MPI task to print an announcement to stdout every time an MPI call is made. Reviewing this "trace" of MPI calls may be helpful in determining the cause of a program deadlock, segmentation fault, or unexplainable errors in program output.
Here's an example compile statement:
mg56 % mpicc mpi_example.c -mpitrace -o mpi_example
The following output gives an example of this "verbose" processor output:
mt350 % mpirun -np 2 ./mpi_example Starting MPI_Init...  Ending MPI_Init  Ending MPI_Init  Starting MPI_Comm_size...  Ending MPI_Comm_size  Starting MPI_Comm_rank...  Ending MPI_Comm_rank  Starting MPI_Alltoall..  Starting MPI_Comm_size...  Ending MPI_Comm_size  Starting MPI_Comm_rank...  Ending MPI_Comm_rank  Starting MPI_Alltoall...  Ending MPI_Alltoall  Ending MPI_Alltoall  Starting MPI_Finalize...  Starting MPI_Finalize...  Ending MPI_Finalize  Ending MPI_Finalize Cleaning up all processes ... done.
The "-mpitrace" option is available for all MPI compilers on midnight (i.e. C, C++ and Fortran).
PathScale Compiler Environment Variables
The PathScale compilers are aware of several environment variables that are used at compile time. These environment variables allow you to pass flags to a compile, and include:
PSC_CFLAGS: used by pathcc during compilation and linking
PSC_CXXFLAGS: used by pathCC during compilation and linking
PSC_FFLAGS: used by pathf90 and pathf95 during compilation and linking
PSC_GENFLAGS: used by pathcc, pathCC, path90, and pathf95 during compilation and linking
Here's an example which will tell the compiler to generate C intermediate code:
mg56 % export PSC_CFLAGS="-CLIST:=ON:show=ON:emit_pfetch=ON" mg56 % mpicc hello.c -o hello.exe /import/local/pkg/pathscale/pathscale-3.0/lib/3.0/be translates ...
Since these environment variables are interpreted by the compiler they can be convenient when a configure script isn't doing what you expect, or when you don't want to change a Makefile to try different options.
If you have a code which uses more than one programming language, you can set the PSC_GENFLAGS variable to ensure each compiler gets the same options (e.g. include and library directories).
ARSC Technology Panel Talks
There will be two talks by members of the ARSC Technology Panel next week.
|Presenter:||Becky Verastegui, CIO Oak Ridge National Laboratory|
|Title:||Building a World Class IT Infrastructure to enhance R&D at Oak Ridge NationalLaboratory|
|Time:||Friday, April 11th, 2008 -- 1:00 PM|
|Location:||West Ridge Research Building Room 010|
|Abstract:||Over the last 12 months IT at ORNL has rapidly deployed many new technologies to enhance thework of the staff at ORNL. These technologies include: portals, blogs, Instant Messaging, VoiceOver IP, desktop video conferencing, 3GB inboxes, executive dashboards, expertise locators, visualsearch engines, mapping software, and a lot of chaos. The presentation will give an overview of theIT modernization initiative at ORNL. It will also include live demonstrations of how the integratedinformation technology is being used to transform the R&D at ORNL.|
|Presenter:||Keith Bergeron, USAF Academy Instructor and ARSC User|
|Time:||Friday, April 11th 2008 -- 1:45 PM|
|Location:||West Ridge Research Building Room 010|
|Title:||Modeling and Simulation Research at the U.S. Air Force Academy|
|Abstract:||The USAF Academy's (USAFA) Modeling and Simulation Research Center (M&SRC) provides thecomputational foundation in expertise, equipment, and personnel to facilitate M&S and HighPerformance Computing (HPC) research and education. Researchers bring M&S tools into theclassroom and laboratories for students to learn the capabilities, and limitations, of moderncomputational tools. Working with the DoD HPC Modernization Program, the M&SRC enriches cadetexperience and improves support of the Air Force Research Laboratory and other USAF organizationsin completing their missions. Descriptions of how M&S is performed at the Academy are included,as well as details about the computational resources being used. In addition, descriptions of themethods used to bring these tools to Department of Aeronautics classes are described indetail.|
Quick-Tip Q & A
A:[[ Here's one proposed a long time ago by friend and former [[ co-editor, Guy Robinson. [[ [[ There are some interesting collective nouns in the English [[ language. [[ E.g., a "pride" of lions, a "knot" of toads. [[ [[ The question is, what collectives would you propose for terms in [[ the supercomputing vernacular? For instance, we want collectives [[ for terms such as cores, nodes, infinite loops, bugs, compiler [[ options, defunct HPC vendors, ARSC consultants, irrelevent [[ benchmarks, etc., [[ etc. Where there is a known collective, maybe you can improve [[ on it. This should be interesting... maybe even fun. [[ # # Here's a response from Derek Bastille # A cluster of cores A stack of nodes A tangle of infinite loops A gaggle of consultants A swarm of bugs A (hungry) pack of HPC vendors A morass of compiler options A brace of compilers A thicket of libraries An assemblage of engineers A camp of users A bestiary of benchmarks # # Thanks to Rick Roberts for this response. # suggestion what it actually refers to ==================== =========================================== Clowder of cores clowder is a set of cats Nodule of nodes Murder of Crays set of HPC vendors; murder is a set of crows YiMP set of defunct HPC vendors Scalin-marks a set of irrelevent benchmarks Do-Mobius infinite loop Blasphemous Bill cost of new system Hypo-thermiants ARSC consultants ice-worms ARSC consultants (sorry, I couldn't help myself!) Guy de Part fondly remembered ARSC consultants. # # Here's are several suggestions from Lee Higbie. # A blyght of programmers. Maybe we also have flops of supercomputers? And, of course, it's the blyght of programmers using the flop of supercomputers that HPCMP has. Q: I just generated a bunch of image files that are sequentially named: image1.png, image2.png, ..., image100.png. Unfortunately when I do an "ls" on the directory, the files aren't sorted the way I would like. mg56 % ls -1 image100.png image10.png image11.png image12.png image13.png image14.png ... ... How can rename the files with padded zeros so the files show up in numerical order? For example, I would like the file image1.png to become image001.png and image33.jpg to become image033.png, etc. Show me how to do this shell gurus! [[ Answers, Questions, and Tips Graciously Accepted ]]
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669 Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678 Arctic Region Supercomputing Center University of Alaska Fairbanks PO Box 756020 Fairbanks AK 99775-6020
Subscribe to (or unsubscribe from) the e-mail edition of the
ARSC HPC Users' Newsletter.
Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.