ARSC HPC Users' Newsletter 351, October 27, 2006


SC is the premier international conference on high performance computing, networking, storage and analysis. SC06 is in two weeks, Nov 11-17, in Tampa.

This is a BIG year for us, as ARSC Associate Director, Barbara Horner-Miller, is serving as the SC06 Conference General Chair. Barbara will be many places at once, so she should be easy to find.

Other staff, including co-editor, Don Bahls, will be there, hoping to meet readers of This Newsletter. To locate ARSC folks: 1) Follow the smell of sunburn cream -- we burn easily; and, 2) look for people shivering pathetically in the air-conditioning -- Hey! Alaskans in Florida do NOT wear sweaters!

You might also look for ARSC at:

Booth 912:
University of Alaska Fairbanks/Arctic Region Supercomputing Center
Booth 2224:
DOD High Performance Computing Modernization Program

For more on the conference:

(Some of us will) See you there...

ARSC Sun System, Hardware Delivery, Next Week

As announced in issue 345, ARSC is installing a new 9.88 TFLOP Sun Linux system. We can report that hardware delivery is expected late next week. After that, we commence the rigorous process of setup, acceptance testing, etc...

Weather Modeling Symposium, Call For Papers

The Great Alaska Weather Modeling Symposium provides a forum for the exchange of research results and discussion of problems and new developments in weather modeling activities related to the Alaska environment.

Abstract Submission:

The deadline for one-paragraph abstracts for a 20-30-minute presentation is Tuesday, 9 January 2007.


The University of Alaska Fairbanks campus from Tuesday through Thursday, 13-15 March.

For details, see:

Link No Longer Available

Java for Fortran Programmers: Part V

[[ Thanks to Lee Higbie of ARSC for this tutorial. ]]

In the beginning of this tutorial, I said that how far and deep I take this tutorial depends on your feedback. Thanks again to those who responded initially. At this juncture, I again need to know your interest. It will help to ensure and guide future chapters.

The topic today is:

  • Interfacing Java and Fortran Programs

Java has a syntactic and semantic affinity for C, so calling between Java and C programs is common and accommodated by JNI, the Java native interface. The tool javah generates C include (.h) files. Thus, one way to call between Java and Fortran is to pass the call through a trivial C program. This is of less interest and more of a kludge than I want to discuss. I will discuss how to create GUIs for setting up and running Fortran programs.

One of the ongoing hassles that HPC users have is compiling their programs. There must be at least half a dozen different Fortran compilers, each with scores to hundreds of options. Though diddling with compiler options may interest a few, most HPC users have work they want to do and the compiler is a tool that cannot be avoided. My plan over the next few chapters is to create a GUI for generating compile statements. The target of this exercise will be a table-driven compile-statement generator that will present all the options that should interest most users in an intuitive way and create syntactically correct strings that a user can paste into his makefile or compile script.

For this Chapter I will create a user interface--a GUI. This could be done with one of the many GUI creators, but I will do it from scratch because I think it is more instructive and the code I write will be easier to understand than machine-generated code. Because the program for just the GUI is about 150 lines, I will only present snippets of the code here. The full code can be found as at: . In outline, I took the HelloBill code and added some widgets for the new Compiler Controller.

HelloBill is not very exciting--one widget, the exit Button, filling the Frame. When there are many widgets there, the situation become much more exciting, especially when windows are resized. Java has a group of classes that organize the visual objects in a container, all of which directly or indirectly implement the LayoutManager interface. Most of these will completely automate putting your widgets in a Frame or Window. At least for now, I will use the most structurally complex layout manager, but the only one that allows me to specify the layout with precision, the GridBagLayout.

If you want to place widgets in a grid on the screen, the GridLayout and GridBagLayout are the two obvious choices. GridLayout places the widgets in the frame with equal vertical spacing and equal horizontal spacing. GridBagLayout is a grid layout with the capability of having components span several columns and/or rows. This makes a much more pleasing layout when the components differ in size. First the declarations,

  GridBagLayout      gbl  = new GridBagLayout();
  GridBagConstraints gbc  = new GridBagConstraints();

The GridBagLayout object, gbl, will use the GridBagConstraints object, gbc, for each object in the Frame. As the Frame changes size, it will move the widgets around but keep them placed and their relative sizes as designed. The first statements of the constructor will use these objects and sets values used throughout the layout.

  gbc.anchor  = GridBagConstraints.CENTER;

Because the CompilerGUIa class extends Frame, it has a setLayout method and the first line above applies to the objects of this class. It is short for


which specifies the layout manager for this object. GridBagConstraints objects have a number of field variables that determine layout parameters, such as:

  anchor,  which determines where to place an object relative
    to its space.  It is an integer value, whose default value is
    the class constant CENTER, whose numerical value is unimportant.

  gridheight,  which specifies the number of rows the object
    will occupy.  It is used below.

 1      gbc.gridy      = rowNo++;
 2      gbc.gridx      = 0;
 3      gbc.gridwidth  = 1;
 4      gbc.gridheight = 2;
 5      gbl.setConstraints(compilerChoice, gbc);
 6      add(compilerChoice);
 7      int wid = 0;
 8      for (int i = 0; i < compilers.length; i++) {
 9         compilerChoice.add(compilers[i]);
10         wid = Math.max(wid, compilers[i].length());
11      }     // end for i loop:
12      compilerChoice.setBounds(0,0,wid*15, 40);

rowNo is an integer that was initialized to zero. It counts down the table and specifies the vertical position of objects. gbc.gridx specifies the horizontal position. In all the first five lines specify that the object compilerChoice will be double height, single width, in the upper (because rowNo is zero) left corner of the Frame. Notice that I can add the object to the Frame before I know how big it is, which is determined by the five lines and set in the last line. The List object compilerChoice is a Component, which has a setBounds method specifying the location, (0,0), in its space for compilerChoice as well as how wide and tall it is.

The last row of the Frame is added by:

 1      // last row
 2      gbc.gridy      = rowNo++;
 3      // gbc.gridx      = 0;        // done above
 4      // gbc.gridwidth  = 1;        // done above
 5      // gbc.gridheight = 1;        // done above
 6      gbl.setConstraints(compileBtn, gbc);
 7      add(compileBtn);
 7      gbc.gridx      = 1;
 8      gbl.setConstraints(updateBtn, gbc);
 9      add(updateBtn);
10      gbc.gridx      = 2;
11      gbl.setConstraints(disposeBtn, gbc);
12      add(disposeBtn);
13      setVisible(true);
14      disposeBtn.addActionListener(this);
15      addWindowListener(this);
16      this.pack();
17      setSize(new Dimension(width, height));

The three commented values, lines 3 to 5, are set earlier in the program and are repeated here so you can see the values. Except for the horizontal or x position, all three buttons are added (lines 7, 9, 12) with the same constraints. Line 13 makes this, the implied object for the method, visible so you can see the Frame. The mouse click and window click listeners are added and all the components are packed into the Frame by this.pack(). In line 16, "this" is not necessary, but is added here and could be added in all the other places where it is implicitly present. Finally, in line 17, the size of the Frame is set. Container's setSize method takes an argument of type Dimension, which is created just for passing to the method in the parameter field of setSize in line 17.

As with the earlier examples, you can download them from the web page. Once downloaded, open a terminal window and type,

  javac *.java

which will try to compile all your Java files. To run the application above, type,

  java CompilerGUIa <your name>

The print and debug statements will generate output to the terminal window where you ran java CompilerGUIa.

BLACS Point-to-Point Communication

The BLACS (Basic Linear Algebra Communications Subprograms), is a library of message passing subroutines designed specifically to support the development of parallel linear algebra routines. ScaLAPACK is built on the BLACS, and certain BLACS initialization routines must be used by anyone using ScaLAPACK.

Given its purpose, the BLACS doesn't need or want the huge variety of low- and high-level message passing functionality available in MPI.

However, the BLACS does offer point-to-point send and receive, broadcast, and barrier. Reflecting on the quick-tip question of last week, it seems reasonable that if you've already set up the BLACS communication environment, you might as well build the occasional reduction or other specialized operation on that base.

If you anticipated a lot of "BLACS coding," you'd probably be better off adding MPI to your BLACS/ScaLAPACK program.

It's common to mix MPI and BLACS/ScaLAPACK in the same program to get the strengths of both. You initialize them separately, and they exist in happy isolation from one another. You do a little MPI, do some BLACS/ScaLAPACK, do some more MPI, all the while operating on the same data. Just don't do anything silly like substitute a BLACS "context" for an MPI "communicator" or try to match an "MPI_SEND" with the receive function from BLACS, "xGERV2D"!

Back to the problem, here's one way to add a global minimum function using BLACS communication routines.

This example adds a function to the sample program, slv_part5.f90, in ARSC's ScaLAPACK Tutorial, which was published in issues 304, 305, 306, 308 , and 309 of this newsletter.

Here's the BLACS initialization from that Tutorial:

! -----    Initialize the blacs.  Note: processors are counted starting at 0.
      call blacs_pinfo( iam, nprocs )
! -----    Set the dimension of the 2d processors grid.
      call gridsetup(nprocs,nprow,npcol)
! -----    Initialize a single blacs context.  Determine which processor I
!          am in the 2D process or grid.
      call blacs_get( -1, 0, context )
      call blacs_gridinit( context, 'r', nprow, npcol )

For our new global_min function, we'll use the BLACS send and receive routines:

BLACS Send: 
  call DGESD2D (context, 1, 1, mymin, 1, 0, 0)

BLACS Receive: 
  call DGERV2D (context, 1, 1, tmpmin, 1, prowN, pcolN)

- In both DGESD2D and DGERV2D, the fourth argument is the 2D array or
  array section to be sent/received.  We're only sending one value,
  so the array in this example is 1x1.

- The last two arguments are processor row and column numbers.  In MPI,
  we usually think of processor numbers ranging from 0 to NPROCS-1
  (i.e., a 1-D virtual array of processors).    In the BLACS, we
  always think in terms of a 2-D array of processors, and, thus each
  processor is always identified by two numbers, processor row and
  processor column.
Here's the function:

      real(kind=8) function global_min (context,nprow,npcol,myrow,mycol,arr)
! Accumulates global minimum of arr on processor (0,0).  On (0,0), returns 
! it.  On other procs, returns huge.  All processors must call this at same
! time.
      implicit none
      integer :: context,nprow,npcol,myrow,mycol,prowN,pcolN
      real(kind=8) ::  arr(:,:), mymin(1,1), tmpmin(1,1)

      global_min = huge (global_min)
      mymin = minval (arr);

      if (myrow.eq.0 .and. mycol.eq.0)  then 

        global_min = mymin(1,1)

        do prowN = 0, nprow - 1
          do pcolN = 0, npcol - 1
            if (.not. (prowN.eq.myrow .and. pcolN.eq.mycol)) then
!              proc (0,0) receives local minimum from other processor
               call DGERV2D (context, 1, 1, tmpmin(1,1) , 1, prowN, pcolN)

               if (tmpmin(1,1) .lt. global_min) then 
                 global_min = tmpmin(1,1)



!       each proc sends its local minimum to (0,0)
        call DGESD2D (context, 1, 1, mymin(1,1), 1, 0, 0)

      end function global_min

You may download the tar file for the previously mentioned ARSC ScaLAPACK Tutorial:

add this routine to the code for Part V of the tutorial, and play with it as you wish. Makefiles and batch submission scripts are ready go for both the IBM systems and the dearly departed Cray X1.

Quick-Tip Q & A

A:[[ I use ScaLAPACK, therefore I use BLACS.  I'm on AIX, therefore I use
  [[ ESSL/PESSL.  I don't use MPI.  Here's my question:
  [[ I need the global minimum value of a distributed matrix. SHMEM provides
  [[ a routine to determine this:
  [[     shmem_double_min_to_all : 
  [[     "Performs a minimum reduction on array of doubles"
  [[ I don't see anything in ScaLAPACK and the routine provided in the
  [[ BLACS library is no help:
  [[   DGAMN2D: 
  [[     "Perform an elementwise absolute value max/min." ... "Remember, ... 
  [[     The absolute value is used to determine the max/min."
  [[ Suggestions? 

  # No one replied, so we'll just go with the example given above.

Q: I'd like the ESC key on my keyboard to be larger.  What can I do?

[[ Answers, Questions, and Tips Graciously Accepted ]]

Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top