ARSC T3E Users' Newsletter 156, October 16, 1998

Parallel Programming and T3E Course at ARSC

Massively Parallel Programming Basics on the ARSC T3E


Date:        Wednesday, December 2, 1998
Time:        1:00pm - 5:00pm
Location:    University of ALASKA Fairbanks, Butrovich Building,
             Room 007 Access Lab. 
Instructor:  Guy Robinson, ARSC MPP Specialist.

Course Description:

This course will provide an introduction to the parallel systems and associated software available at ARSC and how to take full advantage of these resources.

More Information:

Visit:

http://www.arsc.edu/user/classes/ClassMPPintro.html

or on any ARSC host, type:

news training

Co-array Fortran sync_images()

In Co-array Fortran, the sync_images() subroutine replaces MPI or SHMEM "barrier" calls. It operates in three different modes depending on the type of the arguments with which it's called. A user ran into trouble with these modes this week, so here's an alternate explanation to the man pages and a test code.

You can call sync_images() with:

  • No arguments
  • A scalar integer argument
  • An integer array argument

We'll take these case by case:

CALL SYNC_IMAGES () ! NO ARGS

The calling image blocks until all images make this call.

CALL SYNC_IMAGES ( PARTNER ) ! "PARTNER" is SCALAR INTEGER

Synchronizes pairs of images. Each image calls sync_images() with its partner's image number. The first member of the pair waits for its partner and then they both proceed.

This example assumes images 2 and 7 need to synchronize (other images are unaffected):




     Action on        Action on
Time  Image 2          Image 7
----  -------          -------

 1     work            work   
 2     work          sync_images(2)       
 3     work            waiting       
 4     work            waiting       
 5     work            waiting       
 6   sync_images(7)    waiting
 7     work            work                
 8     work            work                
 9   sync_images(7)    work
 10    waiting         work
 11    waiting         work
 12    waiting       sync_images(2)       
 13    work            work                
 14    work            work                

CALL SYNC_IMAGES ( ITEAM ) ! "ITEAM" is INTEGER ARRAY

The array MUST contain the local (or calling) image's image number.

If the array does NOT contain the local image's image number, then (according to the man page) the call's behavior is "undefined." From our tests, such calls may be ignored or cause OTHER images to hang.

If the array contains only the local image number, it's a no-op.

If it contains other image numbers in addition to its own, then it waits for those images to call sync_image() with an array which includes its image number.

Here are three examples... where "2,7" (for instance) indicates that the argument is a two element array with those values.


     Action on            Action on
Time  Image 2              Image 7
----  -------              -------

 1     work                work   
 2     work              sync_images("2,7")       
 3     work                waiting       
 4     work                waiting       
 5     work                waiting       
 6   sync_images("2,7")    waiting
 7     work                work                
 8     work                work                




     Action on            Action on          Action on
Time  Image 2              Image 7            Image 8                 
----  -------              -------            --------

 1     work                work                work
 2     work              sync_images("7,8")    work 
 3     work                waiting             work
 4     work                waiting             work
 5     work                waiting           sync_images("2,7,8")
 6     work                work                waiting
 7     work                work                waiting
 8   sync_images("2,8")    work                waiting
 9     work                work                work
 10    work                work                work




     Action on            Action on          Action on
Time  Image 2              Image 7            Image 8                 
----  -------              -------            --------

 1     work                work                work
 2     work              sync_images("2,7,8")  work 
 3     work                waiting             work
 4     work                waiting             work
 5     work                waiting           sync_images("2,7,8")
 6     work                waiting             waiting
 7     work                waiting             waiting
 8   sync_images("2,7,8")  waiting             waiting
 9     work                work                work
 10    work                work                work

A test code for sync_images() follows.

In this code, the final two cases concern the behavior when an array argument neglects to include its local image number. In the first of these two, every image makes this mistake. When it runs, the sync_images() calls are seemingly ignored and the code proceeds. The final case, however, mixes undefined calls with some good ones, and the code hangs. There's apparently some global interaction which confuses the good calls.

Compile by switching to PrgEnv3.1. To do this on yukon, execute,

module switch PrgEnv PrgEnv.test

Then execute:

f90 -Z -o synctest synctest.f90

And run. For instance:

mpprun -n6 ./synctest


!===============================================================
!===============================================================
program synctest

  implicit none
  integer, parameter :: master = 1
  integer :: nimgs, myimg, i, partner, pairing(2)

  nimgs = num_images()
  myimg = this_image()

  if (mod (nimgs, 2) == 1) then
    print*, "Must run on even number of processors--not ", nimgs
    stop
  endif


!========

  call sleep (1)

  if (mod (myimg - 1, 2) == 0) then
    partner=myimg+1
  else
    partner=myimg-1
  endif

  print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ SCALAR VALUE: ", partner
  call sync_images(partner)

  print*,"IMAGE", myimg, "DONE"

!========
  
  call sleep (1)
  if (myimg == master) print*,"-------------"
  call sleep (1)

  if (myimg <= nimgs/2) then 
    print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ ARRAY VALUE: ", (i, i=1,nimgs/2)
    call sync_images( (/ (i, i=1,nimgs/2)  /) )
  else
    print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ ARRAY VALUE: ", (i, i=nimgs/2 + 1, nimgs)
    call sync_images( (/ (i, i=nimgs/2 + 1, nimgs)  /) )
  endif

  print*,"IMAGE", myimg, "DONE"

!========

  call sleep (1)
  if (myimg == master) print*,"-------------"
  call sleep (1)

  if (myimg == master) then
    print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ ARRAY VALUE: ", (i, i=1,nimgs)
    call sync_images( (/ (i, i=1,nimgs)  /) )
  else
    pairing(1) = master
    pairing(2) = myimg

    print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ ARRAY VALUE: ", pairing
    call sync_images(pairing) 
  endif

  print*,"IMAGE", myimg, "DONE"

!========

  call sleep (1)
  if (myimg == master) print*,"-------------"
  call sleep (1)

  pairing(1) = mod (myimg-1+1, nimgs) + 1 
  pairing(2) = mod (myimg-1+2, nimgs) + 1 

  print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ ARRAY VALUE: ", pairing
  call sync_images(pairing) 

  print*,"IMAGE", myimg, "DONE"

!========

  call sleep (1)
  if (myimg == master) print*,"-------------"
  call sleep (1)

  if (myimg == master) then
    print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ ARRAY VALUE: ", (i, i=1,nimgs/2)
    call sync_images( (/ (i, i=1,nimgs)  /) )
  else
    if (myimg <= nimgs/2) then
      pairing(1) = master
      pairing(2) = myimg
    else
      pairing(1) = myimg-2
      pairing(2) = myimg-1
    endif   


    print*,"IMAGE", myimg, "CALLING SYNC_IMAGES W/ ARRAY VALUE: ", pairing
    call sync_images(pairing) 
  endif

  print*,"IMAGE", myimg, "DONE"

end program synctest
!===============================================================
!===============================================================


yukon$ mpprun -n6 ./synctest
 IMAGE 2 CALLING SYNC_IMAGES W/ SCALAR VALUE:  1
 IMAGE 1 CALLING SYNC_IMAGES W/ SCALAR VALUE:  2
 IMAGE 4 CALLING SYNC_IMAGES W/ SCALAR VALUE:  3
 IMAGE 5 CALLING SYNC_IMAGES W/ SCALAR VALUE:  6
 IMAGE 3 CALLING SYNC_IMAGES W/ SCALAR VALUE:  4
 IMAGE 1 DONE
 IMAGE 2 DONE
 IMAGE 6 CALLING SYNC_IMAGES W/ SCALAR VALUE:  5
 IMAGE 3 DONE
 IMAGE 4 DONE
 IMAGE 6 DONE
 IMAGE 5 DONE
 -------------
 IMAGE 1 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2,  3
 IMAGE 2 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2,  3
 IMAGE 3 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2,  3
 IMAGE 3 DONE
 IMAGE 2 DONE
 IMAGE 1 DONE
 IMAGE 4 CALLING SYNC_IMAGES W/ ARRAY VALUE:  4,  5,  6
 IMAGE 6 CALLING SYNC_IMAGES W/ ARRAY VALUE:  4,  5,  6
 IMAGE 5 CALLING SYNC_IMAGES W/ ARRAY VALUE:  4,  5,  6
 IMAGE 4 DONE
 IMAGE 5 DONE
 IMAGE 6 DONE
 -------------
 IMAGE 3 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  3
 IMAGE 2 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2
 IMAGE 1 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2,  3,  4,  5,  6
 IMAGE 4 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  4
 IMAGE 5 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  5
 IMAGE 6 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  6
 IMAGE 6 DONE
 IMAGE 5 DONE
 IMAGE 4 DONE
 IMAGE 3 DONE
 IMAGE 2 DONE
 IMAGE 1 DONE
 -------------
 IMAGE 5 CALLING SYNC_IMAGES W/ ARRAY VALUE:  6,  1
 IMAGE 6 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2
 IMAGE 5 DONE
 IMAGE 6 DONE
 IMAGE 4 CALLING SYNC_IMAGES W/ ARRAY VALUE:  5,  6
 IMAGE 3 CALLING SYNC_IMAGES W/ ARRAY VALUE:  4,  5
 IMAGE 4 DONE
 IMAGE 3 DONE
 IMAGE 2 CALLING SYNC_IMAGES W/ ARRAY VALUE:  3,  4
 IMAGE 2 DONE
 IMAGE 1 CALLING SYNC_IMAGES W/ ARRAY VALUE:  2,  3
 IMAGE 1 DONE
 -------------
 IMAGE 5 CALLING SYNC_IMAGES W/ ARRAY VALUE:  3,  4
 IMAGE 5 DONE
 IMAGE 6 CALLING SYNC_IMAGES W/ ARRAY VALUE:  4,  5
 IMAGE 6 DONE
 IMAGE 4 CALLING SYNC_IMAGES W/ ARRAY VALUE:  2,  3
 IMAGE 4 DONE
 IMAGE 3 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  3
 IMAGE 2 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2
 IMAGE 1 CALLING SYNC_IMAGES W/ ARRAY VALUE:  1,  2,  3
^C SIGNAL: Interrupt ( from process 0 )

 Beginning of Traceback (PE 0):
  Interrupt at address 0x8000d3e0c in routine '_LIST_SYNC'.
  Called from line 89 (address 0x8000022e8) in routine 'SYNCTEST'.
  Called from line 475 (address 0x800000c98) in routine '$START$'.
 End of Traceback.
yukon$ 

MPP, HPC, CUG, ETC. Meetings and Conferences

At SC98 there were a great many meeting announcements and calls for papers/participation. Below is a a selection of those which might be of interest to readers of the ARSC newsletter.
CUG

Cray Users Group Annual Meeting
May 24-28, 1999:
Minneapolis, MN, USA

http://www.cug.org/

HUG98


The 3rd Annual HPF User Group Meeting 
May 24-25, 1999: 
Maui, Hawaii.

This meeting provides an opportunity to meet and exchange ideas and experiences regarding the current HPF implementations and obtain up-to-date information on the future of HPF. Submissions are invited by February 26, 1999. Further information about this can be found at http://www.icase.edu/hug99 .

Frontiers'99


The 7th Symposium on the Frontiers of Massively Parallel Computation
February 21-25, 1999: 
Annapolis, Maryland.

Looking at the next generation of architectures, applications and problems. See http://cesdis.gsfc.nasa.gov/frontiers/front99.html for more information. This meeting will feature a workshop on Petaflop computing on February 22 for which more information can be found at http://www.capsl.udel.edu/CFP-TPF3 .

Ptools


Parallel Tools Consortium 1999 Annual Meeting
April 19-21, 1999:
Boulder, Colorado

Recent conferences have exposed the fact that parallel tool use is appallingly low among the high-performance computing community, despite increasingly vocal demands for software support. This meeting will bring together representatives from the federal, industrial, and academic sectors to address the the issues involved. For complete information see: http://www.ptools.org/meetings/apr99/mission.html

The Fifth European SGI/CRAY MPP Workshop


September 9-10, 1999:
Bologna, Italy.

Building on the 4 previous meetings this meeting will look at the applications running on the SGI-Cray systems such at T3D/T3E and Origin2000. More information will soon be available at http://www.cineca.it

And, after all the Y2K bugs are ironed out :-)

VecPar2000


4th International meeting on Vector and Parallel Processing
June 21-23, 2000: 
Porto, Portugal.

A multidisciplinary meeting on vector and parallel processing aiming to disseminate research results from many areas of science and engineering. More information on http://www.fe.up.pt/vecpar2000 .

CLK_TCK Revisited

As described in newsletter #154 , CLK_TCK is 75MHz. It's not 300, 450, or 600 MHz as might be expected on the T3E-600, -900, or -1200. And it's not 300 MHz as it was on the first T3E-600s.

Why?

A reader wrote to explain that this allows T3Es with a mix of processors to have a uniform timer. A single T3E can have 300MHz modules mixed with 450Mhz modules, for example.

75MHz is a common denominator to all of the available speeds. It is also the frequency of the module oscillator, which synchronizes the processors, and which is subsequently multiplied in hardware to the actual processor clock speeds.

Programmers! Ptools Survey

Your input is requested! The following survey will help the Ptools consortium improve the uniformity and value of HPC programming tools:

http://www.nacse.org/projects/HPCreqts/survey.html

Intended for people who develop HPC applications, this survey is sponsored by the Parallel Tools Consortium and the National Task Force on Requirements for HPC Software and Tools. It shouldn't take more than just a few minutes to fill out, and your input will help gauge user preferences.

Please distribute this message to other HPC users you know.

Quick-Tip Q & A



A: {{ Here's one for the Minnesotans, Norwegians, and other Arctic
      dwellers out there:

      You're obsessed to the point of absent-mindedness by the
      excitement of parallel programming.  What's the best way to
      remember that you've plugged your car in (so you don't drive
      away, rip the socket out of the wall, and run over your own
      extension cord)? }}




Eleven responses!  Lots of suggestions, and two confessions.  Wow!


The confessions:
----------------

- Having driven away without unplugging several times, this is a
  problem.  One winter when I was getting distressed at buying new
  cords (that can get rather expensive), I did resort to taking a bent
  coat hanger to prop up the cord off the bumper so it was more visible
  from inside the car.  The coat hanger, teasing by friends,  & minor
  embarrassment was enough to retrain me...  mostly, I rarely go through
  more than 3 cords a winter now.


- I must admit I DO have a very bad habit of driving off with the cord
  plugged in. If you should chance to look at the cord on my car you
  will see copious applications of electrical tape around the cord at
  both ends where I've pulled it not-quite-apart.



The suggestions:
----------------

COMMON SENSE:

- Unplug and wrap up the cord BEFORE you get in the car to start it.



REMINDERS INVOLVING THE CORD:

- Wind cord around wing mirror, through door handle, or over 
  hood-ornament.
  
- Tie the cord to the windshield wiper blade with a half-hitch:
       - The wiper is a good visual cue 
       - It makes a big "t-wang" when activated 
       - It is an essential component for vehicle operation 
             (read: must be replaced)
       - Large economic impact (read: Negative re-enforcement)
  
- Use a long cord with a status light.
  
- I back into the spot and run the cord down the driver side so that I
  have plenty of opportunity to trip over the cord while getting into 
  the vehicle.



OTHER REMINDERS:

- Put a check list on your steering wheel that includes: "unplug the
  car!"

- Put a mailbox flag on the front of your car.  When it's plugged in,
  put the flag up.  When you unplug it, put the flag down.

- Put a blanket on the hood when plugged in (requires blanket).

- Put some object (ashtray, gloves, floor mat, etc.) on dashboard.

- Low-tech:
  
    Get a blank ignition key for your car.  Whenever you plug the
    heater in, put the blank in the ignition so you won't be able to
    insert the real key when you get back.

- High-tech:

    Hey, if you're plugged in, you've got 110AC in the car! Rig up a
    flashing LED, strobe, horn, computer-generated voice, etc.  (This
    won't work, however, at those plugs where the power supply is
    cycled on and off.)

- Change radio station to hated station before plugging in, and turn
  the volume way up.

- Move seat way back or way forward.



MISCELLANEOUS

- Jack up the rear wheels.

- Dog-teams don't need cords.

- Well, staying home and leaving the car in the garage sounds
  like a good bet to me.

- Get a chauffeur.  If he forgets, fire him/her.  Also means you can
  continue working on parallel programming while someone else drives.





Q: The syntax:  (\  STUFF  \) , appears in the "sync_images" article
   sample code, above.  What does this kind of bracket do and where's
   it from?  Is it Fortran?

[ Answers, questions, and tips graciously accepted. ]


Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top