We Are Not Alone...

Steve Oberlin, the MPP Program Director, sends out congratulations to T3D users:

"One year ago today (August 29, 1993), the first CRAY T3D system was accepted by Pittsburgh Supercomputer Center (PSC), marking Cray Research's formal entry into the massively parallel computing market. Since then, we`ve shipped 26 systems containing ~3200 processors: Almost half a TFLOPS of peak processing horsepower. It's been a *very* busy year!

Among the T3Ds shipped, we have these publicly announced systems:

  Universities:    R&D Labs:       Environmental:
  MC256 at PSC     MC128 at CEA    MCA128 at ECMWF
  MC128 at UAK     MC128 at CEA    MCA64 at NCAR
  MC128 at EPFL    SC128 at JPL
  MC256 at EPCC    SC128 at LANL   Industry (petroleum):
  MCA32 at OSC     SC128 at LLNL   MC256 at EXXON
  MC128 at MSC     MCA128 at PNC   MC128 at Phillips
  MCA64 at CINECA

Of course, this incredible "ramp" has not been scaled without pain and great effort. I think it's safe to say that few associated with the project won't have at least one battle scar to show to their grandchildren!

As we continue to enhance, sell, build, deliver, and service T3Ds, and push through the development of the follow-on T3E, we should all take a moment from time to time to reflect with pride on the unprecedented leap we've made into this new computing arena.

Congratulations! -Steve O."

More MPP Utilities

Frank Chism from CRI writes:

"I have provided Jim White (one of the CRI on site analysts at ARSC) with a script 'memsize' and a man page for it. This script cuts up the mppsize output and prints out the total decimal bytes used in each PE by a fixed number of PEs T3D absolute."

This is a very useful utility as it provides a one line output on the size of a T3D executable. The executable can not be a "plastic" one, so it must have been compiled and linked for a fixed number of PE. Check out the man page. The utility mppsize provides a much more detailed report.

More C calls Fortran

For the C programmer it is a slight to see a Fortran programmer do something that's not available in C. But because C can call Fortran, the C programmer at least has a workaround until the same feature is implemented in C. Here are two examples of C calling functions written in Fortran or assembler.

Example #1

In this small example, we add a second() function for the T3D that mimics the function SECOND() in the Fortran world by calling the IRTC function of /mpp/lib/libfi.a.

  makefile:

  all:
       /mpp/bin/cc -c main.c
       mppldr main.o
       a.out

  C program:

  main()
  {
    double t1, t2, second();
    int i, j, imax;
    double sin(), x;
        
    x = 0.1;   
    imax = 1;
    for( j = 1; j < 8; j++ ) {
    imax = 10 * imax;
    t1 = second();
    for( i = 0; i < imax; i++ ) {
    x = sin( x );
    }
    t2 = second();
    printf( "%12.8f seconds %8d evals 
    approx of zero %10.6f\n", t2-t1, imax, x );
    }
    }
      double second()
    {
      int IRTC();
        
       return( IRTC( ) / 150000000.0 );
    }

  output:

  0.00002929 seconds       10 eval approx of zero   0.098371
  0.00020649 seconds      100 eval approx of zero   0.085520
  0.00198797 seconds     1000 eval approx of zero   0.046111
  0.01989949 seconds    10000 eval approx of zero   0.016213
  0.19916001 seconds   100000 eval approx of zero   0.005189
  1.99164248 seconds  1000000 eval approx of zero   0.001643
 19.91509610 seconds 10000000 eval approx of zero   0.000520

By running this small program a few times you can get a feel for how precise the the second() function is. By replacing 'IRTC' with 'clock' you can experience how inaccurate clock() is.

Example #2

In the C environment on the T3D, the C programmer has the following tools:

1. PVM
2. SHMEM calls
3. The PSHAPE function
4. Barrier functions
5. Event functions

One capability missing is a synchronized updating of a shared variable. Channon Price of UAF showed me this code and I think its pretty neat:

  makefile:

  comp_run:
            /mpp/bin/cc -c source.c
            cf77 -c -X8 prot_count.f
            mppldr source.o prot_count.o
            a.out

  C program:

  #include <stdio.h>
  #include <stdlib.h>
  #include <fortran.h>     /* needed for the FORTRAN function */
  #define NUM_PES 8        /* substitute # of PE's for NPES before compilation*/
  #define NUM_PASS 5       /* number of passes through task */

  fortran int ICOUNT (void);      /* the FORTRAN function */

  main ()
  {
    int myself = _my_pe();          /* my PE number             */
    int count;                      /* local version of counter */
    int hist[NUM_PASS];             /* history of counter       */
    int pass = 0, loop, i, cnt;     /* some things to do     */
    long seed;                      /*   before incrementing */
    double x, y, pi;                /*     that counter      */

    srand48( 31415+65536*myself );  /*each PE gets a different seed*/
 /* for each PE, increment Pcount NUM_PASS times */
    while (pass < NUM_PASS) {
 /* here is the task; it's duration is not fixed */
          loop = 65536 * drand48();
          cnt = 0;
          for (i=0; i < loop; i++) {
              x = drand48();          /* a random in [0,1] */
              y = drand48();          /* a random in [0,1] */
              if (x*x+y*y <= 1) cnt++;
          }
          pi = (double) 4 * cnt / loop;   /* MC calc of pi... */
          count = ICOUNT();     /* increment the protected counter */
          hist[pass++] = count; /* record the value of the counter */
        }
 /* dump the counter histories in an orderly fashion */
    printf ("pe: %d  count history: ", myself);
    while (pass > 0) printf ("%d ",hist[NUM_PASS - pass--]);
    printf ("\n");

  }

  Fortran Program:

      FUNCTION ICOUNT ()
      COMMON /PROT/IPCOUNT
CDIR$ SHARED    IPCOUNT
CDIR$ CRITICAL
      IPCOUNT = IPCOUNT + 1
      ICOUNT = IPCOUNT
CDIR$ END CRITICAL
      RETURN
      END

  Sample output:

    pe: 1  count history: 4 7 11 16 20
    pe: 6  count history: 10 14 19 26 31
    pe: 3  count history: 2 6 18 25 33
    pe: 7  count history: 9 15 24 30 35
    pe: 4  count history: 1 8 21 27 36
    pe: 2  count history: 3 12 23 29 37
    pe: 5  count history: 13 22 32 38 39
    pe: 0  count history: 5 17 28 34 40

The file /usr/include/fortran.h has a lot of information about how C looks at the Fortran world. And the output of this program comes in the typical 'out of order' sequence of having each PE report as it finished, but the shared variable IPCOUNT allows for 'in order' execution of the final printf with the following modification:

  /* dump the counter histories in an very orderly fashion */

  while( 1 ) {
  if( ( TSTICOUNT() - ( NUM_PASS * NUM_PES ) ) == myself ) {
  printf ("pe: %d  count history: ", myself);
  while (pass > 0)
    printf ("%d ",hist[NUM_PASS - pass--]);
    printf ("\n");
    ICOUNT();
    break;
    }
  }

  and the additional fortran function and its declaration:

      integer function tsticount()
      common /prot/ ipcount
CDIR$ shared    ipcount
      tsticount = ipcount
      return
      end

  in the C program:

  fortran int TSTICOUNT (void);   /* another FORTRAN function */

  Now the output comes out as:

    pe: 0  count history: 5 17 29 34 40 
    pe: 1  count history: 4 7 11 16 20 
    pe: 2  count history: 3 12 23 28 37 
    pe: 3  count history: 2 6 18 25 33 
    pe: 4  count history: 1 8 21 27 36 
    pe: 5  count history: 13 22 32 38 39 
    pe: 6  count history: 10 14 19 26 31 
    pe: 7  count history: 9 15 24 30 35 

  and the output is ordered by PE number.

In coming newsletters I will provide examples data conversion in C using Fortran and Assembly language routines.

---

Current Editors:

Ed Kornkven	ARSC HPC Specialist	ph: 907-450-8669
Kate Hedstrom	ARSC Oceanographic Specialist	ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020

E-mail Subscriptions:

Subscribe to (or unsubscribe from) the e-mail edition of the ARSC HPC Users' Newsletter.

Archives:

Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.