ARSC HPC Users' Newsletter 226, August 17, 2001

Chilkoot Programming Environment Upgrade to PE

On August 29, we will switch the default programming environment on chilkoot to PE3.5.0.1.

This environment is available now as, and we encourage chilkoot users to test their codes with it prior to the upgrade. Please let us know if you see any changes in performance, or detect any problems.

To compile with PE3.5.0.1, execute this command:

module switch PrgEnv

Note that the PE3.5 compilers make the best use of the SV1e processors when you set your "TARGET" environment variable to "cray-sv1". To do so, execute the following:

      # csh users:
      setenv TARGET cray-sv1

      # ksh users:
      export TARGET=cray-sv1

(You might put the appropriate command in your .cshrc or .profile file.)

The current default programming environment, PE 3.4, will remain available after this upgrade.

Meet ARSC Oceanographic Specialist, Kate Hedstrom, PhD

[ We're happy to welcome Kate Hedstrom to ARSC User Services. She's been in Fairbanks for about a month now, and will help ARSC provide a new level of support and collaboration to our users running ocean models.

Thanks to Kate for the following bio... ]

I have been working with Dale Haidvogel since 1988, first while working at the Institute for Naval Oceanography in Mississippi and later while at Rutgers. In that time, I worked with various ocean and sea ice models, wrote the SPEM and SCRUM user manuals, and became the local expert on grid generation, both for the finite difference and finite element ocean models. I first used the ARSC computers to run a coupled ice-ocean model of the Arctic on the Y-MP (denali) when it was new.

In the early days, we ran the model on everything from Cray COS to VMS. Since then, we have narrowed our focus to Unix. Although we had a CM-Fortran version of the model, the current version (ROMS) has been finely tuned to run well on an SGI Origin and other shared memory systems.

There are two current efforts to port ROMS to MPI, one of which is based on the NOAA Scalable Modeling System, or SMS, library. (See Newsletter #219, /arsc/support/news/hpcnews/hpcnews219/index.xml for more on SMS.) The SMS effort is being done by the FSL NOAA lab in support of Al Hermann's work. I have recently obtained that version of ROMS to run on the ARSC computers.

My current project is called SALMON I and is an ocean circulation model of Prince William Sound and some of the surrounding ocean. The work is being done in collaboration with Dave Musgrave and others over in the Institute of Marine Science at UAF. We hope to have an initial configuration running fairly soon, with plans to add higher-resolution nested grids, biology, real-time atmospheric forcing, and data assimilation in the future. SALMON II will cover Cook Inlet where things can freeze up in the winter, requiring yet another sea ice model.

Perfview Back on SV1

Profiling tools are critical for determining the most heavily used routines or loops in your code, and thus focusing your optimization efforts.

For a while now, profview has been the only tool available on the SV1, but in a new release of craytools, our old favorites are back...

To use perfview on chilkoot you need to compile with the PE compilers, and use perftrace from craytools. Assuming you're using the default PrgEnv, here are the steps:

  1. Load PE

    module switch PrgEnv

  2. Compile your program for perftracing

    Fortran: f90 -ef -lperf -o prog prog.f C: cc -F -lperf -o prog prog.f

  3. Run the executable (which will produce a file)


  4. Load craytools.

    module unload craytools_archive module switch craytools. craytools.

  5. Examine output


Following the programming environment upgrade noted earlier, you will be able to skip step 1.

If you haven't used it, perfview is a nice tool. Among the things it will do for you:

  • For every subroutine, present the same statistics that "hpm" presents when run against your entire program.

    Thus, for instance, you can compare the MFLOPS and Cache Hit rates for different subroutines (or use the data to test different versions of a subroutine).

  • Show the percentage of total CPU time consumed by each subroutine, so you can identify hot-spots and bottlenecks.

  • Tell you how many times each subroutine was called, the average time per call, and provide advice concerning inlining.

  • Let you instrument your code to obtain statistics for any code segment... for instance, for a particular loop within a subroutine.

Quick-Tip Q & A

A:[[ I am rather upset, so please be nice.
  [[ My Fortran program has variable called "sec", and I need to find
  [[ wherever I subtract it.  The coding is not always consistent, so
  [[ these statements can take various forms, like:
  [[   dtime=tstart-sec 
  [[   dtime = tstart - sec 
  [[ I figured I'd be clever, so I tried egrep, but it didn't work.  So
  [[ I tried grep, and got the same weird output!  It's a total miss:
  [[  morty$ egrep "-sec
- sec
-  sec" littleprog.F 
  [[      count(4) = 1
  [[      write(*,*) 'parameters for ncvgtg call' 
  [[      write(*,*) 'ncid   ',ncid
  [[        write(*,*) 'sec:   ',sec
  [[      write(*,*) 'Calling ncvgtg'
  [[      call ncvgt(ncid,dataid,start,count,data,error)
  [[      write(*,*) 'Finished calling ncvgtg'
  [[      call ncclos(ncid,error)
  [[         close(1)
  [[        close(7)
  [[  morty$ grep "-sec" littleprog.F 
  [[      count(4) = 1
  [[      write(*,*) 'parameters for ncvgtg call' 
  [[      write(*,*) 'ncid   ',ncid
  [[        write(*,*) 'sec:   ',sec
  [[      write(*,*) 'Calling ncvgtg'
  [[      call ncvgt(ncid,dataid,start,count,data,error)
  [[      write(*,*) 'Finished calling ncvgtg'
  [[      call ncclos(ncid,error)
  [[         close(1)
  [[        close(7)
  [[ So my question is:  WHAT HAVE YOU CHANGED ON YOUR SYSTEM!  (or...
  [[ ahem, am I doing something wrong?)

Guess I stumped everyone with this one:

Though quoted, "-sec", appears to grep and egrep as a command line
option, as if you had typed:

  morty$  grep -sec littleprog.F 

And, by coincidence, "-sec" forms a legal set of options, so grep
doesn't report an error.  From "man grep":

    -s      Suppress the error messages ordinarily written for non-existent
             or unreadable files. Other error messages will not be suppressed.

    -e pattern_list
             Specify one or more patterns to be used during the search for
             input. [...cut...]

The command is searching for the pattern "c" in littleprog.F. Examining
the above grep output, you'll indeed find a "c" on every line.

Here's the real question: 

When the argument to a Unix command begins with a hyphen, how can 
you pass it to the command?

One answer is that most Unix commands have the undocumented option,
"--", which signifies the end of all the options.  Thus, here's one

   morty$  egrep -- "-sec
- sec
-  sec" littleprog.F
   morty$  grep  -- "-sec" littleprog.F

A second solution is to escape the hyphen:

   morty$  egrep "\-sec
- sec
-  sec" littleprog.F
   morty$  grep  "\-sec" littleprog.F

This was similar to our very first Quick-Tip, "How do you remove a 
file named -i"?  See:


Q: My favorite web site uses html frames.  Can I capture the URL of a
   frame, so I can send it to someone?  Or do I have to recite every
   link I follow?

[[ Answers, Questions, and Tips Graciously Accepted ]]

Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top