ARSC HPC Users' Newsletter 223, June 29, 2001

Yukon Users: Change your NQS Scripts

Most yukon users should make a minor change to their NQS scripts. Below is the text of "news noNFS", which explains the situation.

If you forget to add the described "noNFS" attribute to your script, don't worry, nothing terrible will happen. Your jobs will simply lose some potential run-time whenever chilkoot goes down for maintenance.

From "news noNFS" on yukon:

Beginning July 1, 2001, all yukon NQS jobs which do NOT require the /allsys or /viztmp filesystems should be modified to include the following qsub option:

   #QSUB -la noNFS    # Job is not dependent on /allsys or /viztmp

Such jobs will be detected and allowed to continue running, even when chilkoot, which serves /allsys and /viztmp, is taken down. Yukon jobs which access /allsys or /viztmp should not specify "noNFS". Such jobs will be held when chilkoot, and thus /allsys and /viztmp, are down.

Background information:

This change is required because DMF for the local yukon filesystems, /u1, /u2, and /tmp, is now independent of DMF on chilkoot. This is an extremely positive development. Yukon can stay up regardless of chilkoot's status; file system and DMF performance have improved, and network traffic reduced, on both systems. However, yukon does still rely on chilkoot for the two shared file systems.

NFS file systems

/allsys and /viztmp are two NFS mounted, DMF supported, file systems connected directly to chilkoot, and available from most ARSC systems. For details, see "news viztmp", "news allsys", and"

It will benefit many yukon users, those who do not access the NFS filesystem from their batch jobs, to continue running jobs even when chilkoot is down. The "noNFS" mechanism makes this possible without risking jobs which do require the NFS file systems.

NQS job attributes

The qsub "-la" option creates a job "attribute," which is simply an informational field, for the job. It has no immediate effect within NQS other than to cause the name of the attribute to appear in the "qstat -f" output for the job. On yukon, the attribute "noNFS" will be detected by ARSC scripts which handle the clean shutdown of chilkoot. Jobs lacking the "noNFS" attribute will be held and released again after chilkoot comes back up.

Published Work of Newsletter Subscribers

[ This is an occasional series, designed to increase collaboration between readers of this newsletter. Whenever you publish research involving high-performance computing let us know. Send us the citation, relevant web sites, abstract, and your e-mail address (if you like), and we'll pass it on. ]

A Unified Framework for Eliminating Redundant Array Subexpressions

Steven J. Deitz Bradford L. Chamberlain Lawrence Snyder

In 15th ACM International Conference on Supercomputing (ICS'01)

Array programming languages such as Fortran 90, High Performance Fortran and ZPL are well-suited to scientific computing because they free the scientist from the responsibility of managing burdensome low-level details that complicate programming in languages like C and Fortran 77. However, these burdensome details are critical to performance, thus necessitating aggressive compilation techniques for their optimization. In this paper, we present a new compiler optimization called Array Subexpression Elimination (ASE) that lets a programmer take advantage of the expressibility afforded by array languages and achieve enviable portability and performance. We design a set of micro-benchmarks that model an important class of computations known as stencils and we report on our implementation of this optimization in the context of this micro-benchmark suite. Our results include a 125% improvement on one of these benchmarks and a 50% average speedup across the suite. Also we show a speedup of 32% improvement on the ZPL port of the NAS MG Parallel Benchmark and a 29% speedup over the hand-optimized Fortran version. Further, the compilation time is only negligible affected.

Array Language Support for Parallel Sparse Computation

Bradford L. Chamberlain Lawrence Snyder

In 15th ACM International Conference on Supercomputing (ICS'01)

This paper describes an array-based language-level approach to parallel sparse computation. Our approach is unique due to its separation of sparse index sets from arrays, both syntactically and in the implementation. This design allows users to express their computation using dense array syntax, making the code easier for readers to understand and for compilers to parallelize and optimize. This work is done within the context of Advanced ZPL, retaining its crisp syntax and source-level performance model. Our implementation uses a novel sparse storage format that supports general operations such as arbitrary iteration and slicing. We describe how our compiler automatically optimizes this data structure into more compact forms based on the operations required by the program. We demonstrate our approach using the NAS CG and MG benchmarks, comparing our implementations with the original Fortran+MPI versions in terms of clarity and performance. We present performance results on the Cray T3E indicating that our implementation compares favorably to the hand-coded NAS versions in terms of memory requirements and often surpasses them in terms of execution speed.

You can order copies of these papers from the ACM, find them in the ICS proceedings or in the ACM digital library (once they're posted there).

Brad Chamberlain:

Fires, Bears, and other Adventures

In the general-interest category, it's fire season... about 100,000 acres have burned thus far in the Tanana Flats, south of Fairbanks. We had a couple days of bad smoke in town, and some incredible sights of smoke and fire-induced cloud formations.

Also, two of our readers have already had bear encounters this summer.

If you've had a brush with a large predator recently, send us a few sentences... We'll run a compendium such stories in a month or so.

Quick-Tip Q & A

A:[[ I'm using both ja and hpm to collect timing/performance data,
  [[ like this:
  [[    chilkoot$  (ja; hpm ./a.out; ja -cst) > a.out.timing
  [[ The problem is that ja writes to stdout while hpm writes to stderr.
  [[ Thus, the ja output is redirected successfully to a.out.timing, but
  [[ the hpm output gets dumped to the screen.
  [[ I'm not religious... csh, ksh, I don't care... just tell me which
  [[ shell to use and what to do.  Thanks.

# We have three solutions: Using hpm's -o option, then generic csh 
# and ksh methods for file redirection:

# hpm
# Courtesy of Pascale Lherminier, NPS

chilkoot$ (ja; hpm -o a.out.hpm ./a.out; ja -cst) > a.out.timing

# csh  
# Thanks to Ed Anderson, Lockheed Martin Technical Services, csh solution:

If you want both standard output and standard error in the same file,

   hpm date >& date.outerr

If you want standard output and standard error in different files,

   (hpm date > date.out) >& date.err

If you want the ja output in yet another file,

   ja; (hpm date > date.out) >& date.err; ja -cst > date.ja

If you want all the output in the same file,

   ja; hpm date >& date.all; ja -cst >> date.all

# ksh
# Scraped up by editors. Matches Ed Anderson's last case, 
# everything going to the same file:

(ja; hpm ./a.out; ja -cst) > a.out.timing 2>&1

Q: grep ITSELF shows up whenever I grep the output from ps.
   For instance,

   ICEHAWK1$  ps -aef 
 grep xloadl
     mortimer 15172 20020   1 11:27:42  pts/0  0:00 grep xloadl 
     mortimer 19344 20020   0   Jun 26  pts/0  3:09 xloadl 

   I know I'm running grep, so why should grep tell me (like, would I
   grep grep?)?  How can I get rid of this.  Can you switch it off?

[[ Answers, Questions, and Tips Graciously Accepted ]]

Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top