ARSC T3E Users' Newsletter 115, March 7, 1996

Yukon Installation Underway

ARSC's 88PE, liquid cooled, 128Mb/PE, 300Mhz CRAY T3E has been plumbed, electrified, and named. It is: UNICOS/mk 1.4.1 will be installed next week, and user test codes will start running soon after that.

ARSC T3E Press Release

    Arctic Region Supercomputing Center Installs CRAY T3E System
Fairbanks, Alaska -- The Arctic Region Supercomputing Center (ARSC) at
the University of Alaska Fairbanks (UAF) received a new high-
performance computing system on February 26, 1997, complete
installation will be done by mid-March. According to ARSC Director,
Frank Williams, "The CRAY T3E system enables ARSC to provide scientists
and engineers with more powerful computing resources. Some of the
world's foremost faculty and students are discovering new arctic region
knowledge in geophysics, marine sciences, natural sciences, and
engineering using ARSC facilities."

The new system is a CRAY T3E massively parallel processing (MPP)
computer from Cray Research, the supercomputing subsidiary of Silicon
Graphics, Inc.  (SGI). The system consists of 88 processors, each with
128 Mbytes of memory, or double the per-processor memory of the older
CRAY T3D system. Improved architecture of the CRAY T3E supercomputer
makes it three-to-four times faster than the CRAY T3D system. The
Center's operation of the CRAY T3D system will be phased out over the
next few months.

The new CRAY T3E supercomputer, named Yukon after Alaska's largest
river, will support the research and development efforts of scientists
from the academic community, federal research agencies, the state of
Alaska, private enterprise, and the Department of Defense. Over the
next month, researchers from sciences such as geophysics and
astrophysics will move their computing activities over to the new CRAY
T3E system. For example, the increased memory and speed permit enhanced
modeling of the polar oceans and ice cover to include input from the
various continental rivers. Currently, scientists are unable to follow
the flow from each river once it enters the ocean.  Greater computing
capabilities will allow 'tracing' of each river's flow after entering
the ocean providing increased information on the contribution of each
river and the possible environmental impacts.

Another CRAY T3E-related project teams ARSC with the UAF Geophysical
Institute, the Alaska Volcano Observatory, and the State of Alaska
Seismology Office to develop a parallel version of a 3D seismic
modeling code. Information gathered from existing seismic surveys and
naturally occurring events are used to construct a high-resolution
numerical model of the geological structure of the mountain range and
surrounding areas. This model will predict the results of possible
earthquake events throughout Alaska. The CRAY T3E system, combined with
ARSC's sophisticated Silicon Graphics visualization resources, make
ARSC an important asset to scientists seeking to perform such high-end,
detailed simulations.

ARSC continues to operate an eight-processor CRAY Y-MP large memory
vector supercomputer with 8GB (1 GigaWord) of globally addressable
memory. This machine, named Denali, has been in operation since January
1993, and serves the same user community as the CRAY T3E system. The
CRAY Y-MP supercomputer remains a workhorse for scientists and

In addition, connected to these CRAY supercomputers, via local
ethernet, FDDI, and HiPPI networks, will be a new SGI Onyx-2 Infinite
Reality visualization supercomputer, an SGI high performance Onyx XL,
and over 40 additional SGI workstations located in ARSC labs across the
UAF campus.

The three visualization labs, the ARSC Video Production Lab, and the
ARSC Training Lab are undergoing significant hardware improvements in
1997 to keep pace with user needs and advanced technology. An
integrated digital video/audio recording studio will be built around
the SGI Onyx-2's high-end video output, upgrading essential multimedia
capabilities of ARSC.

In operation since 1992, ARSC continues to support computational
research, science, and engineering--with emphasis on the high latitudes
and the Arctic. ARSC provides scalar-vector and massively parallel
processing (MPP) talent and related resources--high-performance
computing, networking facilities, training, and technical support--to
federal and state agencies, the academic community, and commercial

OSC, PSC, and ARSC co-sponsor Parallel Computing Conference

[ For the complete announcement, see: .

Guy Robinson and Don Morton will both be presenting papers. Don is a frequent contributor to this newsletter and annual ARSC visiting researcher; and Guy, of course, is ARSC's new MPP specialist. ]


            Parallel Computing: The T3D And Beyond

                        April 2-3, 1997



 Sample Session Topics:

      * Lattice QCD on the Cray T3D/E                       

      * Parallelization of the RMATRX codes for atomic physics calculation

      * DNS-based Fluid Flow Simulation on the T3D        

      * A T3D Implementation of an SPMD Code for Adaptive Finite 

        Element Methods


 Learn more about:

      * What your colleagues are doing.  

      * How to select the best parallel architectures for your science.  

      * How the DoD/MRSC are promoting parallel processing.  

      * The future direction of parallel processing.


 Sponsored by:

      * Pittsburgh Supercomputing Center

      * Arctic Region Supercomputing Center

      * Ohio Supercomputer Center


 The three centers are a part of PhAROh, a National Science Foundation

 (NSF) Metacenter Regional Alliance.


 The PhAROh MRA is a natural alliance between centers with similar and

 compatible computational facilities and operational philosophies. All

 of the centers have Cray T3D systems and share documentation and

 information resources through e-mail and WWW applications. By pooling

 resources, the PhAROh MRA centers effectively share developing

 expertise and understanding of the T3D environment, share code porting

 strategies and performance analysis tools, and co-develop outreach

 training and remote user service programs.


 Partial funding for the conference is provided by the NSF through the

 Metacenter Regional Alliance Program.

Accessing a Single T3D PE Through NQS

We had a frustrated T3D user a couple weeks ago -- a grad student who, in the wee hours of the morning, couldn't get NQS to run a simple 1 PE job which was part of his benchmark suite. The same script and program had worked as expected for 8 and more PEs.

Here's an NQS script which reproduced the problem:

    #QSUB -q mpp
    #QSUB -l mpp_p=1          # Request 1 PE from NQS
    #QSUB -l mpp_t=600        
    #QSUB -l p_mpp_t=600
    #QSUB -s /bin/ksh
    cd /tmp/baring/IO
    ./iompp 125 -npes 1       # Tell mppexec to run on 1 PE

Here's what happens when I submit this:

    denali$ qsub io1.tst.qsub
    nqs-181 qsub: INFO 
      Request <4754.denali>: Submitted to queue <mpp> by <baring(1235)>.

    denali$ cat io1.tst.e4754
      mppexec: MPP JOB PE limit exceeded

This is exactly the same error message mppexec gives if you try to exceed your interactive PE limit. Why would it occur in a batch job, on a request for only 1 PE?

It is actually impossible to allocate only one PE on a T3D. There are 2 PEs per node, and, at the minimum, you get 1 node. mppexec is smart enough that when you request 1 PE, it will essentially disable the second PE on the node you are given, your program will run on the other PE, and it will look like you have actually obtained a single PE. mppexec, however, has allocated 2PEs from the system, which is what you do see in mppview.

The above NQS script specifies:

    #QSUB -l mpp_p=1

NQS looks at this, and correctly puts the request in the m_8pe_24h queue (the smallest ARSC queue that will hold the request of 1PE). However, NQS interprets the 1PE request literally, and, in its own allocation system, reserves 1PE (not a whole 2PE node) for the job.

When mppexec gets the request:

  ./iompp 125 -npes 1
it knows to allocate 2PEs, checks with NQS to make sure that does not violate the request, and gets the message that, yep, it is an attempt to grab more than requested, and aborts with the error message.

It is like trying this:

  #QSUB -l mpp_p=2
  ./iompp 125 -npes 4
except that the error is hidden.

The fix is to ask NQS for 2PEs.

  #QSUB -l mpp_p=2
But to ask mppexec to execute on only one PE:

  ./iompp 125 -npes 1
as before.

Good news on the T3E: you may request and receive exactly one PE.

Newsletter News

After its three month vacation, this newsletter is back as the "ARSC T3E Users' Newsletter". It will continue to carry T3D articles, but will focus increasingly on the T3E.

It is now biweekly. The general style, intent, and audience, as established by Mike Ess, however, will remain essentially the same. Please send us your T3E questions, suggestions, experiences, ideas, and tips.


Quick-Tip Q & A

A: {{ In 'vi' is there a way to tell which column the cursor is in?  (Say
      you're editing Fortran and hope the line ends before column 72.) }}

   # I guess the answer is we need GNU vi! Thanks to the readers who
   # responded.

   1) Try "set ruler" in ed mode (you know, ESC, then ":") It works
   with my brand (GNU) of vi.
   2) Use the pipe character '|' preceded by a number to move the cursor
   to a particular column.  For instance: 
   in command mode moves the cursor to column 10. Hunt and peck until the 
   cursor lands where it was, and you'll know the column number.....
   3) Create a vi map to insert your own ruler, as fortran programmers
   have done before. In ed mode (ESC, ":") type:

:map v 0oC23456789-123456789-123456789-123456789-123456789-123456789-^[
   the ^[ is the ESC. To create it, hit CTRL-v, then ESC. Once the map
   has been created, hitting "v" while in command mode will open a line
   below the current, and enter the ruler into it.  Just delete the 
   ruler once you measure to the column in question.

Q: How would you condense every occurrence of multiple blank lines in a
   file into a single blank line?

[ Answers, questions, and tips graciously accepted. ]

Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top