[Menu Bar]Resourses at ARSCScience at ARSCNewsroomSupportAbout ARSCARSC Home

ARSC system news for all systems

Menu to filter items by type

Type Downtime News
Machine All Systems linuxws midnight pingo seawolf

Contents for all systems

linuxwsmidnightpingoseawolf
  • archive
  • compilers
  • filezilla
  • idl
  • kerberos
  • modules
  • software
  • storage
  • support
  • PET_HOME
  • PrgEnv
  • file_striping
  • filezilla
  • kerberos
  • matlab
  • modules
  • mpi
  • pbs_10
  • pin
  • projects
  • queues
  • retirement
  • samples_home
  • software
  • storage
  • support
  • totalview
  • PET_HOME
  • PrgEnv
  • eslogin
  • file_striping
  • filezilla
  • kerberos
  • modules
  • projects
  • queues
  • samples_home
  • software
  • storage
  • support
  • totalview
  • archive
  • idle
  • kerberos
  • largefile
  • pin
  • software
  • ssh
  • support
  • News Items

    "PET_HOME"

    Last Updated: Wed, 20 Jan 2010 - Alec Bennett
    Machines: pingo
    PET_HOME
    ========
    The PET-Computational Environment (CE) consists of a number of packages and
    libraries used in High Performance Computing.  These tools are available in
    the $PET_HOME directory on Pingo.
    
       Current Versions in $PET_HOME
       =============================
    
       Package Name         Version
       CMake		2.6.4
       FFTW-2 (sequential)	2.1.5
       FFTW-3 (distributed)	3.1.3
       GotoBLAS		1.26
       GSL			1.12
       HDF5			1.8.3
       MatlabMPI		1.2
       netCDF		4.0.1
       Octave		3.0.5
       PAPI			3.6.2
       PETSc		2.3.3
       ScaLAPACK		1.8.0
       Scalasca		1.1
       SPRNG		3.0
       SuperLU		3.0
       TAU			2.18.1
       Trilinos		9.0.3
       Valgrind		3.4.0

    "PET_HOME"

    Last Updated: Wed, 20 Jan 2010 - Alec Bennett
    Machines: midnight
    PET-Computational Environment (CE)
    ==================================
    
    The PET-Computational Environment (CE) consists of a number of packages and
    libraries used in High Performance Computing.  These tools are available in
    the $PET_HOME directory on Midnight.
    
       Current Versions in $PET_HOME
       =============================
    
       Package Name         Version
       ARPACK		96
       CMake		2.6.4
       FFTW-2 (sequential)	2.1.5
       FFTW-3 (distributed)	3.1.3
       GotoBLAS		1.26
       GSL			1.12
       HDF5			1.8.3
       MatlabMPI		1.2
       netCDF		4.0.1
       Octave		3.0.5
       PAPI			3.6.2
       PETSc		2.3.3
       ScaLAPACK		1.8.0
       Scalasca		1.1
       SPRNG		3.0
       SuperLU		3.0
       TAU			2.18.1
       Trilinos		9.0.3
       Valgrind		3.4.0

    "PrgEnv"

    Last Updated: Wed, 22 Oct 2008 - Donald Bahls
    Machines: midnight
    Programming Environments on midnight
    ====================================
    Compiler and MPI Library versions on midnight are controlled via
    the modules package.  New accounts load the "PrgEnv" module by
    default.  This module adds the Pathscale compilers and Voltaire
    MPI compilers to the PATH.  
    
    Should you experience problems with a compiler or library in 
    many cases a new programming environment may be available.
    
    Below is a description of available Programming Environments:
    
    Module Name      Description
    ===============  ==============================================
    PrgEnv           Programming environment using Pathscale 
                     compilers and MPI stack (default version).
    
    PrgEnv.new       programming environment using the latest available
                     version of the Pathscale compilers.
    
    PrgEnv.path      Alternate name for PrgEnv
     
    PrgEnv.path.new  Alternate name for PrgEnv.new
    
    PrgEnv.gcc       Programming environment using GNU compilers 
                     and MPI stack.
    
    PrgEnv.pgi	 Programming environment using PGI compilers
    		 and MPI stack.
    
    PrgEnv.sun       Programming environment using Sun Studio compilers
                     and MPI stack.
    
    
    For a list of the latest available Programming Environments, run:
    
       mg56 % module avail PrgEnv
    
       --------------------- /usr/local/pkg/modulefiles ----------------
       PrgEnv           PrgEnv.new        PrgEnv.pgi 
       PrgEnv.gcc       PrgEnv.path       PrgEnv.sun 
       PrgEnv.gcc.new   PrgEnv.path.new   PrgEnv.sun.new
    
    
    ARSC Policy for Maintaining Programming Environment Module Versions
    ===================================================================
    ARSC maintains a "current" programming environment, the last
    programming environment and the most recently released version using
    the "modules" facility.  These policies explain the naming scheme
    for the programming environment modules as well as the schedule for
    changing the versions referenced by those modules.
    
    - These policies pertain to the PathScale, PGI, and SunStudio compilers.
    PathScale is our "official" compiler, hence it is pointed to by the
    special names PrgEnv, PrgEnv.old and PrgEnv.new reflecting the
    "current", "old" and "new" versions respectively.  These are aliases for:
       PrgEnv     == PrgEnv.path
       PrgEnv.old == PrgEnv.path.old
       PrgEnv.new == PrgEnv.path.new
    There will be "current", "old" and "new" versions for the other
    compilers as well
       PrgEnv.pgi, PrgEnv.pgi.old, PrgEnv.pgi.new
       PrgEnv.sun, PrgEnv.sun.old, PrgEnv.sun.new
    and in the discussion that follows, it is understood that policies that
    apply to "PrgEnv", "PrgEnv.old" and "PrgEnv.new" apply to all three
    compilers.
    
    - The above module names are aliases for modules named by their version,
    e.g., PrgEnv.path-3.0.
    
    - When versions newer than "PrgEnv.new" are made available, we will 
    assign them a version-specific module name and notify users of their
    availability.  We will install new Programming Environment versions as
    soon as practical, balanced against the benefits of the new version as
    judged by ARSC staff.
    
    - Under normal circumstances, i.e., no "extreme" need to act sooner, we
    will rotate "PrgEnv", "PrgEnv.old" and "PrgEnv.new" no more frequently
    than twice annually, on or about October 1 and April 1.  This rotation
    will be done only if "PrgEnv.new" is actually a newer version than
    "PrgEnv" and only if "PrgEnv.new" has been available for testing for at
    least one month.  The rotation keeps the old version and updates the
    current version as follows:
       "PrgEnv.old" <-- "PrgEnv"
       "PrgEnv"     <-- "PrgEnv.new"
       "PrgEnv.new" <-- a newer version (if it exists)
    
    - Under extreme circumstances (e.g., a version that we cannot tolerate
    being used or a significant system upgrade), ARSC staff may elect to
    rotate these versions sooner or remove a version entirely.
    
    - Insofar as it is feasible, we will maintain old versions (older than
    "PrgEnv.old") as long as possible and will always maintain at least the
    "PrgEnv.old" version.
    
    - Any changes to "PrgEnv.old", "PrgEnv" and "PrgEnv.new" will be
    preceded by at least two weeks notice to users.  
    
    
    Programming Environment Changes
    ================================
    The following is a table of recent additions and changes to the
    Programming Environment on midnight.
    
      Date         Module Name            Description
      ----------   ---------------------  -----------------------------------
      2009-10-21   PrgEnv.path            Fall updates to Programming
                   PrgEnv.path.old        Environment.
                   PrgEnv.pgi.new
                   PrgEnv.pgi.old
                   PrgEnv.pgi
                   PrgEnv.pgi.new
      2008-10-23   PrgEnv.path.new        Fall updates to Programming 
                   PrgEnv.path            Environment.
                   PrgEnv.path.old
                   PrgEnv.pgi.old
                   PrgEnv.pgi
                   PrgEnv.pgi.new
      2008-05-05   PrgEnv.pgi.new         Latest release of the PGI compiler   
                   PrgEnv.pgi-7.1.6       Suite.
      2007-12-05   PrgEnv.pgi	      New Programming Environment using
    	       PrgEnv.pgi-7.0.2       PGI Compiler Suite
      2007-11-19   PrgEnv.path-3.1        New Programming Environment using 
                                          PathScale 3.1 compilers.
      2007-09-10   PrgEnv.sun-2007-06     New Programming Environment using 
                                          Sun Studio 2007-06 compilers. 
    
    2009-10-21 Updates
    ===================
    During system maintenance on October 21, 2009 the PathScale and 
    Portland Group Programming Environments will be updated as follows
    
      Module Name      Now Aliased To          Previously Aliased To
      ---------------  -------------------    ----------------------
      PrgEnv.path.old  PrgEnv.path-3.1        PrgEnv.path-3.0
      PrgEnv.path      PrgEnv.path-3.2        PrgEnv.path-3.1
    
      PrgEnv.pgi.old   PrgEnv.pgi-7.1.6       PrgEnv.pgi-7.0.2
      PrgEnv.pgi       PrgEnv.pgi-7.2.2       PrgEnv.pgi-7.1.6
      PrgEnv.pgi.new   PrgEnv.pgi-8.0.6       PrgEnv.pgi-7.2.2
    
    
    2008-10-23 Updates
    ===================
    During system maintenance October 22, the PathScale and Portland Group
    Programming Environments were updated as follows:
    
      Module Name      Now Aliased To          Previously Aliased To
      ---------------  -------------------    ----------------------
      PrgEnv.old       PrgEnv.path.old        PrgEnv.path-2.5
      PrgEnv.path.old  PrgEnv.path-3.0        PrgEnv.path-2.5
      PrgEnv           PrgEnv.path            PrgEnv.path-3.0
      PrgEnv.path      PrgEnv.path-3.1        PrgEnv.path-3.0 
      PrgEnv.new       PrgEnv.path.new        PrgEnv.path-3.1
      PrgEnv.path.new  PrgEnv.path-3.2        PrgEnv.path-3.1
    
      PrgEnv.pgi.old   PrgEnv.pgi-7.0.2       nothing -- new module
      PrgEnv.pgi       PrgEnv.pgi-7.1.6       PrgEnv.pgi-7.0.2
      PrgEnv.pgi.new   PrgEnv.pgi-7.2.2       PrgEnv.pgi-7.1.6

    "PrgEnv"

    Last Updated: Wed, 22 Oct 2008 - Donald Bahls
    Machines: pingo
    Programming Environments on pingo
    ========================================
    Compilers on pingo are controlled via the modules package.  All
    accounts load the "PrgEnv-pgi" module by default.  This module 
    adds the PGI compilers with MPI, OpenMP and SHMEM support into
    the PATH.
    
    Should you experience problems with a compiler or library a newer 
    version may be available.
    
    Below is a description of available Programming Environments:
    
    Module Name        Description
    =================  ==============================================
    PrgEnv-pgi         Programming environment using the PGI
                       compilers and MPI stack (default module).
    PrgEnv-pathscale   Programming environment using the PathScale
                       compilers and MPI stack 
    PrgEnv-gcc         Programming environment using the GNU compilers 
                       and MPI stack.
    
    For a list of the latest available Programming Environments, run:
    
       pingo1 % module avail PrgEnv
    
       ---------------------- /opt/modulefiles --------------------------
       PrgEnv-gnu/2.1.24HD                PrgEnv-pathscale/2.1.24HD          
       PrgEnv-pgi/2.1.24HD                PrgEnv-gnu/2.1.27HD(default)       
       PrgEnv-pathscale/2.1.27HD(default) PrgEnv-pgi/2.1.27HD(default)
    
    
    Available Compute Node Compilers
    =================================
    Several compilers are available on pingo.  The following compilers can 
    be used to compile MPI, OpenMP, serial and SHMEM applications for use 
    on compute nodes.
    
       Module             C      C++    Fortran  Notes
       ================= ====== ====== ======== ============================
       PrgEnv-pgi        cc     CC     ftn      Supports MPI, OpenMP, Serial
       PrgEnv-pathscale  cc     CC     ftn      Supports MPI, OpenMP, Serial
       PrgEnv-gnu        cc     CC     ftn      Supports MPI, OpenMP, Serial
      
    Pre and Post processing applications which will be run on login nodes
    must be compile with one of the following compilers.
     
       Module             C      C++    Fortran  Notes
       ================= ====== ====== ======== ============================
       PrgEnv-pgi        pgcc   pgCC   pgf90    Supports Serial
       PrgEnv-pathscale  pathcc pathCC pathf90  Supports Serial
       PrgEnv-gnu        gcc    g++    gfortran Supports Serial
    
    
    Programming Environment Changes
    ================================
    The following is a table of recent additions and changes to the
    Programming Environment on pingo.
    
      Date         Module Name            Description
      ----------   ---------------------  -----------------------------------
    
    
    

    "archive"

    Last Updated: Tue, 28 Aug 2008 - Oralee Nudson
    Machines: linuxws seawolf
    Pending $ARCHIVE Changes
    ========================================
    During the first week of January 2009 all data currently in $ARCHIVE on
    nanook.arsc.edu will be moved to seawolf.arsc.edu in order to
    consolidate the ARSC long term storage infrastructure.
    
    There is a possibility that identically
    named files and/or directories could exist on both long term storage
    systems.  In cases where identically named files and/or directories
    are present, we are asking each user to determine which copy they wish
    to keep and to remove the copy they do not need.  If there are naming
    conflicts in the user's $ARCHIVE directory that are unresolved when
    data is moved in January, data with naming conflicts will be moved
    to a subdirectory called $ARCHIVE/nanook.
    
    NOTE: This change only affects users with data in the /archive/u1/uaf directory.
    Data in other directories has already been moved to seawolf.
    
    
    Frequently asked questions:
    
    1) Which systems use/used nanook.arsc.edu for the $ARCHIVE server?
       * SGI systems (retired)
       * Iceflyer (retired)
       * Linux Workstations - list of systems:
           http://www.arsc.edu/cgi/status.cgi?Page_mode=1
    
    2) Which systems use/used seawolf.arsc.edu for the $ARCHIVE server?
       * klondike.arsc.edu (retired)
       * iceberg.arsc.edu (retired)
       * cerebro.arsc.edu (retired)
       * nelchina.arsc.edu (retired)
       * midnight.arsc.edu
       * ognip.arsc.edu
       * pingo.arsc.edu
    
    3) Do I need to move my files from nanook.arsc.edu to seawolf.arsc.edu
       myself?
    
       No, we can perform this move in an automated fashion if there are
       no filename conflicts.  When possible, we prefer to move
       data in an automated fashion.
    
    4) How will I know if I have identicially named files on both $ARCHIVE
       filesystems?
    
       The ARSC help desk will contact users with identically named
       files and/or directories. We will request that the naming conflicted
       be resolved as soon as possible.
    
    5) What will happen if I have identically named files and/or directories
       when the $ARCHIVE move occurs in January?
    
       Any naming conflicts still present in January will be resolved by
       ARSC.  We will create a "nanook" subdirectory in the user's $ARCHIVE
       on seawolf.  The files and/or directories will be moved into the new
       "nanook" directory.
    
       E.g.
       If you had a directory called $ARCHIVE/data on both nanook and
       seawolf, the data directory formerly on nanook would be available
       on seawolf as $ARCHIVE/nanook/data.  The existing directory on
       seawolf would continue to be available as $ARCHIVE/data.
    
    6) How do I know whether or not my $ARCHIVE directory will be moved?
    
       If your $ARCHIVE directory is in the /archive/u1/uaf filesystem,
       any contents in your $ARCHIVE directory will be moved.  You can
       echo the value of $ARCHIVE to determine whether or not your $ARCHIVE
       directory is in the /archive/u1/uaf filesystem.
    
       e.g.
       mallard 3% echo $ARCHIVE
       /archive/u1/uaf/username
    
    
    
    If you have questions about this pending change, please contact the
    ARSC Help Desk.

    "compilers"

    Last Updated: Wed, 21 Jun 2006 - Donald Bahls
    Machines: linuxws
    Compilers
    ========================================
    The ARSC Linux Workstations have two suites of compilers available:
    
    * The GNU Compiler suite version 4.0 including:
      - gcc  C compiler
      - g++  C++ compiler
      - f95  Fortran 95 compiler
    
    * The Portland Group (PGI) compiler suite version 6.1 including:
      - pgcc C compiler 
      - pgCC C++ compiler
      - pgf90 Fortran 90 compiler
      - pgf95 Fortran 95 compiler
    
    
    The PGI compilers require several environment variables to be set: 
    
    For ksh/bash users:
    ===================
    export PGI=/usr/local/pkg/pgi/pgi-6.1
    pgibase=${PGI}/linux86-64/6.1 
    export PATH=$PATH:${pgibase}/bin
    if [ -z "$MANPATH" ]; then
        export MANPATH=${pgibase}/man
    else
        export MANPATH=${pgibase}/man:$MANPATH
    fi
    if [ -z "$LD_LIBRARY_PATH" ]; then
        export LD_LIBRARY_PATH=${pgibase}/lib
    else
        export LD_LIBRARY_PATH=${pgibase}/lib:$LD_LIBRARY_PATH
    fi
    unset pgibase
    
    For csh/tcsh users:
    ===================
    setenv PGI /usr/local/pkg/pgi/pgi-6.1/
    set pgibase=${PGI}/linux86-64/6.1
    setenv PATH ${PATH}:${pgibase}/bin
    if ( ! ${?MANPATH} ) then
        setenv MANPATH ${pgibase}/man
    else
        setenv MANPATH ${pgibase}/man:${MANPATH}
    endif
    if ( ! ${?LD_LIBRARY_PATH} ) then
        setenv LD_LIBRARY_PATH ${pgibase}/lib
    else
        setenv LD_LIBRARY_PATH ${pgibase}/lib:${LD_LIBRARY_PATH} 
    endif   
    
    unset pgibase
    

    "eslogin"

    Last Updated: Mon, 28 Dec 2009 - Craig Stephenson
    Machines: pingo
    eslogin Nodes
    =============
    
    The esLogin nodes (external service Login) provide additional functionality
    for users needing resources beyond what is available on the standard login 
    nodes.
    
    The configuration of these nodes is as follows:
    
    	- 128GB shared memory
    	- 4x quad core AMD Opteron processors (for a total of 16 cores)
    	- 1x 10 Gigabit Ethernet Interconnect
    
    Currently there are two esLogin nodes available, and they can be accessed
    with the following addresses:
    
    	- pingob.arsc.edu
    	- pingoc.arsc.edu
    
    Also, due to the increased resources available on these nodes, it has allowed 
    for the increase of available limits on the nodes.  Memory limits on these nodes
    are currently set at 4GB for soft limits, and 16GB for hard limits.  If you 
    require additional resources, please contact ARSC Support.
    
    Limitations
    ===========
    
    Currently, there are a couple of differences between the esLogin nodes and the
    standard login nodes.  The esLogin nodes do not have the following capabilities:
    
    	- The Pathscale compiler suite is unavailable on esLogin nodes
    	- Totalview debugging is currently unavailable from the esLogin nodes
    	- Some library versions differ.  If you have dynamically linked 
    	  executables, there may be some inconsistency between the node types.
    	- Some packages available on the standard login nodes may not be 
    	  available. If there is a package you require to complete work, and it
    	  is not available, please contact us.
    	
    Questions
    =========
    
    If you have any questions regarding the capability of the esLogin nodes, or 
    experience any difficulty while using them, please contact the ARSC Support 
    staff and we will do our best to work through any issues you may experience.

    "file_striping"

    Last Updated: Wed, 03 Mar 2010 - Oralee Nudson
    Machines: pingo
    File Striping
    ========================================
    The $WORKDIR Lustre filesystem on midnight is composed of multiple object 
    storage targets (OSTs).   The default filesystem settings place each file on 
    one OST.  These settings are appropriate for many applications, however some 
    applications may benefit by enabling striping.  Striping places a file on 
    multiple OSTs.  This can:
    
    * Provide better bandwidth to files accessed by many tasks simultaneously.
    * Increase read and write performance for I/O intensive codes.
    * More evenly distribute large files across OSTs.  Striping should be 
      considered for any file larger than 50 GB.
    
    The following command shows how to enable striping across 6 OSTs for an empty 
    directory:
    
    lfs setstripe $WORKDIR/case1 -s 0 -i -1 -c 6
    
    NOTE: All new files created in the $WORKDIR/case1/ directory will take on 
    these striping attributes.  Files moved within the same filesystem or files 
    that already exist within the directory will maintain their existing striping 
    characteristics.
    
    See "lfs setstripe" for a description of striping parameters.
    
    The following set of commands shows how to stripe a directory across 6 OSTs 
    and how to copy an existing large file into that directory (so the large file 
    will also be striped.)
    
    cd $WORKDIR/case1
    mkdir striped
    lfs setstripe striped -s 0 -i -1 -c 6
    cp largeFile.txt striped/largeFile.txt
    # confirm largeFile.txt data looks normal
    rm largeFile.txt
    
    To confirm the file striping was successful, check the particular file
    or directory with the following commands:
    
    lfs getstripe filename
    lfs getstripe directoryname
    
    
    
    

    "file_striping"

    Last Updated: Mon, 26 Jan 2009 - Oralee Nudson
    Machines: midnight
    File Striping
    ========================================
    The $WORKDIR Lustre filesystem on midnight is composed of multiple object 
    storage targets (OSTs).   The default filesystem settings place each file on 
    one OST.  These settings are appropriate for many applications, however some 
    applications may benefit by enabling striping.  Striping places a file on 
    multiple OSTs.  This can:
    
    * Provide better bandwidth to files accessed by many tasks simultaneously.
    * Increase read and write performance for I/O intensive codes.
    * More evenly distribute large files across OSTs.  Striping should be 
      considered for any file larger than 50 GB.
    
    The following command shows how to enable striping across 6 OSTs for an empty 
    directory:
    
    lfs setstripe $WORKDIR/case1 0 -1 6
    
    NOTE: All new files created in the $WORKDIR/case1/ directory will take on 
    these striping attributes.  Files moved within the same filesystem or files 
    that already exist within the directory will maintain their existing striping 
    characteristics.
    
    See "lfs setstripe" for a description of striping parameters.
    
    The following set of commands shows how to stripe a directory across 6 OSTs 
    and how to copy an existing large file into that directory (so the large file 
    will also be striped.)
    
    cd $WORKDIR/case1
    mkdir striped
    lfs setstripe striped 0 -1 6
    cp largeFile.txt striped/largeFile.txt
    # confirm largeFile.txt data looks normal
    rm largeFile.txt
    
    To confirm the file striping was successful, check the particular file
    or directory with the following commands:
    
    lfs getstripe filename
    lfs getstripe directoryname
    
    

    "filezilla"

    Last Updated: Mon, 28 Dec 2009 - Craig Stephenson
    Machines: pingo
    Filezilla
    =========
    
    This news item is for users of the Filezilla ftp client which is
    included with the HPCMP Kerberos kit for Windows.  
    
    Some users have reported problems connecting to the Linux Workstations
    with the Filezilla client.  Should you experience these problems 
    with Filezilla, try the following:
    
    Option 1) Set the transfer mode to passive:
      a) Select the menus "Edit | Settings" 
      b) Select the tree option "Firewall Settings " 
      c) Then check the passive mode check box
    
    Option 2) Set the ftp protocol to sftp:
      a) Select the menus "File | Site Manager" 
      b) Click "New Site" button.  Enter the machine name.  
      c) Fill in the "host" (e.g. iceberg.arsc.edu) 
      d) Set "Servertype" to "SFTP using SSH2" 
      e) Set "Logontype" to "Normal" 
      f) Set the "User" to your username.  
      g) Click the "Connect" button to connect.
    
    Sites entered in the Site Manager can be also be selected using the
    leftmost icon on the Filezilla toolbar.  
    
    If you experience problems with the Filezilla client using these 
    recommendations, please report them to the ARSC help desk 
    (consult@arsc.edu).

    "filezilla"

    Last Updated: Mon, 28 Dec 2009 - Craig Stephenson
    Machines: linuxws
    Filezilla
    =========
    
    This news item is for users of the Filezilla ftp client which is
    included with the HPCMP Kerberos kit for Windows.  
    
    Some users have reported problems connecting to the Linux Workstations
    with the Filezilla client.  Should you experience these problems 
    with Filezilla, try the following:
    
    Option 1) Set the transfer mode to passive:
      a) Select the menus "Edit | Settings" 
      b) Select the tree option "Firewall Settings " 
      c) Then check the passive mode check box
    
    Option 2) Set the ftp protocol to sftp:
      a) Select the menus "File | Site Manager" 
      b) Click "New Site" button.  Enter the machine name.  
      c) Fill in the "host" (e.g. iceberg.arsc.edu) 
      d) Set "Servertype" to "SFTP using SSH2" 
      e) Set "Logontype" to "Normal" 
      f) Set the "User" to your username.  
      g) Click the "Connect" button to connect.
    
    Sites entered in the Site Manager can be also be selected using the
    leftmost icon on the Filezilla toolbar.  
    
    If you experience problems with the Filezilla client using these 
    recommendations, please report them to the ARSC help desk 
    (consult@arsc.edu).
    

    "filezilla"

    Last Updated: Thu, 14 May 2009 - Craig Stephenson
    Machines: midnight
    Filezilla
    =========
    
    This news item is for users of the Filezilla ftp client which is
    included with the HPCMP Kerberos kit for Windows.  
    
    Some users have reported problems connecting to Midnight
    with the Filezilla client.  Should you experience these problems 
    with Filezilla, try the following:
    
    Option 1) Set the transfer mode to passive:
      a) Select the menus "Edit | Settings" 
      b) Select the tree option "Firewall Settings " 
      c) Then check the passive mode check box
    
    Option 2) Set the ftp protocol to sftp:
      a) Select the menus "File | Site Manager" 
      b) Click "New Site" button.  Enter the machine name.  
      c) Fill in the "host" (e.g. midnight.arsc.edu) 
      d) Set "Servertype" to "SFTP using SSH2" 
      e) Set "Logontype" to "Normal" 
      f) Set the "User" to your username.  
      g) Click the "Connect" button to connect.
    
    Sites entered in the Site Manager can be also be selected using the
    leftmost icon on the Filezilla toolbar.  
    
    If you experience problems with the Filezilla client using these 
    recommendations, please report them to the ARSC help desk 
    (consult@arsc.edu).

    "idl"

    Last Updated: Tue, 25 Apr 2006 - Andrew Lee
    Machines: linuxws
    IDL at ARSC
    ========================================
    IDL is installed at ARSC on the Linux workstations and iceberg. 
    
    To run, you must first make sure the following two steps are done:
    
    1)    Files named idl_setup and idl_setup.ksh are
           automatically created by the installation script.  These
           files contain appropriate definitions for IDL-related
           environment variables and aliases. Users must set up their 
           accounts as shown below:
    
               C-shell users add the line:
    
                   source /usr/local/pkg/idl/idl-6.2/idl/bin/idl_setup
                   source /usr/local/pkg/idl/idl-6.2/envi/bin/envi_setup
    
               to your .cshrc files.
    
               Korn shell users add the line:
    
                  . /usr/local/pkg/idl/idl-6.2/idl/bin/idl_setup.ksh
                   . /usr/local/pkg/idl/idl-6.2/envi/bin/envi_setup.ksh
    
               to your .profile files.
    
               Bash shell users add the line:
    
                   . /usr/local/pkg/idl/idl-6.2/idl/bin/idl_setup.bash
                   . /usr/local/pkg/idl/idl-6.2/envi/bin/envi_setup.bash
    
               to your .bashrc files.
    
    
    2) Make sure the license variable is set correctly, if not add this
    line to the same file as listed above for your shell:
    
    C-shell (csh, tcsh)
     setenv LM_LICENSE_FILE 7111@license1.arsc.edu:7111@license.arsc.edu
    
    Korn shell (ksh)
     export LM_LICENSE_FILE=7111@license1.arsc.edu:7111@license.arsc.edu
    
    Bash shell (bash)
     export LM_LICENSE_FILE=7111@license1.arsc.edu:7111@license.arsc.edu
    
    3) Some users have reported malloc errors with the 64 bit version
    on envi and idl.  We are currently investigating these errors.
    
    Should you encounter a malloc error try using the 32 bit version of
    idl or envi.
    
         The "-32" flag will invoke the 32 bit executable:
    
         e.g.
          % idl -32
    

    "idle"

    Last Updated: Wed, 01 Feb 2006 - Andrew Lee
    Machines: seawolf
    Idle Time-outs
    ========================================
      As of February 1, 2006, ARSC has instituted time limits for login sessions to many of
    our systems.  In order to minimize the exposure of your login sessions to 'session
    hijacking' vulnerabilities, we are limiting the total login time to 24 hours.   All logins will
    has been open for more than 6 hours, it will be terminated when it has been idle for more
    than 4 consecutive hours within that 24 hr period.    All login sessions will be terminated
    once they have reached 24 hours -- regardless of if there is activity at the 24 hour mark or
    not.  Please see:  http://www.arsc.edu/support/policy/secpolicy.html or contact the ARSC
    Help Desk (consult@arsc.edu, 907-450-8602) for more information.
    

    "kerberos"

    Last Updated: Wed, 22 Apr 2009 - Donald Bahls
    Machines: linuxws midnight pingo seawolf
    Required Kerberos Updates
    ========================================
    Recently MIT announced two vulnerablities in Kerberos.  Due to the
    nature of these vulnerablities, ARSC and HPCMP are requiring all
    kerberos clients to be updated to the latest release by May 22nd, 2009.
    The following versions of Kerberos or newer will be the only supported
    versions beyond May 22nd:
    
     * HPCMP_RELEASE_20090331 (Linux / Unix / Mac OS X) 
       and
     * HPCMP_RELEASE_20090422 (Windows).
    
    Updated clients are available from the HPCMP Kerberos website.
    Precompiled clients can be found under the "Software" link on the
    left of this page.
    
       https://www.hpcmo.hpc.mil/security/kerberos/ 
    
    
    The known vulnerability also affects HPCMP OpenSSH client kits.
    All versions older than OpenSSH-5.0p1c will not be allowed after
    May 22nd.  Updated versions of the HPCMP OpenSSH client kits are also
    available at the HPCMP Kerberos website.
    
    
    Frequently Asked Questions:
    
    1) How can I tell which version of Kerberos I have installed?
    
       a) For Linux, Solaris and Mac OS X:
    
       # run the krlogin command with the -V option.
       % krlogin -V
       usage: rlogin host [-option] [-option...] [-k realm ] [-t ttytype] [-l username]
            where option is e, 7, 8, noflow, n, a, x, f, F, c, PO, or PN
       
       version: @(#) $NRL: HPCMP_RELEASE_20081006 20081006 $
    
    
       The version of this kit is HPCMP_RELEASE_20081006.
    
       b) For Windows:
    
       Start the PuTTY client, then select the "About" button in the lower
       left corner.  The HPCMP_RELEASE_20090409 release should indicate 
       "PuTTY HPCMP Release 0.60" or newer is being used.
    
    
    2) How can I check the version of ssh I have installed?
    
       a) For Linux, Solaris and Mac OS X
    
       % ssh -V
       OpenSSH_5.2p1a, OpenSSL 0.9.8k 25 Mar 2009
    
       b) For Windows 
    
       (See question 1.b above)
    
    
    3) I'm not using Linux, Solaris, Mac OS X or Windows, how can I
       obtain a kit for my Operating System?
    
       Client kits for operating systems not listed above must be 
       compiled from source.  Kerberos source code is available from:
    
       https://www.hpcmo.hpc.mil/security/kerberos/ 
    
    4) Are there a direct links to the client kits available? 
    
       Yes.
    
       a) Linux and Solaris
          https://www.hpcmo.hpc.mil/security/kerberos/software/clients/kerberos/index.html 
    
       b) Windows
          https://www.hpcmo.hpc.mil/security/kerberos/software/clients/kerberos/indexWin.html 
    
       c) Macintosh
          https://www.hpcmo.hpc.mil/security/kerberos/software/clients/kerberos/indexMac.html
    

    "largefile"

    Last Updated: Sat, 13 Feb 2010 - Don Bahls
    Machines: seawolf
    Large File Support
    ========================================
    
    Some commands on seawolf do not have support for files larger that 2 GB.  
    See "man largefile" for a list of commands which have large file support.  
    In particular the "gzip" and "bzip2" are 32 bit executables on seawolf and 
    therefore may truncate large files.
    
    If you need to manipulate files larger than 2 GB, it is recommended that an
    HPC system or Linux Workstation be used.  
    
    If you have questions, please contact User Support.

    "matlab"

    Last Updated: Wed, 20 Jan 2010 - Don Bahls
    Machines: midnight
    Matlab License Changes
    ========================================
    
    Several changes to matlab licensing are occurring in order to provide
    better support for licensed product through Advanced Reservations.
    These changes will require users of matlab to make changes to their
    normal workflow as described below.
    
    This change in policy will officially go into effect on January
    26th, 2010.
    
    
    Jobs Run Through PBS:
    =====================
    On January 20th, 2010 a PBS resource will be added for matlab.
    This resource will ensure that licenses are available prior to the
    start of a PBS job.  Matlab requires one license per node used by
    the job using the matlab resource (e.g. #PBS -l matlab=1):
    
    #!/bin/bash
    #PBS -l walltime=1:00:00
    #PBS -l matlab=1
    #PBS -l select=1:ncpus=4:node_type=4way
    #PBS -q standard
    #PBS -j oe
    
    cd $PBS_O_WORKDIR
    matlab < input.m
    
    
    Matlab Changes for Login Nodes:
    ================================
    All matlab work must request license reservations via the
    "matlab_license_check" script.  This script requires that you specify
    the length of time you intend to run matlab.  Any work requesting
    more than 1 hour should be run through PBS as described above.
    
    e.g. 
    # Request a 60 minute reservation
    mg56 % /usr/local/bin/matlab_license_check -w 60
    Reservation successfully created!
    
    module load matlab-7.8.0
    matlab < input.m
    
    
    Running Matlab On Compute Nodes:
    =================================
    Matlab can be run on compute nodes with full access to the GUI by using the
    following technique:
    
    A) Log into midnight1 or midnight2 with X11 forwarding enabled.
    
       % ssh -X -Y username@midnight.arsc.edu
    
    B) Run the "tunnelx" command to setup X11 forwarding for the 
       compute nodes.
          
       midnight% tunnelx 
    
    C) Start an interactive PBS job requesting matlab:
    
       midnight% qsub -l select=1:ncpus=1:node_type=4way -lmatlab=1 -I
    
    D) Once the interactive PBS job starts, rerun the "tunnelx"
       command to initialize the X11 settings for your shell.
          
       mt101% tunnelx
          
       Next, source the file ~/.tx_local which was generated by the tunnelx
       command.
          
       # bash/ksh syntax
       mt101% . ~/.tx_local
         
       # csh/tcsh syntax
       mt101% source ~/.tx_local          
    
    E) You should be able to run matlab with X11 support.
    
       mt101% module load matlab-7.6.0
       mt101% matlab
    
    F) When you are done using matlab, be sure to run "exit" to end the interactive
       PBS job.
    
    

    "modules"

    Last Updated: Mon, 28 Dec 2009 - Craig Stephenson
    Machines: pingo
    Using the Modules Package
    =========================
    
    The modules package is used to prepare the environment for various
    applications before they are run.  Loading a module will set the
    environment variables required for a program to execute properly.
    Conversely, unloading a module will unset all environment variables
    that had been previously set.  This functionality is ideal for
    switching between different versions of the same application, keeping
    differences in file paths transparent to the user.
    
    
    Sourcing the Module Init Files
    ---------------------------------------------------------------------
    For some jobs, it may be necessary to source these files, as they 
    may not be sourced automatically as with login shells.
     
    Before the modules package can be used, its init file must first be
    sourced.
    
    To do this using tcsh or csh, type:
    
       source /opt/modules/default/init/
    
    To do this using bash, ksh, or sh, type:
    
       . /opt/modules/default/init/
    
    For either case, replace  with the shell you are using.
    If your shell is bash, for example:
    
       . /opt/modules/default/init/bash
    
    Once the modules init file has been sourced, the following commands
    become available:
    
    Command                     Purpose
    ---------------------------------------------------------------------
    module avail                - list all available modules
    module load            - load a module file from environment
    module unload          - unload a module file from environment
    module list                 - display modules currently loaded
    module switch     - replace module  with module 
    module purge                - unload all modules
    

    "modules"

    Last Updated: Mon, 28 Dec 2009 - Craig Stephenson
    Machines: linuxws
    Using the Modules Package
    =========================
    
    The modules package is used to prepare the environment for various 
    applications before they are run.  Loading a module will set the 
    environment variables required for a program to execute properly.  
    Conversely, unloading a module will unset all environment variables 
    that had been previously set.  This functionality is ideal for 
    switching between different versions of the same application, keeping 
    differences in file paths transparent to the user.
    
    The following modules commands are available:
    
    module avail                - list all available modules
    module load            - load a module file from environment
    module unload          - unload a module file from environment
    module list                 - display modules currently loaded
    module switch     - replace module  with module 
    module purge                - unload all modules
    
    Before the modules package can be used in a script, its init file may
    need to be sourced.
    
    To do this using tcsh or csh, type:
    
       source /usr/local/pkg/modules/init/
    
    To do this using bash, ksh, or sh, type:
    
       . /usr/local/pkg/modules/init/
    
    For either case, replace  with the shell you are using.  
    If your shell is bash, for example:
    
       . /usr/local/pkg/modules/init/bash
    
    
    Known Issues:
    =============
    
    2009-09-24  Accounts using bash that were created before 9/24/2009
                are missing the default ~/.bashrc file.  This may cause
                the module command to be unavailable in some instances.
    
                Should you experience this issue run the following:
    
                # copy the template .bashrc to your account.
                [ ! -f ~/.bashrc ] && cp /etc/skel/.bashrc ~
    
                If you continue to experience issues, please contact the 
                ARSC Help Desk.
    

    "modules"

    Last Updated: Fri, 30 Oct 2008 - Donald Bahls
    Machines: midnight
    Using the Modules Package
    =========================
    
    The modules package is used to prepare the environment for various
    applications before they are run.  Loading a module will set the
    environment variables required for a program to execute properly.
    Conversely, unloading a module will unset all environment variables
    that had been previously set.  This functionality is ideal for
    switching between different versions of the same application, keeping
    differences in file paths transparent to the user.
    
    
    Sourcing the Module Init Files
    ---------------------------------------------------------------------
    For some jobs, it may be necessary to source these files, as they 
    may not be auto-sourced as with login shells.
     
    Before the modules package can be used, its init file must first be
    sourced.
    
    To do this using tcsh or csh, type:
    
       source /usr/share/modules/init/
    
    To do this using bash, ksh, or sh, type:
    
       . /usr/share/modules/init/
    
    For either case, replace  with the shell you are using.
    If your shell is bash, for example:
    
       . /usr/share/modules/init/bash
    
    Once the modules init file has been sourced, the following commands
    become available:
    
    Command                     Purpose
    ---------------------------------------------------------------------
    module avail                - list all available modules
    module load            - load a module file from environment
    module unload          - unload a module file from environment
    module list                 - display modules currently loaded
    module switch     - replace module  with module 
    module purge                - unload all modules

    "mpi"

    Last Updated: Wed, 30 Apr 2008 - Donald Bahls
    Machines: midnight
    MPI Environment
    ========================================
    Midnight uses the Voltaire versions of the MVAPICH MPI stack.  A list of 
    MVAPICH specific features is available on the ARSC webpage here:
    
      http://www.arsc.edu/support/howtos/usingsun.html#mpi
    
    
    Changes to MPI Stack
    ====================
    The following are a list of changes to the MPI stack on midnight.
    
    2008-05-19:
    Task affinity is now enabled by default for MPI jobs.  It can be disabled
    by using "-noaff".  See 2008-04-30 changes for details.
    
    
    2008-04-30:
    The mpirun command now includes an option to enable task affinity (-aff).  
    This option helps decrease the variability in run time for many applications,
    however it should not be used for hybrid applications (e.g. MPI/OpenMP).  
    The "-noaff" option can be used do disable task affinity.  This option should
    be used for hybrid MPI/OpenMP or MPI/pthread applications.
    
    Task affinity will be enabled for all MPI applications not specify "-noaff" 
    during maintenance on May 17-18th.  
    
      
    
    
    MPI Feature Support
    ===================
    The following MPI features are not currently supported on midnight.
    
      1) MPI-2 and MPI-IO.  MPI-2 features including MPI-IO are not currently 
         available on midnight.
    
      2) MPMD Support.  The MVAPICH stack does not support the multiple program
         multiple data (MPMD) programming model used by some codes (e.g. CCSM).

    "pbs_10"

    Last Updated: Wed, 13 May 2009 - Donald Bahls
    Machines: midnight
    PBS Pro 10 Upgrade
    ========================================
    During scheduled maintenance on May 16th - 18th, PBS Pro will
    be upgraded from version 7.1.3 to version 10.0.   Most users will not
    need to make changes to job scripts to ensure unaltered operation of
    those scripts.  There are several exceptions listed below.
    
    1) Node Placement:
       Jobs using 1/2 of the cores on a node or less will need to
    include a placement statement to ensure the proper number of cores
    per node are used.
    
       e.g.,
    
       # use 4 nodes with 2 cores per node
       #PBS -l select=4:ncpus=2:node_type=4way
       # place only one set of 2 tasks per node.
       #PBS -l place=scatter:excl
       ...
    
    
    2) Backfill:
       The PBS scheduler will be updated to use job based priorities and
    eligible time with backfill.  Jobs will gain priority based on the time
    the job is in the queue and eligible to run.  If a job can run without
    interfering with the start time of the highest priority job, that job
    will be eligible for backfill.  Jobs specifying shorter walltimes
    and smaller processor counts generally have more opportunities to
    take advantage of backfill windows.  Significant overestimates in
    walltime requests decrease the likelihood backfill can occur.
    
       e.g.,
       # Example of limits which may backfill nicely
       #PBS -l select=4:ncpus=4:node_type=4way
       #PBS -l walltime=2:00:00
       ...
    
    
    3) Use of the -V flag:
    
       Following the upgrade to PBS Pro 10, it was determined that the -V
    option can cause PBS to work improperly under certain circumstances.
    
    The following workarounds can be used:
    
      a) Use the -v (small v) option to set individual environment variables
         during job submission:
    
         e.g.
         qsub -v VARIABLE=value myscript.pbs
    
      b) If the variable value is static, you can add the required
         environment variables to your ~/.profile or ~/.login 
    
      c) Unset unneeded environment variables prior to submitting your PBS 
         script:
    
         # bash/ksh
         unset LM_LICENSE_FILE   #unset license file string
         qsub -V myscript.pbs
    
         # csh/tcsh
         unsetenv LM_LICENSE_FILE #unset license file string
         qsub -V myscript.pbs
     
    
    4) Script names not starting with a letter.
    
    Script names not starting with a letter may not function properly
    at runtime.
    
    The following workarounds can be used:
    
        Rename the script so the first character is a letter rather than a 
        number or symbol.
    
          e.g.
          mv 2test.pbs a2test.pbs
    
    
    If you experience issues not described by this news item, please
    contact User Support.  The following command will provide contact
    information:
    
         % support
    

    "pin"

    Last Updated: Fri, 27 Aug 2007 - Derek Bastille
    Machines: midnight seawolf
    New SecurID card PINs to be 5 digit
    ========================================
    In order to improve the security and effectiveness of our authentication
    system, ARSC will begin issuing 5 digit SecurID card PINs as of 1
    May, 2007.  Existing SecurID cards will NOT have their PINs updated
    at this time.  However, if a PIN reset is needed, the replacement
    PIN will be 5 digit.  All new SecurID cards issued after 1 May will
    be set to 5 digit PINs.  If you have any questions please contact
    the ARSC Help Desk.

    "projects"

    Last Updated: Wed, 17 Dec 2008 - Donald Bahls
    Machines: midnight
    Instructions for Users with Multiple Projects
    ========================================
    This news item is intended for users that are members of more than one
    project.  Users in a single project will automatically have use 
    charged against the allocation for their primary group (i.e. project).
    
    Users in more than one project can select an alternate project to 
    charge use to by using the "-W group_list" PBS option.  If the 
    "-W group_list" option is not specified the account number will default 
    to your primary group (i.e. project).
    
    Below is an example "-W group_list" statement.
    
       e.g.
       #PBS -W group_list=proja
    
    The "-W group_list" option can also be used on the command line.
    
       e.g.
       mg56 % qsub -Wgroup_list=proja script.bat
    
    Each project has a corresponding UNIX group, therefore the groups 
    command will show all projects and other groups of which you are a 
    member. 
    
       e.g.
       mg56 % groups
       proja projb
    
    In this case use would be charged to proja by default, but could be 
    charged to projb by setting "-W group_list=projb" in the PBS script.
    
    If you have questions about this news item, please contact the ARSC
    help desk (consult@arsc.edu).
    
    
    
    'show_usage' Available
    =======================
    Project utilization information is now available via the BCT Compliant 'show_usage' 
    command.  'show_usage' with no parameters will display the remaining 
    allocation for each project of which you are a member. 
    
        e.g.
        mg56 % show_usage
    
                      ARSC - Subproject Usage Information (in CPU Hours)
                               As of 01:11:21 hours ADT 11 Nov 2007
                    For Fiscal Year 2008 (01 October 2007 - 30 September 2008)
                            Percentage of Fiscal Year Remaining: 88.77% 
    
                                    Hours      Hours      Hours      Percent  Background
          System     Subproject     Allocated  Used       Remaining  Remaining Hours Used
          ========== ============== ========== ========== ========== ========= ==========
          midnight   projectA        100000.00   39663.48   60336.52    60.34%       0.00
          midnight   projectB          5000.00       0.00    5000.00   100.00%       0.00
    
    
    Project utilization for all allocated systems can be obtained by using 'show_usage'
    with the '-s' option as in 'show_usage -s'.
    
          mg56 % show_usage -s
    
                      ARSC - Subproject Usage Information (in CPU Hours)
                               As of 13:00:00 hours ADT 11 Nov 2007
                   For Fiscal Year 2008 (01 October 2007 - 30 September 2008)
                            Percentage of Fiscal Year Remaining: 88.77% 
    
                                    Hours      Hours      Hours      Percent  Background
          System     Subproject     Allocated  Used       Remaining  Remaining Hours Used
          ========== ============== ========== ========== ========== ========= ==========
          iceberg    projectA         30000.00     309.84   29690.16    98.97%       0.00
          iceberg    projectB         10000.00    3873.46    6126.54    61.27%       0.00
          midnight   projectA        100000.00   39663.48   60336.52    60.34%       0.00
          midnight   projectB          5000.00       0.00    5000.00   100.00%       0.00
    

    "projects"

    Last Updated: Wed, 17 Dec 2008 - Donald Bahls
    Machines: pingo
    Instructions for Users with Multiple Projects
    ========================================
    This news item is intended for users that are members of more than one
    project.  Users in a single project will automatically have use 
    charged against the allocation for their primary group (i.e. project).
    
    Users in more than one project can select an alternate project to
    charge use to by using the "-W group_list" PBS option.  If the "-W
    group_list" option is not specified the account number will default
    to your primary group (i.e. project).
    
    Below is an example "-W group_list" statement.
    
     e.g.
     #PBS -W group_list=proja
    
    The "-W group_list" option can also be used on the command line.
    
     e.g.
     pingo4 % qsub -Wgroup_list=proja script.bat
    
    Each project has a corresponding UNIX group, therefore the groups 
    command will show all projects and other groups of which you are a 
    member. 
    
     e.g.
     pingo4 % groups
     proja projb
    
    In this case use would be charged to proja by default, but could be 
    charged to projb by setting "-W group_list=projb" in the PBS script.
    
    If you have questions about this news item, please contact user
    support.
    
    
    'show_usage' Available
    =======================
    Project utilization information is now available via the BC Compliant
    'show_usage' command.  'show_usage' with no parameters will display
    the remaining allocation for each project of which you are a member.
    
     e.g.
     pingo4 % show_usage
    
                 ARSC - Subproject Usage Information (in CPU Hours)
                          As of 01:11:21 hours ADT 11 Nov 2007
               For Fiscal Year 2008 (01 October 2007 - 30 September 2008)
                       Percentage of Fiscal Year Remaining: 88.77% 
    
                                 Hours      Hours      Hours      Percent  Background
     System     Subproject     Allocated  Used       Remaining  Remaining Hours Used
     ========== ============== ========== ========== ========== ========= ==========
     pingo      projectA        100000.00   39663.48   60336.52    60.34%       0.00
     pingo      projectB          5000.00       0.00    5000.00   100.00%       0.00
    
    
    Project utilization for all allocated systems can be obtained by using
    'show_usage' with the '-s' option as in 'show_usage -s'.
    
     pingo4 % show_usage -s
    
                 ARSC - Subproject Usage Information (in CPU Hours)
                          As of 13:00:00 hours ADT 11 Nov 2007
              For Fiscal Year 2008 (01 October 2007 - 30 September 2008)
                       Percentage of Fiscal Year Remaining: 88.77% 
    
                               Hours      Hours      Hours      Percent  Background
     System     Subproject     Allocated  Used       Remaining  Remaining Hours Used
     ========== ============== ========== ========== ========== ========= ==========
     pingo      projectA         30000.00     309.84   29690.16    98.97%       0.00
     pingo      projectB         10000.00    3873.46    6126.54    61.27%       0.00
     midnight   projectA        100000.00   39663.48   60336.52    60.34%       0.00
     midnight   projectB          5000.00       0.00    5000.00   100.00%       0.00
    
    

    "queues"

    Last Updated: Fri, 02 Mar 2009 - Oralee Nudson
    Machines: midnight
    Midnight Queues
    ===============
    
    The queue configuration is as described below.  It is subject to
    review and further updates.
    
    
       Login Nodes Use:
       =================
       Login nodes are a shared resource and are not intended for
       computationally or memory intensive work.  Processes using more
       than 30 minutes of CPU time on login nodes may be killed by ARSC
       without warning.  Please use compute nodes for computationally or
       memory intensive work.
    
    
       Queues:
       ===============
       Specify one of the following queues in your PBS qsub script
       (e.g., "#PBS -q standard"):
    
         Queue Name     Purpose of queue
         -------------  ------------------------------
         standard       General use by all allocated users.
         debug          Quick turnaround queue for debugging work.
         data           Provides access to long term storage (i.e. $ARCHIVE_HOME) 
         background     For projects with little or no remaining allocation. 
                        This queue has the lowest priority, however projects
                        running jobs in this queue do not have allocation deducted.      
                        The number of running jobs or processors available to this
                        queue may be altered based on system load.
         special        For jobs which do not fit into normal queue limits. (Limited Access)
         high           For jobs requiring higher priority than normal. (Special Access Only)
         urgent         For jobs requiring highest priority. (Special Access Only)
         challenge      For challenge projects only. 
    
       See 'qstat -q' for a complete list of system queues.  Note, some 
       queues are not intended for general use.
    
    
       Maximum Walltimes:
       ===================
       The maximum allowed walltime for a job is dependant on the number of 
       processors requested.  The table below describes maximum walltimes for 
       each queue.
    
       Queue             Min   Max     Max       
                        Procs Procs  Walltime Notes
       ---------------  ----- ----- --------- ------------
       standard      
                            1    16  84:00:00  See (A)
                           17   256  16:00:00    
                          257   512  12:00:00
       challenge
                            1    16  96:00:00  See (B)
                           17   256  48:00:00  See (F)
                          257   516  48:00:00  See (G)
    
       background           1   512  12:00:00  
     
       debug                1    32  01:00:00  See (D)
       data                 1     1  12:00:00  See (E)
    
       NOTES:
       (A) The max walltime for standard_sm jobs (i.e. 1 to 16 processors) was 
           increased from 42 hours to 84 hours on October 5th 2007.  
       (B) The max walltime for challenge_sm jobs (i.e. 1 to 16 processors) was 
           increased from 48 hours to 96 hours on October 5th 2007.  
       (C) The max walltime for challenge_med jobs (i.e. 17 to 256 processors) was 
           increased from 24 hours to 96 hours on November 28th 2007.  
       (D) As of January 2009 the debug queue now has 4 dedicated nodes for
           quick turn around debugging work.  2- 4way nodes and 4- 16way nodes.
       (E) Do not specify a node_type when submitting jobs to the data queue.
           Please refer to the sample PBS data staging script below for an example.
       (F) The max walltime for challenge_med jobs (i.e. 17 to 256 processors) was 
           decreased from 96 hours to 48 hours on September 25, 2008.
       (G) The max number of CPUs was increased from 512 to 516 on August 19, 2009.
           The max walltime was increased from 12 hours to 48 hours on August 21, 2009.
    
       PBS Commands:
       =============
       Below is a list of common PBS commands.  Additional information is
       available in the man pages for each command.
    
       Command         Purpose
       --------------  -----------------------------------------
       qsub            submit jobs to a queue
       qdel            delete a job from the queue   
       qsig            send a signal to a running job
       qmap            display a map of the system (ARSC only)
    
    
       Running a Job:
       ==============
       To run a batch job, create a qsub script which, in addition to
       running your commands, specifies the processor resources and time
       required.  Submit the job to PBS with the following command.   (For
       more PBS directives, type "man qsub".)
    
         qsub <script file>
    
       Sample PBS scripts:
       --------------
       #### Beginning of MPI Example Script using Sun Fire x2200 nodes ############
       #!/bin/bash
       #PBS -q standard 
       #PBS -l select=8:ncpus=4:node_type=4way
       #PBS -l walltime=8:00:00
       #PBS -j oe
    
       cd $PBS_O_WORKDIR
    
       mpirun -np 32 ./myprog
       
       #### End of Sample Script  ##################
    
       #### Beginning of MPI Example Script using Sun Fire x4600 nodes ############
       #!/bin/bash
       #PBS -q standard
       #PBS -l walltime=8:00:00
       #PBS -l select=2:ncpus=16:node_type=16way
       #PBS -j oe
    
       cd $PBS_O_WORKDIR
    
       mpirun -np 32 ./myprog
       #### End of Sample Script  ##################
    
       #### Beginning of OpenMP Example Script using Sun Fire x4600 nodes ############
       #!/bin/bash
       #PBS -q standard
       #PBS -l select=1:ncpus=16:node_type=16way
       #PBS -l walltime=8:00:00
       #PBS -j oe
    
       cd $PBS_O_WORKDIR
       export OMP_NUM_THREADS=16
       export PSC_OMP_AFFINITY=TRUE
    
       ./myprog    
       #### End of Sample Script  ##################
    
       #### Beginning of Data Staging Example Script ############
       #!/bin/bash
       #PBS -q data 
       #PBS -l walltime=4:00:00
       #PBS -l select=1:ncpus=1
       #PBS -j oe
       
       cd $PBS_O_WORKDIR
       cp -r $ARCHIVE_HOME/mydataset/* . || exit 1
       qsub mpi_job.pbs
       #### End of Sample Script  ##################
    
    
       Resource Limits:
       ==================
       The only resource limits users should specify are walltimes and select, 
       ncpus, and node_type limits.  The "select" statement requests a job be 
       allocated a number of chunks with the given "ncpus" size.  The "node_type"
       limit is used to explicitly request a job be placed on x2200 or x4600 nodes.   
         
    
       Tracking Your Job:
       ==================
       To see which jobs are queued and/or running, execute this
       command:
    
         qstat -a
    
    
    
       Current Queue Limits:
       =====================
       Queue limits are subject to change and this news item is not always
       updated immediately.  For a current list of all queues, execute:
    
         qstat -Q
    
       For all limits on a particular queue:
    
         qstat -Q -f <queue-name>
    
    
    
       Maintenance
       ============
       Scheduled maintenance activities on Midnight use the Dedicated Time 
       functionality of PBS to reserve all available nodes on the system.  
       This reservation keeps PBS from scheduling jobs which would still be 
       running during maintenance.  This allows the queues to be left running
       until maintenance.  Because walltime is used to determine whether 
       or not a job will complete prior to maintenance, using a shorter 
       walltime in your job script may allow your job to be run sooner.  
    
       e.g.
       If maintenance begins at 6PM and it is currently 4PM, jobs specifying
       walltimes of 2 hours or less will start if there are available nodes.
    
       
       CPU Usage
       ==========
       Only one job may run per node on midnight (i.e. jobs may not share nodes).  
       If your job uses fewer than the number of available processors on a node 
       the job will be charged for all processors on the node.  
    
       * A job running on a X2200 node (i.e. node_type=4way) consumes 4 CPU hours 
         of allocation per hour regardless of the number of tasks running on each 
         node.
    
       * A job running on a X4600 node (i.e. node_type=16way) consumes 16 CPU hours 
         of allocation per hour regardless of the number of tasks running on each 
         node.
    
    
       PBS Scripts - Default Walltimes
       ===============================
       As of January 22, 2008, any PBS script submitted to the scheduler not 
       including a walltime for the job will be given a default walltime of 8 
       hours.  The only exception to this is the debug queue, which will default
       to 1 hour.  To indicate a walltime, add the following line to your 
       PBS script:
    
                #PBS -l walltime=1:30:00   
       
       This command will request a walltime of 1 hour and 30 minutes.

    "queues"

    Last Updated: Wed, 17 Dec 2008 - Donald Bahls
    Machines: pingo
    Pingo Queues
    ========================================
    The queue configuration is as described below.  It is subject to
    review and further updates.
    
       Queues:
       ===============
       Specify one of the following queues in your PBS qsub script
       (e.g., "#PBS -q standard"):
    
        Queue Name     Purpose of queue
        -------------  ------------------------------
        standard       General use by all allocated users.
    
        challenge      For challenge projects only. 
    
        background     For projects with little or no remaining allocation.
                       This queue has the lowest priority, however
                       projects running jobs in this queue do not have
                       allocation deducted.  The number of running jobs or
                       processors available to this queue may be altered
                       based on system load.
    
        debug          Quick turn around debugging queue.
    
        data           Provides access to long term storage 
                       (i.e. $ARCHIVE_HOME) 
    
        high           For jobs requiring higher priority than
                       normal. (Special Access Only)
    
        special        For jobs which do not fit into normal queue
                       limits. (Limited Access)
    
        urgent         For jobs requiring highest priority. (Special
                       Access Only)
    
    
       See 'qstat -q' for a complete list of system queues.  Note, some
       queues are not intended for general use.
    
       Maximum Walltimes:
       ===================
       The maximum allowed walltime for a job is dependent on the number of
       processors requested.  The table below describes maximum walltimes
       for each queue.
    
       Queue             Min    Max      Max       
                        Procs  Procs   Walltime  Notes
       ---------------  -----  -----  ---------  ------------
       standard      
                            1     16   84:00:00  routes to standard_sm 
                           17   1024   16:00:00  routes to standard_lg
       challenge
                            1     16   96:00:00  routes to challenge_sm 
                           17   1536   48:00:00  routes to challenge_lg
    
       background           1    512    8:00:00  
     
       debug                1    128    1:00:00  
    
       data                 1      1   12:00:00  
    
       PBS Commands:
       =============
       Below is a list of common PBS commands.  Additional information is
       available in the man pages for each command.
    
       Command         Purpose
       --------------  -----------------------------------------
       qsub            submit jobs to a queue
       qdel            delete a job from the queue   
       qsig            send a signal to a running job
    
       Running a Job:
       ==============
       To run a batch job first create a qsub script which, in addition
       to running your commands, specifies the processor resources and
       time required.  Submit the job to PBS with the following command.
       (For more PBS directives, type "man qsub".)
    
         qsub <script file>
    
       Sample PBS scripts:
       --------------------
       Example: MPI-based Application
    
       #!/bin/bash
       #PBS -q standard 
       #PBS -l mppwidth=32 
       #PBS -l walltime=8:00:00
       #PBS -j oe
    
       cd $PBS_O_WORKDIR
    
       aprun -n 32 ./myprog
       
    
       Example: OpenMP-based Application
    
       #!/bin/bash
       #PBS -q standard
       #PBS -l mppwidth=8
       #PBS -l walltime=8:00:00
       #PBS -j oe
    
       cd $PBS_O_WORKDIR
       export OMP_NUM_THREADS=8
    
       aprun -n 1 -d 8 ./myprog    
    
    
       Example: Long Term Data Staging
    
       #!/bin/bash
       #PBS -q data 
       #PBS -l walltime=4:00:00
       #PBS -j oe
       
       cd $PBS_O_WORKDIR
       cp -r $ARCHIVE_HOME/mydataset/* . || exit 1
       qsub mpi_job.pbs
    
    
       Resource Limits:
       ==================
       The only resource limits users should specify are walltimes 
       and mppwidth requests.  
    
       Tracking Your Job:
       ==================
       To see which jobs are queued and/or running, execute this
       command:
    
         qstat -a
    
       Current Queue Limits:
       =====================
       Queue limits are subject to change and this news item is not always
       updated immediately.  For a current list of all queues, execute:
    
         qstat -Q
    
       For all limits on a particular queue:
    
         qstat -Q -f <queue-name>
    
       Maintenance
       ============
       Scheduled maintenance activities on pingo use the Dedicated Time
       functionality of PBS to reserve all available nodes on the system.
       This reservation prevents PBS from scheduling jobs which would still
       be running during maintenance, and enables the queues to run until
       maintenance begins.  Because walltime is used to determine whether
       or not a job will complete prior to maintenance, using a shorter
       walltime in your job script may allow your job to run sooner.
    
       e.g.
       If maintenance begins at 6PM and it is currently 4PM, jobs specifying
       walltimes of 2 hours or less will start if nodes are available.
    
       CPU Usage
       ==========
       Only one job may run per node on pingo (i.e. jobs may not share nodes).  
       If your job uses fewer than the number of available processors on a node 
       the job will be charged for all processors on the node.  
    
       * A job running on a node consumes 8 CPU hours of allocation per hour 
         regardless of the number of tasks running on each node.
    
       Queue Changes
       =============
       2009-07-10  Maximum walltime for challenge_lg was increased from
                   24 hours to 48 hours.
       2009-07-28  Maximum mppwidth for standard_lg was increased from
                   768 to 1024.

    "retirement"

    Last Updated: Wed, 10 Feb 2010 - Don Bahls
    Machines: midnight
    Midnight Retirement
    ========================================
    On June 1st 2010, midnight.arsc.edu will be taken out of regular
    production status.  Existing allocations on the system will be honored
    through June 1st.
    
    Following June 1st and up through September 30, 2010, midnight.arsc.edu
    will continue to operate and will remain available to existing users,
    but without full vendor support.  If certain hardware and/or software
    issues arise between June 1st and September 30th, the system may
    become unavailable for unscheduled prolonged outages.
    
    If you have questions regarding these planned changes, please contact
    User Support.
    
    

    "samples_home"

    Last Updated: Thu, 01 Jan 2009 - ARSC Help Desk
    Machines: midnight
    Baseline Configuration Sample Code Repository
    =============================================
    
    Filename:       INDEX.txt 
    
    Description:    This file contains the name,location, and brief 
                    explanation of "samples" included in this Sample 
    		Code Repository.  There are several subdirectories within 
    		this code repository containing frequently-used procedures, 
    		routines, scripts, and code used on this allocated system,
    		midnight.  This sample code repository can be
    		accessed from midnight by changing directories to 
                    $SAMPLES_HOME, or changing directories to the following 
    		location: mg56% /usr/local/pkg/samples.  This particular
    		file can be viewed from the internet at:
    
                    http://www.arsc.edu/support/news/systemnews/midnightnews.xml#samples_home	
    
    Contents:       AVAILABLE_TOOLS.txt
                    applications
                    dataManagement
                    debugging
                    jobSubmission
                    libraries
                    parallelEnvironment
                   
    ******************************************************************************
    Directory:	applications 
    
    Description:    This directory includes example uses of the software 
                    packages available on midnight.  Each directory name
    		corresponds to the name of the software package example.
    		The contents of each subdirectory may include the
    		following information:
    		   * sample input files 
    		   * how to access a software license (if required) 
    		   * instructions for executing a sample test using both
    				PBS and the interactive job option 
    		   * applicable module and/or environment 
    		   		information (if required) 
    		   * sample scripts for submitting job types using the 
    		   		software 
    		   * additional resources and references available
    
    Contents:	abaqus
                    cobalt
                    gaussian
                    gaussian_09
                    gaussian_linda_09
                    ls-dyna
                    matlab
                    nwchem 
    
    *****************************************************************************
    Directory:	dataManagement 
    
    Description:	This directory contains information about data management 
    		techniques including use of the data queue, storing files in
    		$WORKDIR, $ARCHIVE, and proper data retrieval methods.
    
    Contents:       storageManagement
                    storagePolicies 
                    usingArchive
    
    *****************************************************************************
    Directory:	debugging 
    
    Description:    This directory contains basic information on how to start up 
                    and use	the available debuggers on midnight.
    
    Contents:       totalview 
    
    *****************************************************************************
    Directory:	jobSubmission 
    
    Description:	This directory contains sample PBS batch scripts
    		and helpful commands for monitoring job progress.  
                    Examples include options to submit a jobs such as
    		declaring which group membership you belong to
    		(for allocation accounting), how to request a particular  
    		software license, etc.
    
    Contents:       dataStaging
                    MPI_OpenMP_scripts 
                    MPI_scripts 
    		OpenMP_scripts
    
    *****************************************************************************
    Directory:	libraries 
    
    Description:    This directory contains a summary of the various libraries 
                    available and examples of how to compile a program linking to 
                    those libraries.  Example source code, Makefiles,
    		and data files may be included in this directory.
    
    Contents:       scalapack 
                    papi
    
    *****************************************************************************
    Directory:	parallelEnvironment 
    
    Description:    This directory contains sample code and scripts containing 
                    compiler options for common parallel programming practices
                    including code profiling.  
    
    Contents:       auto_tau_pi
                    autoloops_tau_pi
                    hello_world_mpi
                    manual_tau_pi
                    tau_papi_counters
    
    *****************************************************************************

    "samples_home"

    Last Updated: Wed, 23 Dec 2009 - ARSC Help Desk
    Machines: pingo
    Baseline Configuration Sample Code Repository
    =============================================
    
    Filename:       INDEX.txt 
    
    Description:    This file contains the name,location, and brief 
                    explanation of "samples" included in this Sample 
    		Code Repository.  There are several subdirectories within 
    		this code repository containing frequently-used procedures, 
    		routines, scripts, and code used on this allocated system,
    		midnight.  This sample code repository can be
    		accessed from pingo by changing directories to 
                    $SAMPLES_HOME, or changing directories to the following 
    		location: pingo1% /usr/local/pkg/samples.  This particular
    		file can be viewed from the internet at:
    
    		http://www.arsc.edu/support/news/systemnews/pingonews.xml#samples_home
    
     + AVAILABLE_TOOLS.txt   (list of locally developed utilities)
    
    Directory             Description
    ===================== =======================================================
    applications          This directory includes example uses of the software
                          packages available on pingo
     + cobalt
     + gaussian
     + gaussian_09
     + namd
     + nwchem
      
    -----------------------------------------------------------------------------
    dataManagement        This directory contains information about data 
                          management techniques including use of the data queue, 
                          storing files in $WORKDIR, $ARCHIVE, and proper data 
                          retrieval methods.
     + storageManagement  
     + storagePolicies    
    
    -----------------------------------------------------------------------------
    debugging             This directory contains basic information on how to 
                          start up and use	the available debuggers on pingo.
    
    -----------------------------------------------------------------------------
    jobSubmission         This directory contains sample PBS batch scripts
    		      and helpful commands for monitoring job progress.  
                          Examples include options to submit a jobs such as
    		      declaring which group membership you belong to
    		      (for allocation accounting), how to request a 
                          particular software license, etc.
     + dataStaging        
     + MPI_OpenMP_scripts 
     + MPI_scripts 
     + OpenMP_scripts
    
    -----------------------------------------------------------------------------
    libraries             This directory contains a summary of the various 
                          libraries available and examples of how to compile a 
                          program linking to those libraries.  Example source 
                          code, Makefiles, and data files may be included in this 
                          directory.
     + scalapack 
     + papi
    
    -----------------------------------------------------------------------------
    parallelEnvironment   This directory contains sample code and scripts 
                          containing compiler options for common parallel 
                          programming practices including code profiling.  
     + hello_world_mpi
    
    *****************************************************************************
    
    

    "software"

    Last Updated: Tue, 17 Feb 2009 - Oralee Nudson
    Machines: pingo
    Software
    ========
        ferret: ferret-6.3 (2010-02-12)
            ferret 6.3 is now available on pingo via the "ferret/6.3" module
    
        nco: nco-3.9.9 (2009-11-11)
            nco 3.9.9 is now available on pingo and can be loaded with
            the following module command:
                    module load nco/3.9.9
    
        git: git-1.5.6 (2009-10-29)
            git 1.5.6 is now available on pingo, and a module has been 
    	created for use (git/1.5.6)
    
        paraview: paraview-3.4.0 (2009-10-13)
            paraview 3.4.0 is now available on pingo and can be loaded with
            the following module command:
                    module load paraview/3.4.0
    
        ncl: ncl-5.1.1 (2009-08-05)
            ncl 5.1.1 is now available on pingo and can be loaded with
            the following module command:
                    module load ncl/5.1.1
    
        subversion: subversion-1.6.3 (2009-07-31)
            The latest version of subversion is available via the 
            subversion/1.6.3 module.
    
        gaussian: gaussian-09.A.01 (2009-07-31)
            Gaussian 09 vA.01 is now available in /usr/local/pkg/gaussian.  
            Examples are located in:
    
            $SAMPLES_HOME/applications/gaussian_09
    
        git: git-1.6.1.3 (2009-07-28)
            git 1.6.1.3 is now available on pingo, and is located in
            /usr/local/pkg/git/git-1.6.1.3.gnu.snl  It is also available
            by loading the module, with the following:
                    module load git/1.6.1.3
    
        ncl: ncl-5.1.0 (2009-07-21)
    	ncl 5.1.0 is now available on pingo and can be loaded with
    	the following module:
    		module load ncl/5.1.0
    
        python: python-2.6.2 (2009-05-20)
    	Python 2.6.2 is now available on pingo with the following 
    	modules:
    		basemap-0.99.3, gnuplot-1.8, Imaging-1.1.6,
    		matplotlib-0.98.5.2, netCDF4-0.8, numpy-1.3.0,
    		pycdf-0.6-3b, pytools-9, scipy-0.7.0
    	It is available by running the following:
    		module load python/2.6.2
    		python
    
        geos: geos-2.2.3 (2009-05-19)
    	GEOS is an implementation of GIS in C++.  It is located in 
    	/usr/local/pkg/geos/geos-2.2.3.gnu
    
        cobalt: cobalt-4.2.1 (2009-05-01)
            Version 4.2.1 of cobalt is now available on pingo.  You must
            be in the cobaltcg group to access this application.  An 
            example cobalt job is available in:
               $SAMPLES_HOME/applications/cobalt
    
        mpt: mpt-3.1.1 (2009-04-21) (Upgrade from 3.0.2 on 2009-04-29)
    	The default MPT will be upgraded to 3.1.1 on April 29th during
    	system maintenance.  This will not affect previously compiled
    	applications, but will affect future compiles using the default
    	compiler environment.  This fix should resolve a memory leak related
    	to user defined types.  Users can access the old MPT version by running
    	"module switch mpt mpt/3.0.2" if you experience problems with the 
    	new default version.
    
        gaussian: gaussian-03.E.01 (2009-04-10)
            Gaussian 03 is now available.  It can be accessed in the following
            directory:
                    /usr/local/pkg/gaussian/gaussian-03.E.01
            This software is group controlled.  Please contact consult@arsc.edu
            if you are interested in using it.
    
        hdf: hdf-5-1.8.1 (2009-04-03)
            HDF 5-1.8.1 is now available with shared libraries for the GNU
            compiler.  It is located in /usr/local/pkg/hdf/hdf-5-1.8.1.gnu.gnl
    
        nwchem: nwchem-5.1.1 (2009-03-30)
            The latest version of NWChem has been installed on pingo.
            Contact user support to get access to this package.
            An example PBS Script and job file are available in 
            $SAMPLES_HOME/applications/nwchem 
    
        namd: namd-2.7b1 (2009-03-30)
            NAMD 2.7b1 has been installed on pingo in the directory:
           
            /usr/local/pkg/namd/namd-2.7b1/NAMD_2.7b1_Source/CRAY-XT-g++/   
    
            An example PBS Script and job file are available in 
            $SAMPLES_HOME/applications/namd
    
        netcdf: netcdf-3.6.3 (2009-03-23)
    	NetCDF 3.6.3 is now available for Pathscale compilers on pingo.
    	The libraries are available in:
    	/usr/local/pkg/netcdf/netcdf-3.6.3.path
    
        mpscp: mpscp-1.3a (2009-03-17)
            mpscp 1.3a is now available.  This package has also been made
            the default version on the system, and is available as either mpscp
            or in /usr/local/pkg/mpscp/mpscp-1.3a
    
        silo: silo-4.6.2.gnu.snl (2009-02-17)
    	The silo software is now available on pingo 
    	and is located in the following directory:
    	/usr/local/pkg/silo/silo-4.6.2.gnu.snl 
    	Please note that silo has been installed with 
    	pingo login node support.  Compute node 
    	applications using the silo library may not 
    	operate correctly.

    "software"

    Last Updated: Tue, 17 Feb 2009 - Alec Bennett
    Machines: linuxws
    Software on Linuxws
    ========================================
    
        gsl-1.13: GNU Scientific Library 1.13 (2010-01-14)
            The newest version of GSL is now available.  This version
            was compiled using the GNU compiler and is now available
            on the Workstations in the following location:
            /usr/local/pkg/gsl/gsl-1.13/
    
        pgi: pgi-9.0.4 (2010-01-11)
            The pgi 9.0.4 compiler is now available via the
            pgi-9.0.4 module.
    
        paraview: paraview-3.6.2 (2010-01-11)
            Paraview 3.6.2 is now available via the paraview-3.6.2
            module.
    
        vapor: vapor-1.5.0 (2009-10-21)
            Vapor 1.5.0 is now available via the vapor-1.5.0 module.
    
        paraview: paraview-3.6.1 (2009-10-20)
            Paraview 3.6.1 is now available via the paraview-3.6.1
            module.
    
        idv: idv-2.7u2 (2009-10-20)
            idv 2.7 update 2 is now available via the idv-2.7u2 module.
    
        subversion: subversion-1.6.3 (2009-09-03)
            The latest version of subversion is available via the
            subversion-1.6.3 module.
    
        google-earth: google-earth-5.0 (2009-08-31)
            Google Earth 5.0 is now available on the Workstations in the
            following location:
            /usr/local/pkg/google-earth/google-earth-5.0
    
        ncl: ncl-5.1.1 (2009-08-05)
            The latest version of ncl is now available via the "ncl-5.1.1"
            module.
    
        git: git-1.6.1.3 (2009-07-28)
    	git 1.6.1.3 is now available. This package is available
    	by loading the "git-1.6.1.3" module.
    
        abaqus: Abaqus 6.9 (2009-07-09)
            Abaqus 6.9 is now available.  This package is available
            by loading the "abaqus-6.9" module.
    
        matlab: matlab-7.8.0 (2009-06-19)
            Matlab 7.8.0 is now available.  A module is 
            available for this software and can be loaded with:
                    module load matlab-7.8.0
            After loading the module, type 'matlab' at the prompt
            to open a new matlab session.
    
        cmake: cmake-2.6.4 
            The latest version of cmake is available via the cmake-2.6.4 
            module.    
    
        blender: blender-2.48a
    	Blender 2.48a is now available on the Workstations.  It is 
    	available in a module, as blender-2.48a
    
        avizo: avizo-6.0 (2009-04-30)
    	Avizo 6.0 is now available in the following directory:
    	/usr/local/pkg/avizo/avizo60
    
        visit: visit-1.11.2 (2009-04-29)
    	VisIt 1-11.2 is now available via the "visit-1.11.2" module.
    
        ncl: ncl-5.1.0 (2009-04-29)
            The latest version of ncl is now available via the "ncl-5.1.0"
            module.
    
        paraview: paraview-3.4.0 (2009-03-10)
    	Paraview 3.4.0 is now available. It can be accessed with the
    	paraview-3.4.0 module file and is located in the
    	/usr/local/pkg/paraview/paraview-3.4.0 directory.
    
        visit: visit-1.11.1 (2009-03-02)
    	VisIt is now available in /usr/local/pkg/visit/visit-1.11.1
    	Module files are available as both "visit" and "visit-1.11.1"
    
        comsol: comsol-3.5a (2009-02-06)
    	The newest version of comsol is now available in
    	/usr/local/pkg/comsol/comsol-3.5a.  This version appears
    	to resolve the previous errors when starting the software.
    
        matlab: matlab-7.7.0 (2009-02-05)
            The latest version of Matlab is available for 
            use by loading the matlab-7.7.0 module.  
    	
        matlab: matlab-7.6.0 (2008-07-24)
            The latest version of Matlab is available for 
            use by loading the matlab-7.6.0 module.  
    	
        paraview: paraview-3.0.2 (2007-09-19)
    	Paraview version 3.0.2 has been installed into
    	/usr/local/pkg/paraview/paraview-3.0.2.  It is 
    	available via a module (paraview-3.0.2).
    
        acml: acml-3.6.0 & acml-4.0.0 AMD Core Math Library (2007-09-14)
    	The ACML has been installed and is available in
    	/usr/local/pkg/acml. The following versions were installed:
    		acml-3.6.0.gcc
    		acml-3.6.0.pgi
    		acml-4.0.0.gcc
    	These libraries are available as of Sep 14th, 2007 and
    	the current link was set to point to acml-3.6.0.gcc.
    
        idv: idv-2.2: Integrated Data Viewer (2007-07-28)
            The new version of idv(2.2) has been installed
            in /usr/local/pkg/idv/idv-2.2 and will be made
            the default version on July 12th, 2007.

    "software"

    Last Updated: Fri, 04 Jan 2008 - Donald Bahls
    Machines: seawolf
    Software on Seawolf
    ========================================
    mpscp: mpscp-1.3a (2009-03-20)
           The default version of mpscp was updated from 1.2b
           to 1.3a.
    
    mpscp: mpscp-1.2b (2008-01-04)
           On January 7th, 2008 the default version of mpscp on 
           seawolf will be upgraded from version 1.2 to 1.2b.

    "software"

    Last Updated: Tue, 17 Feb 2009 - Alec Bennett
    Machines: midnight
    Software on Midnight
    ========================================
        nco: nco-3.9.9 (2009-11-11)
            nco 3.9.9 is now available via the nco-3.9.9 module.
    
        gsl: gsl 1.13 (2009-11-05)
            The newest version of GSL is now available.  This version
            was compiled using the GNU compiler and is located
            in /usr/local/pkg/gsl/gsl-1.13.gcc/
    
        pgi: PGI 9.0.4 (2009-10-08)
            The latest version of the PGI compiler is available for
            serial or parallel codes by loading the PrgEnv.pgi-9.0.4
            module.
    
        subversion: subversion-1.6.3 (2009-09-02)
            The latest version of subversion is available via the
            subversion-1.6.3 module.
    
        ncl: ncl-5.1.1 (2009-08-05)
    	ncl 5.1.1 is now available (without OpenDAP support).  This 
            software is accessible by loading the ncl-5.1.1 module. 
    
        gaussian: gaussian-09.A.01 (2009-07-14)
    	Gaussian 09 vA.01 is now available in /usr/local/pkg/gaussian.  
            Examples are located in:
    
            $SAMPLES_HOME/applications/gaussian_09
            $SAMPLES_HOME/applications/gaussian_linda_09 
    
        git: git-1.6.1.3 (2009-07-09)
    	git-1.6.1.3 is now available on midnight and is available
    	in /usr/local/pkg/git/git-1.6.1.3
    
        abaqus: Abaqus 6.9 (2009-07-09)
    	Abaqus 6.9 is now available.  This package is available
    	by loading the "abaqus-6.9" module.  
    
        matlab: matlab-7.8.0 (2009-06-17)
    	Matlab 7.8.0 is now available.  A module is 
    	available for this software and can be loaded with:
    		module load matlab-7.8.0
    	After loading the module, type 'matlab' at the prompt
            to open a new matlab session.
    
        pgi: PGI 8.0.6 (2009-06-10)
            The latest version of the PGI compiler is available for
            serial or parallel codes by loading the PrgEnv.pgi-8.0.6
            module. 
    
        nwchem: nwchem-5.1.1 (2009-05-29)
            The latest release of nwchem is now available on midnight.
            Contact user support to request access to this package.
    
        netcdf: netcdf-4.0.1 (2009-05-21)
            netCDF 4.0.1 is now available, located in:
            /usr/local/pkg/netcdf/netcdf-4.0.1.gcc.shared
    
        python: python-2.6.2 (2009-05-20)
            Python 2.6.2 is now available on pingo with the following
            modules:
                    basemap-0.99.3, gnuplot-1.8, Imaging-1.1.6,
                    matplotlib-0.98.5.2, netCDF4-0.8, numpy-1.3.0,
                    pycdf-0.6-3b, pytools-9, scipy-0.7.0
            It is available by running the following:
                    module load python-2.6.2
                    python
    
        geos: geos-2.2.3 (2009-05-20)
            GEOS is an implementation of GIS in C++.  It is located in
            /usr/local/pkg/geos/geos-2.2.3.gcc
    
        ncl: ncl-5.1.0 (2009-04-09)
    	ncl 5.1.0 with OPeNDAP support is now available.  This software
    	is accessible by loading the ncl-5.1.0 module. 
    
        boost: boost-1.38.0 (2009-04-09)
    	boost 1.38.0 is now available.  Boost is an extension to C++ and
    	includes a number of high levels libraries.  It is available in
    	/usr/local/pkg/boost/boost-1.38.0
    
        curl: curl-7.19.4 (2009-04-09)
    	curl 7.19.4 is now available in /usr/local/pkg/curl/curl-7.19.4.
    	This install is to supprt NCL 5.1, and may not have all 
    	functionality included in the default system curl package.
    
        libnc-dap: libnc-dap-3.7.3 (2009-04-08)
    	libnc-dap has been installed in order to support NCL OPeNDAP
    	support.  It is located in:
    	/usr/local/pkg/libnc-dap/libnc-dap-3.7.3.gcc
    
        libdap: libdap-3.8.2 (2009-04-08)
    	libdap has been installed in order to support NCL OPeNDAP
    	support.  It is located in:
    	/usr/local/pkg/libdap/libdap-3.8.2.gcc
    
        fftw: fftw-2.1.5 w/ shared libs / OpenMP (2009-04-02)
    	FFTW 2.1.5 has been installed with additional support
    	added for OpenMP and Shared Libraries for Pathscale.  This
    	version is available in the following directory:
    	- /usr/local/pkg/fftw/fftw-2.1.5.path.shared
    
        mpscp: mpscp-1.3a (2009-01-28)
            mpscp has been updated to the newest available version.
    
        mexnc: mexnc-2.9.4 (2008-12-19)
    	The mexnc software is now available for use with
    	matlab, and is located in the /usr/local/pkg/mexnc
    	directory.
            To use this version of mexnc, "module load matlab-7.7.0"
            then enter the following at the matlab command prompt: 
            "addpath /usr/local/pkg/mexnc/mexnc-2.9.4/mexcdf/mexnc"
    
        matlab: matlab-7.7.0 (2008-12-19)
    	Matlab 7.7.0 is now available.  A module is 
    	available for this software and can be loaded with
    		module load matlab-7.7.0
    	and running 'matlab'
    
        mpscp: mpscp-1.3a (2008-12-15)
            The latest version of mpscp is available.  To access this
            version run "module load mpscp-1.3a".
    
        pgi: PGI 8.0.1 (2008-12-12)
            The latest version of the PGI compiler is available for
            serial or parallel codes by loading the PrgEnv.pgi-8.0.1
            module.
    
        show_queues: (2008-12-08)
            The show_queues command is now available on midnight.
    
        mexnc: mexnc-2.0.31-2 (2008-10-01)
    	The mexnc software is now available for use with
    	matlab, and is located in the /usr/local/pkg/mexnc
    	directory.
    
        python: Python 2.5.2 (2008-09-05)
    	Python 2.5.2 is available with additional modules:
    	matplotlib,numpy,PIL,pycdf,scipy 
    	This package is available by loading the python-2.5.2
    	module.
    
        abaqus: Abaqus 6.8 (2008-07-28)
    	Abaqus 6.8 is now available.  This package is available
    	by loading the "abaqus-6.8" module.  
    
        gnuplot: gnuplot-4.2.3 (2008-07-25)
    	GNUPlot is now available.  The executables is located
    	in /usr/local/pkg/gnuplot/gnuplot-4.2.3/bin as gnuplot.
    
        pgi: PGI 7.2.2 (2008-07-24)
            The latest version of the PGI compiler is available for
            serial or parallel codes by loading the PrgEnv.pgi-7.2.2
            module.
    
        pathscale: PathScale 3.2 (2008-07-23)
            The latest version of the PathScale compiler has been
            installed.  Use the "PrgEnv.path-3.2" module to access this
            environment.    
    
        git: git-1.5.6 (2008-07-10)
    	git (revision control) has been installed on midnight.
    	This software is available in /usr/local/pkg/git
    
        matlab: matlab-7.6.0 (2008-07-10)
    	Matlab 7.6.0 is now available.  A module is also
    	available for this software and can be loaded with
    		module load matlab-7.6.0
    	and running 'matlab'
    
        mexnc: mexnc-2.0.29 (2008-07-01)
    	The mexnc software is now available for use with
    	matlab, and is located in the /usr/local/pkg/mexnc
    	directory.
    
        cobalt: cobalt-4.2.1 (2008-06-25)
            Cobalt 4.2 was rebuilt to include HDF5 support.  This
            version is otherwise the same as cobalt-4.2 
    
        ncl: ncl-5.0.0 (2008-06-19)
            The latest release of the NCAR Command Language is
            now available on midnight.  To use the latest release
            run: 
                   module load ncl-5.0.0
    
            This version includes both NCL and NCAR Graphics.
         
        idv: idv-2.5 (2008-06-19)
    	The Integrated Data Viewer (IDV) has been installed on 
    	midnight in /usr/local/pkg/idv/idv-2.5  This software is 
    	available by loading the idv module with the following:
    		module load idv-2.5
    	and then running the command "IDV".
    
        pgi: PGI 7.1.6 (2008-05-05)
            The latest version of the PGI compiler is available for 
            serial or parallel codes by loading the PrgEnv.pgi.new
            module (or PrgEnv.pgi-7.1.6).
    
        cobalt 4.2:
            The latest release of cobalt is now available on midnight.
    
        NCAR Graphics 4.4.1 (PathScale):
            The PathScale version of NCAR Graphics library is now 
            available in /usr/local/pathscale/lib.
    
        ezViz: 
            ezViz has been available on midnight for some time
            now.  There is now a module available which will 
            add the ezViz tools to your PATH.  To load ezViz,
            run "module load ezViz".  For more informations on
            ezViz, see: 
            https://visualization.hpc.mil/wiki/EzViz_Batch_Mode 
    
        gaussian: gaussian-03.E.01 & gaussview-4.1 (2008-03-03)
    	Gaussian 03 vE.01 is now available in
    	/usr/local/pkg/gaussian.  Additionally, GaussView
    	has been installed in /usr/local/pkg/gaussview and
    	is available by running "module load gaussview" and
    	"gview".  This software is group controlled, but please
    	contact consult@arsc.edu if you would like to use this
    	software. Also, there is now a sample file in the
    	$SAMPLES_HOME/applications/gaussian directory.
    
        totalview: totalview-8.4.0-0 (2008-02-26)
            The latest version of the totalview debugger is now 
            available via the modules environment.  To use the 
            latest version run "module load totalview-8.4.0-0"
    
        mpscp: mpscp-1.2b (2008-01-04)
            During scheduled maintenance on January 9, 2008, the
            default version of mpscp on midnight will be upgraded
            from version 1.2 to 1.2b. 
    
        pgi: PGI 7.0.2 (2007-12-05)
    	The PGI compilers have been installed on midnight.
    	Additionally, the following packages have been built to
    	support this compiler suite:
    		acml-3.6.0, blacs-1.1.3, fftw-2.1.5, fftw-3.1.2,
    		hdf-4.2r1, ncarg-4.4.1, netcdf-3.6.1, scalapack-1.8.0
    	Additionally, modules "PrgEnv.pgi" and "PrgEnv.pgi-7.0.2" are
    	available for ease of use.
    
        pathscale: PathScale 3.1 (2007-11-19)
            The latest version of the PathScale compiler has been 
            installed.  Use the "PrgEnv.path-3.1" module to access this 
            environment.
    
        ncl-4.3.1:  NCAR Command Language 4.3.1 (2007-08-28)
            New version of ncl is available via the "ncl-4.3.1" module.
    
        sunstudio: Sun Studio 2007-06 (2007-09-14)
            The latest release of the Sun Studio Compiler suite
            is available as of Sept 14, 2007.  The new version is
            available via the "PrgEnv.sun-2007-06" module.
    
        pathscale: PathScale (2.5 & 3.0) (2007-08-09)
    	The PathScale compiler defaults file will be installed
    	and active as of Aug 23rd at 8:00am AST.  This sets defaults
    	relating to include/lib paths as well as search path, and
    	shared library paths.  This should only affect new builds,
    	and should not impact executables and objects compiled
    	prior to Aug 23rd, 2007. 
    	
        ncl-4.3.0: NCAR Command Language 4.3.0 (2007-06-27)
            New version of ncl is available via the "ncl-4.3.0" module.
    
        svn: Subversion 1.4.3 installed (2007-06-01)
    	subversion (svn) is a versioning control systems.  See
    	'man svn' or 'svn help' for more information.  This
    	version was built with SSL enabled to allow access to
    	https servers as well as standard http and ssh.

    "ssh"

    Last Updated: Wed, 11 Jul 2008 - Donald Bahls
    Machines: seawolf
    SSH Allowed Versions Change
    ========================================
    The following two releases of the kerberized ssh packages will be
    disallowed after July 28, 2008:
    
      OpenSSH_4.7p1b
    
    and
    
      OpenSSH_5.0p1a
    
    These versions will no longer be allowed to connect to ARSC systems
    after this date.  
    
    You can run "ssh -V" to determine the version of ssh on your system.
    
    E.g.:
    
      iceberg % ssh -V
      OpenSSH_5.0p1b, OpenSSL 0.9.8h 28 May 2008
    
    Version OpenSSH_5.0p1b of the ssh kit is available here:
    
      https://www.hpcmo.hpc.mil/security/kerberos/
    

    "storage"

    Last Updated: Fri, 19 Jun 2009 - Oralee Nudson
    Machines: pingo
    Pingo Storage
    =============
    
     Long Term Storage Use
     ======================
       
       batch_stage:
       ------------
       Files saved in $ARCHIVE_HOME can potentially be offline (i.e. not
       on disk).  When accessing multiple files in $ARCHIVE_HOME, the
       "batch_stage" can significantly speed the process of retrieving
       files from tape.
    
       e.g.
       cd $ARCHIVE_HOME/somedirectory
       find . -type f | batch_stage -i
    
       See "man batch_stage" for additional examples.   
    
    
       /usr/bin/rcp:
       -------------
       While $ARCHIVE_HOME is available as an NFS filesystem, higher
       transfer rates can be obtained by using the "rcp" command for
       large transfers to and from $ARCHIVE_HOME.
    
       The non-kerberosized version of rcp may be used to transfer files to 
       $ARCHIVE_HOME using the "seanfs" hostname.
    
       e.g.
       /usr/bin/rcp results.tar "seanfs:$ARCHIVE_HOME"
     
       NOTE: The full path to rcp (i.e. /usr/bin/rcp) must be used to
       make transfers without a ticket.
    
                
       See http://www.arsc.edu/support/howtos/storage.html for more information
       on storage best practices at ARSC.
    
    
     $HOME quotas
     ============
    
     On July 6th, 2009 ARSC will begin enforcing $HOME quotas on pingo.
     The default $HOME quota will be set to 1GB.
    
     If your current $HOME disk usage is larger 1 GB, we will be contacting
     you directly to request that you either delete or move data to the long 
     term $ARCHIVE_HOME storage server to reduce the overall size of your $HOME.
    
     If you require an exception to this 1GB $HOME quota, please communicate 
     your specific circumstances to the ARSC Consultants and we will work with
     you to find a solution.
    
     Please contact User Support if you have any questions.
    

    "storage"

    Last Updated: Mon, 19 Jun 2006 - Donald Bahls
    Machines: linuxws
    Linux Workstation Storage
    ========================================
    The environment variables listed below represent paths.  They are
    expanded to their actual value by the shell, and can be used in commands
    (i.e. ls $ARCHIVE).  From the command prompt the value and the variable
    are usually interchangeable. However, in non-shell settings like ftp
    you will need to use the actual path, not the variable.
    
    In the listing below, $USER is an environment variable holding your
    ARSC username.
    
       
      Filesystem     Purpose                 Purged   Backed Up   Quota
      -------------  ----------------------  -------  ----------  ------
      $HOME          shared filesystem       No       Yes         256 MB
       
      $WRKDIR        temp filesystem         Yes      No          None (1) 
      $WORKDIR        
      $SCRATCH        
       
      $ARCHIVE       long term storage       No       Yes         None
      $ARCHIVE_HOME  
      
      
      NOTES:
      (1) Use is limited by the available space on the disk. 
     
    
      Environment Variable Definitions
      =================================
    
      Variable        Definition
      --------------  ---------------------
      $HOME           /u1/uaf/$USER  or
                      /u2/wes/$USER  or
                      /u2/red/$USER 
      $WRKDIR         /scratch/$USER 
      $WORKDIR        /scratch/$USER
      $SCRATCH        /scratch/$USER
      $ARCHIVE        /archive/$HOME
      $ARCHIVE_HOME   /archive/$HOME
    
    
    -- Home directories are intended primarily for basic account info
       (e.g.  dotfiles). Please use $WRKDIR (your /scratch/$USER directory)
       for compiles, inputs, outputs, etc.
       
       *  The 'qcheck' command will show quota information for your $HOME
          directory.  This command requires that you have an account on 
          nanook, please contact the help desk should you need your account
          activated.
    
       *  "qcheck -k" will show quota information in kilobytes rather than 
          blocks.
    
    -- The $WRKDIR or $SCRATCH directories are local to each machine.  (On the
       Linux Workstations these two variables both point to the same
       location.)  When moving to another machine you will also need to move
       your files. This file system not backed up, files not accessed in
       over 30 days are purged (deleted).
    
    
    -- Your $SCRATCH directory is not created by default.  If one does
       not exist on the machine you are using, type 'mkdir $SCRATCH' to
       create one.
    
    
    -- Purging: Files not accessed in over 30 days in $WRKDIR ($SCRATCH)
       directories are purged, and these directories are not backed up.
       Please store what you want to keep in $ARCHIVE.
    
    
    -- Long-term backed up storage is only available in your $ARCHIVE
       directory.  No other directories are backed up.  As this is an
       NFS-mounted filesystem from nanook, files will be temporarily
       unavailable when nanook goes down for maintenance.  I/O performance
       in this directory will be much slower.  Compiles and runs in $ARCHIVE
       are not recommended.
    
    
    See http://www.arsc.edu/support/howtos/storage.html for more information
    on storage policies at ARSC.
    

    "storage"

    Last Updated: Thu, 12 Apr 2007 - Donald Bahls
    Machines: midnight
    Midnight Storage
    ========================================
    The environment variables listed below represent paths.  They are
    expanded to their actual value by the shell, and can be used in
    commands (i.e. ls $ARCHIVE_HOME).  From the command prompt, the
    expanded path and the variable are usually interchangeable. However, 
    in non-shell settings like ftp, you will need to use the actual path,
    not the variable.
    
    In the listing below, $USER is an environment variable holding
    your ARSC username.
    
      Filesystem             Purpose                      Quota
    ------------------      ------------------------     -----------
     $HOME                  dotfiles, sm. files          500 MB 
       /u1/uaf/$USER
       /u2/wes/$USER
       /u2/red/$USER
    
     $WORKDIR               do work here                 100 GB (Initial) 
       /wrkdir/$USER                                     (Purged) 
    
     $ARCHIVE_HOME          long-term remote storage     no quota
       /archive/$HOME
    
     $SCRATCH               local drive each node        no quota 
       /scratch/$USER                                    (Purged)
    
    
       -- $HOME: Home directories are intended primarily for basic account
          info (e.g.  dotfiles). Please use $WORKDIR (your /wrkdir/$USER
          directory) for compiles, inputs, outputs, etc.  Files in the
          $HOME are backed up periodically.  Quotas are enabled on this
          filesystem.  Use the command "quota -v" to show your current
          $HOME use.
       
       -- $ARCHIVE_HOME: Long-term backed up storage is only available in
          your $ARCHIVE_HOME directory.  As this is an NFS-mounted 
          filesystem from seawolf, files will be temporarily unavailable 
          when seawolf goes down for maintenance.  I/O performance in this 
          directory will be much slower. Compiles in $ARCHIVE_HOME are not 
          recommended.  $ARCHIVE_HOME is not available from compute nodes.
          The "data" queue provides access to $ARCHIVE_HOME for copying
          data to and from $WORKDIR.  See the "Long Term Storage Use"
          section below for additional Long Term Storage recommendations.
    
       -- $WORKDIR: Short term, not backed up, purged filesystem.  This is
          a large fast local disk.  The $WORKDIR filesystem is available
          to all nodes on midnight.   This is the recommended location
          for input, output, and temporary files.  The $ARCHIVE_HOME 
          filesystem is available for long term storage.  
                
       -- $SCRATCH: drives are local to each node.  For example, the
          $SCRATCH on midnight1 is only available to that system and is
          not available to any other compute node in the midnight cluster.
          You must copy any data you wish to keep from $SCRATCH to $WORKDIR
          at the completion of your job.  When possible $WORKDIR should
          be preferred over $SCRATCH as files in $WORKDIR are available 
          from all compute nodes. 
          
          The $SCRATCH directory is not created by default.  If you wish to
          use $SCRATCH, you must create the directory on each compute node
          that you will be using it on.  Please contact the ARSC Help desk
          for further details.
    
    
    Long Term Storage Use
    ======================
       
       batch_stage:
       ------------
       Files saved in $ARCHIVE_HOME can potentially be offline (i.e. not
       on disk).  When accessing multiple files in $ARCHIVE_HOME, the
       "batch_stage" can significantly speed the process of retrieving
       files from tape.
    
       e.g.
       cd $ARCHIVE_HOME/somedirectory
       find . -type f | batch_stage -i
    
       See "man batch_stage" for additional examples.   
    
    
       /usr/bin/rcp:
       -------------
       While $ARCHIVE_HOME is available as an NFS filesystem, higher
       transfer rates can be obtained by using the "rcp" command for
       large transfers to and from $ARCHIVE_HOME.
    
       The non-kerberosized version of rsh may be used to transfer files to 
       $ARCHIVE_HOME using the "seanfs" hostname.
    
       e.g.
       /usr/bin/rcp results.tar "seanfs:$ARCHIVE_HOME"
     
       NOTE: The full path to rcp (i.e. /usr/bin/rcp) must be used to
       make transfers without a ticket.
    
                
       See http://www.arsc.edu/support/howtos/storage.html for more information
       on storage best practices at ARSC.
    
    

    "support"

    Last Updated: Thu, 07 Aug 2008 - Oralee Nudson
    Machines: linuxws midnight pingo seawolf
    User Support Information
    ========================================
    ARSC hosts accounts for two sets of users:  HPCMP and academic.  To 
    receive the most prompt support, it is important to contact the 
    appropriate support channel for your account.
    
    HPCMP users should contact the Consolidated Customer Assistance 
    Center (CCAC) as their first point of contact.  
    The ARSC Help Desk should be the the first point of contact
    for academic users.  
    
    The support command will display the appropriate support email 
    address and phone number for your account type:
    
    % support
    
    

    "totalview"

    Last Updated: Mon, 02 Jun 2008 - Donald Bahls
    Machines: midnight
    Totalview
    ========================================
    Totalview is available on midnight and can be used to debug MPI, OpenMP and 
    serial applications.  Generally debugging should occur on compute nodes 
    through the use of an interactive PBS job.  Totalview may be run on login 
    nodes to debug short serial applications or to inspect core files.  
    
    The instructions below are prefaced by a prompt corresponding to a system
    name where the command should be run.
       + midnight%  corresponds to a midnight login node 
        (i.e. midnight1 or midnight2).
       + midnight-compute%  corresponds to a midnight compute node.
       + local%  corresponds to the name of your local workstation.  
         
          
    I. Starting an interactive job with X11 forwarding enabled.
      
       A) Log into midnight1 or midnight2 with X11 forwarding enabled.
       
          local% ssh -X -Y username@midnight.arsc.edu
        
       B) Run the "tunnelx" command to setup X11 forwarding for the 
          compute nodes.
          
          midnight1% tunnelx 
          
       C) Start an interactive PBS job requesting the number of processors
          required for your job.
          
          # for an 8 task MPI job on 4way (i.e. X2200) nodes
          midnight% qsub -l select=2:ncpus=4:node_type=4way -I
          
          # for a 16 task OpenMP job on 16way (i.e. X4600) nodes
          midnight% qsub -l select=1:ncpus=16:node_type=16way -I
           
          # for a serial job 
          midnight% qsub -l select=1:ncpus=1:node_type=4way -I 
          
          When there are a sufficient number of nodes available, PBS will 
          start the job. 
       
       D) Once the interactive PBS job starts, rerun the "tunnelx"
          command to initialize the X11 settings for your shell.
          
          midnight-compute% tunnelx
          
          Next, source the file ~/.tx_local which was generated by the tunnelx
          command.
          
          # bash/ksh syntax
          midnight-compute% . ~/.tx_local
         
          # csh/tcsh syntax
          midnight-compute% source ~/.tx_local          
    
          You should be able to run X11 applications on the compute nodes.
          
       E) For additional details on the "tunnelx" command see "man tunnelx"
       
          
    II. Running totalview.
    
       A) For MPI applications, start the application using the "-tv" and 
          "-timeout" flags.
          
          midnight-compute% mpirun -tv -timeout 600 ./a.out
       
          The "-tv" flag instructs mpirun to start the executable (a.out) 
          under the control of totalview.  The "-timeout" flag sets the 
          maximum time the MPI environment should wait before generating 
          a timeout error.  Setting the timeout limit makes it easier to 
          set up breakpoints, etc without totalview. 
       
       B) For OpenMP and serial applications, start the application using 
          totalview.
       
          midnight-compute% totalview ./a.out
          
    
    Additional hints:
    
        1) Code should be compiled with -g.  This makes it possible for 
           totalview to refer back to the source code.  Code compiled without
           -g will appear as assembly and you will not have meaningful access
           to variable values.
    
        2) You can view core files with totalview by passing the executable 
           and core file to totalview.  A core file from an MPI application
           can be viewed without using mpirun.
    
           midnight% totalview ./a.out core.1234
             
        3) The totalview command line version (i.e. totalviewcli) is also 
           available.  This version can be used with MPI environment by 
           setting the TOTALVIEW environment variable before issuing the 
           mpirun command.
           
           # bash/ksh syntax
           midnight-compute% export TOTALVIEW=/usr/local/bin/totalviewcli
           midnight-compute% mpirun -tv -timeout 600 ./a.out
           
           # csh/tcsh syntax
           midnight-compute% setenv TOTALVIEW /usr/local/bin/totalviewcli
           midnight-compute% mpirun -tv -timeout 600 ./a.out
           
           For more information, see http://www.totalviewtech.com
    

    "totalview"

    Last Updated: Tue, 20 Jan 2009 - Donald Bahls
    Machines: pingo
    Totalview
    ========================================
    Totalview is available on pingo and can be used to debug MPI, OpenMP
    and serial applications.  Generally debugging should occur on compute
    nodes through the use of an interactive PBS job.  Totalview may be
    run on login nodes to debug short serial applications or to inspect
    core files, however the majority of debugging work should occur on
    compute nodes.
    
    The instructions below are prefaced by a prompt corresponding to a
    system name where the command should be run.
       + pingo4%  corresponds to a pingo4.arsc.edu login node 
       + pingo4-pbs%  corresponds to a pingo4 node assigned by PBS
       + local%  corresponds to the name of your local workstation.  
    
    NOTE: At this time, totalview is only supported on pingo4.arsc.edu.
    
    I. Starting an interactive job with X11 forwarding enabled.
      
       A) Log into pingo4 with X11 forwarding enabled.
       
          local% ssh -X -Y username@pingo4.arsc.edu
        
       B) Start an interactive PBS job requesting the number of processors
          required for your job.
          
          # for an 16 task MPI job  
          pingo4% qsub -l select=1:host=nid00075 -q debug -l mppwidth=16 -v DISPLAY -I
    
          Here's a breakdown of this command:
          a) -l select=1:host=nid00075
                requests that the job be placed on pingo4 (a.k.a. nid00075)
          b) -q debug
                requests that the job run in the debug queue
          c) -l mppwidth=16
                requests 16 cores
          d) -v DISPLAY
                export the DISPLAY environment variable
          e) -I
                runs a PBS interactive job.
          
          When there are a sufficient number of nodes available, PBS
          will start the job.  The default and maximum walltime for
          "debug" queue jobs is 1 hour.  Should you need to run a longer
          debugging sessions, please use the "standard" queue or contact
          User Support for additional assistance.
    
       C) Once the interactive PBS job starts, run the "xt-totalview" module
          if it's not already loaded.
          
          pingo4-pbs% module load xt-totalview
          
          
    II. Running totalview.
    
       A) For MPI applications, start the application using the following:
          
          pingo4-pbs% totalview aprun -a -n 16 ./a.out 
      
          This starts the aprun executable under the control of totalview.
          The aprun executable will pass control of the application to
          totalview.  Here the "-a" flag indicates that all options after
          that flag should be passed to "aprun" rather than totalview.
    
       B) For OpenMP and serial applications, start the application using 
          totalview.
       
          # csh/tcsh syntax
          pingo4-pbs% setenv OMP_NUM_THREADS 8
          pingo4-pbs% totalview aprun -a -n 1 -d 8 ./a.out
          
          # sh/bash syntax
          pingo4-pbs% export OMP_NUM_THREADS=8
          pingo4-pbs% totalview aprun -a -n 1 -d 8 ./a.out
    
    
    Additional hints:
    
        1) Code should be compiled with -g.  This makes it possible for 
           totalview to refer back to the source code.  Code compiled without
           -g will appear as assembly and may be difficult to debug.
    
        2) You can view core files with totalview by passing the executable 
           and core file to totalview.  A core file from an MPI application
           can be viewed without using mpirun.
    
           pingo% totalview ./a.out core.1234
           
           For more information, see http://www.totalviewtech.com
    


    Arctic Region Supercomputing Center
    PO Box 756020, Fairbanks, AK 99775 | voice: 907-450-8600 | email:

    home | search | about | support | news | science | resources