ARSC HPC Users' Newsletter 342, June 23, 2006

Contents

• Introduction to Modules for Environment Management
• Workshop Announcement
• Reminder: SGI Retirement
• Quick Tip

Introduction to Modules for Environment Management

ARSC's Cray XD1 and Cray X1 have the modules package installed for version control of software packages. The modules package allows you to quickly switch between multiple versions of software on a system. Having multiple versions of a compiler available can be helpful when you discover that a particular version of a compiler or library doesn't work as you expect. It also allows ARSC to install and test the new programming environments without changing the default environment.

On ARSC's Cray XD1 the Portland Group compilers are managed via modules. Each module does several things:

• Alters the PATH to include the directories where the compilers are located.
• Alters the MANPATH to include the man pages for the particular version of the compiler.
• Alters the LD_LIBRARY_PATH so the proper dynamic libraries are used.
• Sets other environment variables required by the compilers.

To get a list of available modules with "pgi" in the name, issue the following:

   % module avail pgi

   ------------------- /usr/local/pkg/modulefiles -------------------
   pgi       pgi.6.0   pgi.6.1.1 pgi.6.1.4 pgi.6.1.6

Here were see five compiler versions available.

A particular module version can be loaded by issuing the "module load" statement:

   % module load pgi.6.0
   % which pgcc
   /usr/local/pkg/pgi/pgi-6.0/linux86-64/6.0/bin/pgcc

Modules that are currently loaded can be displayed using the "module list" statement:

   % module list
   Currently Loaded Modulefiles:
     1) pgi.6.0

You can switch between two module versions using the "module switch" statement:

E.g.

   % module switch pgi.6.0 pgi.6.1.6
   % which pgcc
   /usr/local/pkg/pgi/pgi-6.1.6/linux86-64/6.1/bin/pgcc

If you'd like particular modules to be loaded each time you login, you can add the "module load" statements to your shell login file.

You can check to see which environment variables and other settings will be altered by a module file by using the "module display" statement:

E.g.

   % module display pgi.6.0
   -------------------------------------------------------------------
   /usr/local/pkg/modulefiles/pgi.6.0:

   module-whatis    loads the PGI compiler suite 
   setenv           PGI /usr/local/pkg/pgi/pgi-6.0/linux86-64/6.0 
   prepend-path     PATH /usr/local/pkg/pgi/pgi-6.0/linux86-64/6.0/bin 
   prepend-path     MANPATH /usr/local/pkg/pgi/pgi-6.0/linux86-64/6.0/man 
   prepend-path     LD_LIBRARY_PATH /usr/local/pkg/pgi/pgi-6.0/linux86-64/6.0/lib 
   prepend-path     LM_LICENSE_FILE /var/flexlm/pgi.dat 
   prepend-path     PATH /usr/mpich/mpich-1.2.6-pgi602/bin 
   prepend-path     MANPATH /usr/mpich/mpich-1.2.6-pgi602/man 
   prepend-path     LD_LIBRARY_PATH /usr/mpich/mpich-1.2.6-pgi602/lib/shared:/usr/mpich/mpich-1.2.6-pgi602/lib 
   -------------------------------------------------------------------

Other Considerations

1. Be careful not to load multiple versions of the same package simultaneously. Use either "module unload old-pkg" then "module load new-pkg" or "module switch old-pkg new-pkg".
2. Because the module command needs to affect shell environment variables, they are implemented as aliases or functions depending on the shell that you are using. Aliases and functions are not inherited by child shells (i.e. child processes). This means that the module command will not be available in scripts unless you do a little extra work.

   Here's an example script for nelchina:

         #!/bin/ksh
         #PBS -q computesmall
         #PBS -l nodes=1:ppn=1
         #PBS -l walltime=8:00:00
         #PBS -S /bin/ksh
         
         # First we need to source the initialization script 
         # for the Korn shell
         . /usr/share/modules/init/ksh
         
         # next we'll clear all loaded modules to make sure there are no
         # other matlab modules currently loaded.
         module purge
         
         # last we'll load the module we want to use:
         module load matlab
          
         matlab -nodisplay < myjob.m > matlab.$PBS_JOBID.out
   

   Similarly if you start a new shell on the command line, the module command will not be available, but will be if you source the initialization file for the shell.

3. "modules" is actually an open-source project (http://modules.sourceforge.net/). In our experience modules simplify life for both users and system administrators. We are in the process of evaluating modules on other HPC systems where there isn't direct vendor support for the modules package.

Workshop Announcement

[[ Thanks to Mohamed Iskandarani for this announcement ]]

>          THE FIFTH INTERNATIONAL WORKSHOP ON UNSTRUCTURED MESH
>          NUMERICAL MODELLING OF COASTAL, SHELF AND OCEAN FLOWS
>                  Nov 13th-15th in Miami, USA.
>  
>  The fifth international workshop on unstructured grid modelling
>  of coastal, shelf and ocean flows will take place in Miami at the
>  Rosenstiel School of Marine and Atmospheric Science on November 13-15
>  2006.  The web site is http://www.rsmas.miami.edu/personal/ugom06.
>   
>  This workshop is a forum for ocean and atmospheric scientists and
>  modelers interested in unstructured grids (Finite Volume or Finite
>  Element) to discuss topics of common interests, including solution
>  algorithms, model development, model validation, and applications.
>  The first four workshops took place in Europe and this is the first
>  time the meeting is held in the US.
>    
>  The workshop consists of 30 minutes invited talks (to be announced),
>  15 minutes oral presentations, and poster sessions.  The emphasis is on
>  open discussions of issues pertaining to modelling geophysical flows
>  using unstructured grids. Although the meetings have traditionally
>  centered around ocean modelling, contributions from atmospheric
>  scientists and modelers are welcome.
>    
>    
>  For more information see the workshop website:
>     http://www.rsmas.miami.edu/personal/ugom06
>

Reminder: SGI Retirement

As a reminder to users at the UAF campus, ARSC's SGI workstations are scheduled to be retired at the end of June. If you need assistance transitioning to the new Linux workstations, please contact the ARSC help desk (consult@arsc.edu)

For more information and a list of new access lab machines, see:

System status for new Linux workstations can be obtained here:

A:[[ Is there a way I can reference my shell aliases from scripts?  
  [[ Here's a ksh example of the problem:
  [[
  [[   $ cat t2.ksh
  [[     #!/bin/ksh
  [[     ls -l *.ksh
  [[   $ 
  [[   $ ./t2.ksh  
  [[   -rwx------    1 monty    grp           20 May 26 16:08 t.ksh
  [[   -rwx------    1 monty    grp           23 May 26 16:08 t2.ksh
  [[   $ 
  [[   $ alias ll
  [[   ll='ls -l'
  [[   $ 
  [[   $ cat t.ksh
  [[     #!/bin/ksh
  [[     ll *.ksh
  [[   $ 
  [[   $ ./t.ksh
  [[   ./t.ksh[2]: ll:  not found
  [[   $ 

#
# Thanks to Greg Newby:
#

Shell scripts are not considered interactive, so the shell
initialization files that set up your interactive environment are
not run.  For ksh, this might be your .profile or .login...  tcsh might
use .tcshrc or .cshrc.

If you find where 'll' is aliased, you could just source that file
in your shell script.  Syntax is different for sh/ksh/bash than
tcsh/csh, but generally tcsh/csh are inferior choices for scripts of
any complexity.

The dot is used to source external files in sh/ksh/bash.  So, for
aliases (as well as functions, variables, and other things) defined
in your .profile:

$ cat t3.ksh
        #!/bin/ksh
        . ./.profile
        ll -l *.ksh

But you probably don't just want to run the script from your home
directory, so provide a path that will always work:

# cat t4.ksh
        #!/bin/ksh
        . ${HOME}/.profile
        ll -l *.ksh

Some aliases are defined in system files:

# cat t5.ksh
        #!/bin/ksh
        . /etc/profile
        ll -l *.ksh

You'll need to investigate the various shell initialization scripts
to find where the aliases (or functions, or variables) you want are
defined.  Be on the lookout for initialization scripts that generate
terminal output, since that's likely not what you want.


Q: Can ftp perform recursive "get" and "put"?  I want to retrieve a 
   directory and everything it contains.  If ftp can't do this, is there
   a different way?
      

[[ Answers, Questions, and Tips Graciously Accepted ]]

Thomas J. Baring	ARSC Web Specialist	ph: 907-450-8619
Donald Bahls	ARSC User Consultant	ph: 907-450-8674
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020

Send comments and questions to the current editors using this Contact Form.

To subscribe to the e-mail edition of the ARSC HPC Users' Newsletter, send this message to: "majordomo@arsc.edu":
    subscribe hpc_users
    end
For help with majordomo, send this:
    help
    end
Leave the "subject" line of your message blank.

The ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter is archived and available by anonymous FTP to: ftp://ftp.arsc.edu/pub/publications/newsletters/

Arctic Region Supercomputing Center
PO Box 756020, Fairbanks, AK 99775 | voice: 907-474-6935 | email:

home | search | about | support | news | science | resources