Contents

• ARSC Seminar: The SMS Library for Portable Parallelism
• Unix Tools for Portable Applications, Part I of IV
• The Revised C Standard, C99
• New Cray On-Line Documentation
• ARSC Support for Public Radio and TV
• Next Newsletter, May 31
• Quick Tip

NOAA's Forecast Systems Laboratory has created a portable library to aid in the parallelization of logically rectangular simulations. SMS and its documentation is freely available from:

http://www-ad.fsl.noaa.gov/ac/sms.html

An introductory talk was given (http://www.arsc.edu/~kate/SMS/sms.ppt) describing what SMS does for you. More information is available in "The Scalable Modeling System: A High-Level Alternative to MPI", issue #219:

/arsc/support/news/hpcnews/hpcnews219/index.xml

In this seminar, you will see SMS in action using some simple examples. We will work through some SMS programs and show how to compile and run them on both the T3E and the IBM. An account on one of these ARSC HPC systems or the ARSC Linux cluster is required.

[ Many thanks to Kate Hedstrom of ARSC for contributing this series of articles. ]

I belong to a team of people using and modifying the Regional Ocean Modeling System (ROMS). ROMS consists of over a hundred files and it uses one or more external libraries. The challenge is to be able to compile and run it on as many different computers as possible. We use "make" for compiling and will introduce it here. In future installments of this series, we will explore gnu autoconf and other ways to generate the Makefiles.

Introduction to Make

Make is a tool which is almost as old as Unix, designed primarily for keeping track of how programs are compiled. That is what we will describe here, although it can be used for other things, including web page maintenance. It is just a matter of telling make what commands need to be run to update your files.

Make gets its instructions from a description file, by default named "Makefile". This file is also called the Makefile, but other files can be used by invoking make with the -f option, e.g.:

  make -f Makefile.yukon

When I first got our ocean model, its Makefile looked something like:

model: main.o init.o plot.o
        f90 -o model main.o init.o plot.o

main.o: main.f
        f90 -c -O main.f

init.o: init.f
        f90 -c -O init.f

plot.o: plot.f
        f90 -c -O plot.f

clean:
        rm *.o core

The default thing to build is "model", the first target. The syntax is

target: dependencies
[TAB]  command
[TAB]  command

The target "model" depends on the object files, main.o and friends. They have to exist and be up to date before model's link command can be run. The other targets tell make how to create the object files. The original version of this Makefile turned off optimization on plot.f due to a compiler bug, but hopefully you won't ever have to worry about that.

Compiling "model" is simple, just type "make". Make will look for the file Makefile, read it, and do whatever is necessary to make "model" up to date. If you edit init.f, that file will be newer than init.o. Make would see that init.o is out of date and run the "f90 -c -O init.f" command. Now init.o is newer than model, so the link command "f90 -o model main.o init.o plot.o" must be executed.

Clean up by typing "make clean". The clean target will be brought up to date. "clean" has no dependencies, so the command (rm *.o core) will always be executed.

Macros

MY_MACRO = nothing today

$(MY_MACRO)

The convention is to put the macros near the top of your Makefile and to use upper case. Also, use separate macros for the name of your compiler and the flags it needs:

F90 = f90
F90FLAGS = -O3
LIBDIR = /usr/local/lib
LIBS = -L$(LIBDIR) -lmylib

Let's rewrite our Makefile using macros:

#
# IBM version
#
F90 = xlf90
F90FLAGS = -O3 -qstrict
LDFLAGS = -bmaxdata:0x40000000

model: main.o init.o plot.o
        $(F90) $(LDFLAGS) -o model main.o init.o plot.o

main.o: main.f
        $(F90) -c $(F90FLAGS) main.f

init.o: init.f
        $(F90) -c $(F90FLAGS) init.f

plot.o: plot.f
        $(F90) -c $(F90FLAGS) plot.f

clean:
        rm *.o core

Now when we change computers, we only have to change the compiler name in one place. Likewise, if we want to try different optimization levels, we only specify that in one place.

By the way, you can use comments by starting the line with a #.

Implicit Rules

Make has some rules already built in. For fortran, you might be able to get away with:

OBJS = main.o init.o plot.o

model: $(OBJS)
        $(FC) $(LDFLAGS) -o model $(OBJS)

as your whole Makefile. Make will automatically invoke its default Fortran compiler, possibly f77 or g77, with whatever default compile options it happens to have (FFLAGS). One built in rule often looks like:

.c.o:
        $(CC) $(CFLAGS) -c $<

which says to compile .c files to .o files using the compiler CC and options CFLAGS. We can write our own suffix rules in this same style. The only thing to watch for is that make by default has a limited set of file extensions that it knows about. Let's write our Makefile using a suffix rule:

#
# Cray version
#
F90 = f90
F90FLAGS = -O3
LDFLAGS =

.f.o:
        $(F90) $(F90FLAGS) -c $<

model: main.o init.o plot.o
        $(F90) $(LDFLAGS) -o model main.o init.o plot.o

clean:
        rm *.o core

Dependencies

There may be additional dependencies beyond the source->object ones. In our little example, all our source files include a file called commons.h. If commons.h gets modified to add a new variable, everything must be recompiled. Make won't know that unless you tell it, using the syntax:

# include dependencies
main.o: commons.h
init.o: commons.h
plot.o: commons.h

Fortran 90 introduces module dependencies as well, but we'll save them for another day.

In conclusion, make is a very powerful tool. The book to read is "Managing projects with make" by Andrew Oram and Steve Talbott, 1991, O'Reilly.

Make has a portable subset of features, with system-dependent extensions. If you want to use extensions, I suggest sticking with those supported by gnu make (gmake), since it is available most everywhere.

The most common newbie mistake is to forget that the commands after a target *have* to start with a tab.

[ Thanks to Jim Long of ARSC for this contribution. ]

Did you think that C had stopped evolving? Read on...

ANSI C has a new revision, C99, that many are not familiar with, due in large part to the fact that few compilers implement many of its features. This article will highlight some of the more obvious changes relevant for scientific computing.

1. Some new keywords in C99 include:

   inline

   (hint only, not a requirement)

   long long

   (64 bit int)

   restrict

   (one writer or many readers: if an object accessed with a restricted pointer is modified then all access is through that pointer, but unmodified objects may be aliased.)

2. C99 no longer requires that all declarations be made at the beginning of a block. This sample is thus legal under C99:

   
     #include <stdio.h>
     
     int main()
     {
        int i=10;
        i += 2;
     
        int j;  // declaration after executable statement 
                //  (note C++ style comments)
     
        for(j=0; j<i; j++)
          printf("j = %d\n", j);
     
     }
   

   The declaration of j may even occur in the "for" loop, i.e.

   
     for(int j=0; j<i; j++),
   

   similar to C++. Hint: Delay the declaration of a variable until that variable's first assignment to avoid uninitialized variable errors [1].

3. C99 introduces variable length arrays. When you first learned C, you may have wanted to do something like this:

   
     int main()
     {
        int i;
     
        printf("Enter an integer\n");
        scanf("%d", &i);
     
        int array[foo(i)];   // runtime behavior
     }
   

   Such runtime arrays are OK in C99. Unlike a malloc'ed array, however, variable length arrays cannot be resized while they are in scope. Function prototypes that take a variable length array are declared with a "*" in the array brackets, e.g.:

   
     foo(int array[*]);
   

4. IEEE 754 floating point arithmetic is available in C99, along with complex numbers [2]. Complex numbers in C90 are handled clumsily as structures, while in C++ they are handled as classes. In C99, however, they are built-in types:

   
     _Complex float
     _Complex double
     _Complex long double
     _Imaginary float
     _Imaginary double
     _Imaginary long double
     __I__
   

   _Complex and _Imaginary are not required in freestanding implementations [3]. An example given in [2] solves the equation x^2 - 4x + 13 = 0 with the following snippet:

   
     #include <stdio.h>
     #include <math.h>
     #include <complex.h>
   
     int main() {
        double complex a=1, b=-4, c=13, x1, x2;
   
        x1 = (-b + csqrt(b*b-4*a*c))/(2*a);
        x2 = (-b - csqrt(b*b-4*a*c))/(2*a);
   
        printf("x1 = complex(%f,%f)\n", creal(x1), cimag(x1));
        printf("x2 = complex(%f,%f)\n", creal(x2), cimag(x2));
     }
   

   The output of this program is:

   
     x1 = complex(2.000000,3.000000)
     x2 = complex(2.000000,-3.000000)
   

An example of header file complex.h can be seen at [4], where there is a macro to change a declaration like "double complex" into "double _Complex" (order unimportant), along with various versions of math functions to do complex arithmetic and return real and imaginary parts (like creal() and cimag() above).

Other features in C99 not covered in this article include a bool type, flexible array members that allow struct members without bound, static array parameters, and others (see the various references).

Finally, be aware of differences between C99 versus C++98 when using a C++ compiler [5].

The status of C99 features in gcc can be found at [6]. The current availability of C99 at ARSC is as follows. gcc is installed on icehawk (IBM SP cluster) at version 2.95, which has limited support for C99 features. Quest (our linux cluster) and the SGI servers have gcc 2.91, which offer no support.

  [1] Randy Meyers, "The New C: Declarations and Initializations," 
      C/C++ Users Journal, April 2001, 
      (
http://www.cuj.com/reference/articles/2001/0104/0104d/0104d.htm
)
  
  [2] Harry H. Cheng, "C99 & Numeric Computing", Dr. Dobb's Journal, 
      March 2002
  
  [3] 
http://www.comeaucomputing.com/techtalk/c99/

  
  [4] 
http://www.dinkumware.com/htm_cl/complex2.html

  
  [5] 
http://david.tribble.com/text/cdiffs.htm#C99-vs-CPP98

  
  [6] 
http://gcc.gnu.org/c99status.html

Cray has replaced dynaweb with a new documentation server, "CrayDoc." CrayDoc is a nice improvement: faster and more powerful. We encourage you to snoop around:

http://www.arsc.edu:40/

http://www.arsc.edu:40/3rd/

For the moment, there isn't a link from the CrayDoc welcome page, but our webmaster assures me it's coming soon. Till then, use the above URL. You'll find documentation for:

• IMSL
• HDF
• PGI
• ImmersaDesk
• Ferret

• ARSC is now the sponsor of "Earth and Sky" on our local NPR station, KUAC. Listen at 8:24 am during Alaska Edition.

• ARSC, in conjunction with Spenard Builders Supply and the UAF Journalism Department, is also sponsoring a visit by NPR's Corey Flintoff.

  Flintoff will be in Fairbanks as the UAF commencement speaker, and will give a public lecture called "Inside NPR: Deciding what Makes the News" in the UAF Salisbury Theater at 8pm on Friday, May 10.

• We're also sponsoring NOVA (Tuesday nights at 9) and Nature and American Masters (Sunday nights) on Alaska One.

Both editors will be at CUG in two weeks... looking for good stories! But, this'll make the newsletter a week late.

 

A:[[From time to time I'm instructed to append some path to "PATH", 
  [[ some file to "LM_LICENSE_FILE", or some such thing.  For instance,
  [[ in your last issue, regarding Totalview:
  [[  
  [[     "To use it, add this: 
  [[  
  [[       /usr/local/adm/pkg/flexlm/license.dat 
  [[  
  [[   to the settings of your LM_LICENSE_FILE environment variable."
  [[  
  [[ How specifically, would you suggest I do that?


  # 
  # Many thanks to Rich Griswold for his response:
  # 

  How you do this depends on which shell you are using.  For sh, ksh, bash,
  and similar shells, you can do the following:

    if [ ${MY_VAR} ]; then
      MY_VAR=${MY_VAR}:var_value
    else
      MY_VAR=var_value
    fi
    export MY_VAR

  The curly braces around the variable name in the assignment statement
  are there to prevent the shell from interpreting the rest of the line
  as part of the variable name.  For some of these shells, you may be
  able to combine the assignment and export statements.  For example:

    export MY_VAR=${MY_VAR}:var_value

  For csh and tcsh, you can do the following:
  
    if ( $?MY_VAR ) then
      setenv MY_VAR ${MY_VAR}:var_value
    else
      setenv MY_VAR var_value
    endif
  
  In general both forms will append a colon and the new value to the
  environment variable if it already exists.  Otherwise, the new value
  will be assigned to a new environment variable.




Q: I thought I'd use the Fortran 90 compiler to compile some Fortran 90 
   code. Silly me. This is on the IBM SP, and it complains:

     ICEHAWK1$ xlf90 prog.f90
     xlf90: 1501-218 file prog.f90 contains an incorrect file suffix
     ld: 0711-715 ERROR: File prog.f90 cannot be processed.
         The file must be an object file, an import file, or an archive.
  
   So, what do I do now?

[[ Answers, Questions, and Tips Graciously Accepted ]]

Ed Kornkven	ARSC HPC Specialist	ph: 907-450-8669
Kate Hedstrom	ARSC Oceanographic Specialist	ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020

Subscribe to (or unsubscribe from) the e-mail edition of the ARSC HPC Users' Newsletter.

Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.