ARSC HPC Users' Newsletter 249, June 28, 2002
Unix Tools for Portable Applications, Part IV of IV
[ Many thanks to Kate Hedstrom of ARSC for contributing this series of articles. ]
Last time we just barely touched the surface of the gnu autotools. This time we will introduce what to do when you need more than the tools provide out of the box. One of the most commonly required features is to search for a third-party software package, both its include files and its libraries. Here we will describe how to locate the NetCDF package, but a similar solution can be used for many other packages.Setup
The autotools are written in a mixture of Bourne shell programming and the m4 macro language. The AC_xxx names are all m4 macros that come with autoconf. It is possible to write your own m4 macros to be used with the autotools. First we have to decide where to put our macros. One option is in the [root]/share/aclocal directory where [root] is the place on your system where the autotools reside (possibly /usr/local ). For those of us who can't write to such a place, we have the choice of putting the macros in a directory within each project or in a central directory, e.g., $HOME/aclocal . This last option is what I have chosen.
Now that we have decided on where to put our home-grown macros, let's make a bootstrap script to point to it. Put this in $HOME/bin and set its permissions to executable:bootstrap:
#! /bin/sh # # Bootstrap the autoconf family of tools set -x aclocal -I $HOME/aclocal autoheader automake --foreign --add-missing --copy autoconf
Now we don't have to remember all those autotools commands, but we have to remember to type 'bootstrap'. Also, recall that these are the commands to create the configure script and the Makefile template. We still have to run "configure" and "make depend" on the target computer.Finding NetCDF
The idea here is that we want to tell configure where to find the NetCDF library and include file:
otherwise it will use /usr/local as a default. We write a macro to accept the --with-netcdf argument and to check for the existence of the library and include file. We'll call the macro KH_PATH_NETCDF to make it clear that it is home-grown, not part of autoconf . The macro goes into a netcdf.m4 file in our $HOME/aclocal directory. We now have:configure.ac:
# Process this file with autoconf to produce a configure script. AC_INIT([forcing], [1.0], [email@example.com]) AC_CONFIG_SRCDIR([forcing.F]) AM_CONFIG_HEADER([config.h]) AM_INIT_AUTOMAKE AC_PROG_F77 KH_PATH_NETCDF KH_CHECK_CPP AC_CONFIG_FILES([Makefile]) AC_OUTPUT
The other custom macro here is looking for a nonstandard version of cpp . We use this cpp to convert .F files to .f files and we put that conversion step into the Makefile through Makefile.am:Makefile.am:
## Process this file with automake to produce Makefile.in ## SUFFIXES = .o .f .F .F.o: $(CPP) -P $(AM_CPPFLAGS) $(CPPFLAGS) $(DEFS) < $< > $*.f $(F77) -c $(AM_FFLAGS) $(FFLAGS) $*.f .F.f: $(CPP) -P $(AM_CPPFLAGS) $(CPPFLAGS) $(DEFS) < $< > $*.f .f.o: $(F77) -c $(AM_FFLAGS) $(FFLAGS) $< bin_PROGRAMS = forcing forcing_SOURCES = bes1d.F bes2d.F cppdefs.h crash.F cvector.F \ day_code.F def_info.F def_out.F def_var.F fields.h forcing.F \ forcing.h get_date.F get_grdfld.F get_grid.F get_ncfld.F \ init_vars.F inp_par.F lenstr.F ncparam.h opencdf.F param.h \ scalars.h wrt_out.F depend: sfmakedepend $(MDEPFLAGS) $(forcing_SOURCES) clean-generic: rm -f *.f
There are two versions of some flags, such as FFLAGS and AM_FFLAGS . This is so that we can set some FFLAGS in our macros and still allow the user to set FFLAGS when running configure. For instance, we might be telling it a search path while the user sets the optimization level.
Note that we have added a clean-generic line to get rid of the .f files on a "make clean" . Automake provides the "clean" target that will invoke "make clean-generic" along with the usual removal of *.o.netcdf.m4: (in $HOME/aclocal)
# autoconf macros for detecting NetCDF # AC_DEFUN([KH_PATH_NETCDF],[ AC_CHECKING([for NetCDF]) AC_ARG_WITH([netcdf], [Path to NetCDF installation (default: /usr/local )], [NC_PREFIX="$withval"], [NC_PREFIX="/usr/local"] ) AC_MSG_RESULT([NetCDF location: $NC_PREFIX]) if eval "test -d $NC_PREFIX/include"; then NC_INCLUDES="-I$NC_PREFIX/include" else AC_MSG_ERROR([$NC_PREFIX/include not found]) fi if eval "test -d $NC_PREFIX/lib"; then NC_LDFLAGS="-L$NC_PREFIX/lib" NC_LIBS="-lnetcdf" else AC_MSG_ERROR([$NC_PREFIX/lib not found]) fi save_CPPFLAGS="$CPPFLAGS" save_LDFLAGS="$LDFLAGS" CPPFLAGS="$CPPFLAGS $NC_INCLUDES" MDEPFLAGS="$MDEPFLAGS $NC_INCLUDES" LIBS="$LIBS $NC_LIBS" LDFLAGS="$LDFLAGS $NC_LDFLAGS" AC_SUBST([MDEPFLAGS]) AC_SUBST([NC_PREFIX]) netcdf=no AC_LANG([C]) AC_CHECKING([for netcdf.inc]) AC_CHECK_HEADER([netcdf.inc], [netcdf=yes], [netcdf=no]) AC_MSG_RESULT([NetCDF include result: $netcdf]) if test "x$netcdf" = xno; then AC_MSG_ERROR([NetCDF include not found]) fi AC_LANG([Fortran 77]) AC_CHECKING([for libnetcdf.a]) AC_CHECK_LIB([netcdf], [nf_close], [netcdf=yes], [netcdf=no]) AC_MSG_RESULT([NetCDF link result: $netcdf]) if test "x$netcdf" = xno; then AC_MSG_ERROR([NetCDF library not found]) fi # Put them back to how they used to be and set the AM versions # The AM versions must be substituted explicitly CPPFLAGS="$save_CPPFLAGS" LDFLAGS="$save_LDFLAGS" AM_CPPFLAGS="$NC_INCLUDES $AM_CPPFLAGS" AM_LDFLAGS="$NC_LDFLAGS $AM_LDFLAGS" AC_SUBST([AM_CPPFLAGS]) AC_SUBST([AM_LDFLAGS]) ])
We start by naming our macro and opening its definition. The AC_CHECKING and AC_MSG_RESULT simply write messages when configure is run. AC_ARG_WITH is the heart of the matter where it parses the --with-netcdf argument. AC_SUBST writes the named macros into the Makefile. Most of the rest is checking for the existence of the lib and include directories, then whether it can include netcdf.inc and link to libnetcdf.a . Note that the include check has to take place within a C language section, causing warnings from configure since netcdf.inc doesn't parse correctly as a C file.
I strongly recommend that you read the current autoconf manual if you plan on writing your own macros.Fortran 90
We also need a macro to check for netcdf.mod or NETCDF.mod and to see if we can compile using it:netcdf_f90.m4: (in $HOME/aclocal)
# autoconf macro for detecting NetCDF module file # AC_DEFUN([KH_PATH_NETCDF_F90],[ AC_CHECKING([for NetCDF module file]) save_FFLAGS="$FFLAGS" for flag in "-I" "-M" "-p"; do FFLAGS="$flag$NC_PREFIX/include $save_FFLAGS" AC_TRY_COMPILE(, [ use netcdf], [netcdf=yes; NC_FFLAGS=$flag], [netcdf=no]) if test "x$netcdf" = xyes; then break fi done AC_MSG_RESULT([NetCDF module result: $netcdf, flag: $NC_FFLAGS]) if test "x$netcdf" = xno; then AC_MSG_ERROR([NetCDF module not found]) fi FFLAGS="$save_FFLAGS" AM_FFLAGS="$NC_FFLAGS$NC_PREFIX/include $AM_FFLAGS" AC_SUBST([AM_FFLAGS]) ])
Once again we define our macro, called KH_PATH_NETCDF_F90 . The guts here are the for loop with the AC_TRY_COMPILE . Since different compilers use different flags to specify the path to that netcdf.mod file, we need to find out which one works by compiling a program containing "use netcdf". With this kind of check, we don't need to know the name of the module file, just where it lives.
The configure script must specify KH_PATH_NETCDF before KH_PATH_NETCDF_F90 . The resulting configure script can be run as:
yukon% configure --with-netcdf=/myplace FFLAGS="-em -f free" icehawk% configure FFLAGS="-qfree=f90"
Note that I am setting the free format flags with configure; this should be done in the configure.ac file with a FREE_FORMAT macro. This is left as an exercise for the student :-) Ditto the -em business on the Crays.
In general, the autotools check for how the system works today. It is strongly discouraged to check for what kind of computer we are on and then to set things for how we know that computer behaved last year. Please feel free to contact me if you are interested in learning more about autotools and Fortran - we can figure it out together.
[ Thanks to Richard Griswold for this contribution, on the request of the editors. ]
There are a lot of text editors available for Unix systems, both text-based, such as pico, vi, and Emacs, and graphical, such as Emacs (again), but most of them are not easy to use for the novice user. Other text editors aim for the novice crowd, but end up being very limited. They often lack features that programmers and advanced users want, such as regular expressions, macros, and syntax highlighting. Fortunately there is one editor that is both powerful and easy to use. That editor is the Nirvana Editor, or NEdit.
NEdit is text editor for the X Window System. It is a GUI program top to bottom, unlike some other graphical text editors which are simply a shell over a text-based program. NEdit uses "standard" key bindings that most users of text editors and word-processors are used to. For example, the Home and End keys take you to the beginning and end of a line. CTRL-Home and CTRL-End take you to the beginning and end of the document. CTRL-Up and CTRL-Down take you to the beginning and end of the paragraph. And so on. If you can navigate your way around a document using WordPad or Word, you can use NEdit.
For programmers, however, NEdit has additional advanced features. It supports syntax highlighting and ships with patterns for most popular languages. If the language you want isn't included, you can download patterns off the web, or you can create your own.
NEdit also has a full-fledged macro language and the ability to execute external commands. The search and replace mechanism can take either normal case-sensitive or case-insensitive strings, or regular expressions.
On top of all of this, NEdit has the slickest rectangular selection mechanism of any text editor I've ever seen. You can cut, copy, paste, and move rectangular blocks of text at will. Unlike some other text editors that require strange key-combos to perform rectangular selections, with NEdit you simply hold down the CTRL key when you make your selection.
It has a nice middle-drag mode that allows you to copy text with a single mouse operation. This mode supports rectangular selections as well.
NEdit runs on almost any system. In addition to Unix and Linux systems, it has been ported to VMS, MS Windows (using Cygwin), IBM zSeries (formerly S/390), IBM iSeries (formerly AS/400, using the PASE environment), and MacOS X.
NEdit is simply the best text editor I've ever used, and with each release it keeps getting better. I've used it for over seven years for everything from jotting down quick notes to writing production code. I highly recommend it to anyone who needs to edit text. For more information, see: http://www.nedit.org/.
[ Editor's note: nedit is installed on the ARSC SGIs. ]
A Surprising Story
Thought you'd be interested in some local news concerning an Alaska dermatologist. He specializes in the removal of blemishes from the neck and face.
During the summer he sets up an office on Boating Point Harbor, near Anchorage, and offers a special rate ($29.95) to cruise ship passengers.
He recently caught the world by surprise, setting a new record in the number of Boating Point operations per second, performed on the neck.
Quick-Tip Q & A
A:[[ I've gotten used to NQS, and the other batch schedulers on [[ HPC systems, and would like to "submit" jobs on an SGI workstation [[ in a similar way. I want jobs to start, and keep running, even after [[ I've logged off. Can I do this?
Check the man pages for the "at" and "batch" commands. We prefer you use "batch", as it "nices" your job so that the system remains useful to interactive and other users who may show up. A:[[ More on Fortran 90's exit and cycle # Thanks to Alan Wallcraft: In Fortran 90, the CYCLE statement does exactly what you want: do n=1,m : if (i .eq. j) cycle : enddo As pointed out above, there is also an exit, e.g.: do n=1,m : if (i .eq. j) goto 100 : enddo 100 continue becomes: do n=1,m : if (i .eq. j) exit : enddo With cycle and exit, I suggest always using Fortran 90 "block" do-loops, i.e. every loop has its own enddo (and no loop labels). This avoids confusion over which loop is involved. If you want to cycle to an outer loop, then "name" the loop e.g.: do_l: do l=1,8 do n=1,m : if (i .eq. j) cycle do_l : enddo enddo do_l Q: Let's take a computing break! June is mosquito month in Fairbanks, and 2002 has been impressive by all accounts. Send us your favorite mosquito story, remedy, or advice. Any luck with mosquito traps, DEET-free dope, or personal concoctions?
[[ Answers, Questions, and Tips Graciously Accepted ]]
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669 Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678 Arctic Region Supercomputing Center University of Alaska Fairbanks PO Box 756020 Fairbanks AK 99775-6020
Subscribe to (or unsubscribe from) the e-mail edition of the
ARSC HPC Users' Newsletter.
Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.