ARSC HPC Users' Newsletter 344, July 20, 2006

Contents

• Converting Byte Order of Binary Data Files on the Cray XD1
• Handel-C Programming Language Class, for FPGA Applications
• Parallel Performance Evaluation Tools Workshop
• Quick Tip

Converting Byte Order of Binary Data Files on the Cray XD1

[ Thanks to Ed Korkven of ARSC for this article, the first in a Series on Porting Codes From the Cray X1. ]

In anticipation of the September 30 decommissioning of Klondike, ARSC's Cray X1, ARSC is urging Klondike users to begin planning for that transition. One of the major concerns that users may have is the portability of data files produced on the X1. Text files (i.e. those created with the Fortran OPEN option FORM='FORMATTED') are not a problem since by default the X1 creates ASCII text files and any machine to which users would port those files will read ASCII text files transparently, perhaps with minor end-of-line differences for non-Unix machines. But what about binary data files (i.e. those created with the OPEN option FORM='UNFORMATTED')?

The X1 differs from many earlier Cray machines in that by default it uses 32-bit IEEE standard floating point with two's complement arithmetic. This is an advantage since this format is commonly used on most other platforms likely to be targets for porting from the X1. (We would appreciate hearing of any exceptions.) Even when porting to another IEEE platform however, there are some issues to consider. In this and following newsletters we will discuss strategies for doing these ports.

In this issue, we will begin to take a look at the "byte order" of the contents of a binary data file. The byte order of a machine refers to the order in which it stores multi-byte data items in memory, and then in data files. Klondike is a "big-endian" machine. If moving to another big-endian machine, byte order will not be a consideration. One notable CPU set with opposite byte order however, is the x86 family. This includes the AMD chips that are prevalent in many machines, notably, Cray's XT3 and XD1 platforms. Reading unformatted X1 files on such computers will require some kind of special handling either on the X1 or on the target.

ARSC's XD1 machine has a very handy feature built in to its Portland Group Fortran compiler that automatically converts big-endian to little-endian and vice versa when UNFORMATTED data is read or written. Here is an example. The following is an X1 program that writes a binary data file called "output.dat".

program writer
  implicit none
  integer*4         :: out_i4  = X'deadbeef'
  real*4            :: out_r4  = X'cafefeed'
  integer*8         :: out_i8  = X'ceded0dace0faded'
  real*8            :: out_r8  = X'deafbeadedefface'
  open(11,file='output.dat', form='unformatted', access='sequential')
  write(11) out_i8, out_i4, out_r4, out_r8
  close(11)
end program writer

Compile this code like this: ftn -o writer writer.f90 And then execute: ./writer

Looking at the output with the od utility reveals its contents: od -x output.dat

0000000 0000 0018 cede d0da ce0f aded dead beef
0000020 cafe feed deaf bead edef face 0000 0018

Running the same program on the XD1 gives a different result:

0000000 0018 0000 aded ce0f d0da cede beef dead
0000020 feed cafe face edef bead deaf 0018 0000

Let's now read that file on the XD1 with this program:

program reader
  implicit none
  integer*4         :: in_i4, out_i4  = X'deadbeef'
  real*4            :: in_r4, out_r4  = X'cafefeed'
  integer*8         :: in_i8, out_i8  = X'ceded0dace0faded'
  real*8            :: in_r8, out_r8  = X'deafbeadedefface'

  open(11,file='output.dat', form='unformatted', access='sequential')
  read(11) in_i8, in_i4, in_r4, in_r8
  close(11)
  if (in_i4 .eq. out_i4) print*, 'Integer*4 matches.'
  if (in_r4 .eq. out_r4) print*, 'Real*4 matches.'
  if (in_i8 .eq. out_i8) print*, 'Integer*8 matches.'
  if (in_r8 .eq. out_r8) print*, 'Real*8 matches.'
end program reader

If we compile the reader program with: pgf90 -o reader reader.f90 we get no "match" messages during execution because the X1 big-endian output is read as little-endian values. Recompiling with:

    pgf90 -byteswapio -o reader reader.f90

  Integer*4 matches.
  Real*4 matches.
  Integer*8 matches.
  Real*8 matches.

So the Portland Group compiler provides a simple solution to byte order differences between big-endian and little-endian machines. And this example will hopefully motivate X1 users to start thinking about the porting process -- for data as well as code. ARSC staff will be happy to assist Klondike users during the transition.

Handel-C Programming Language Class, for FPGA Applications

ARSC will offer training for the Handel-C programming language and DK toolkit from Celoxica. The class will use Celoxica's RC-10 reconfigurable computing boards.

  Class schedule: 
    Tuesday July 25 through Friday July 28
    9:00 am - 5:00 pm each day
    West Ridge Research Building (WRRB) room 104

This is a class for electrical & computing engineers and others interested in using field-programmable gate arrays (FPGAs) for a variety of applications. Celoxica's Handel-C is one of several programming toolkits that ARSC is investigating this summer as part of an ongoing study of high-performance reconfigurable computing.

Space is limited. Please email Greg Newby (newby AT arsc.edu) to register. There is no fee to attend this training.

Details on the class are available at:

http://www.celoxica.com/techlib/default.asp?Action=1&CatID=28&CatType=2&OrderBy=1

The class will take place in the EPRS teaching laboratory, on the UAF campus, West Ridge Research Building (WRRB) room 104. Lunch and snacks will be provided.

Parallel Performance Evaluation Tools Workshop

This course is next week, already! For more information and to register, see:

http://www.arsc.edu/support/training/PerfTools20060725.html

A:[[ I have an existing library written in C/C++ that I would like to
  [[ call from a python script.  Is there a way to call this existing
  [[ code without reimplementing the code in python?
      
#
# Thanks to Brad Chamberlain:
#

While I don't have firsthand experience with it, I do know that
the Babel package out of LLNL is designed to support C/C++/Python
interoperability (as well as Fortran and Java).  For more information,
see:

        http://www.llnl.gov/CASC/components/babel.html


#
# Editor's response
#

SWIG (http://www.swig.org/) is a tool which connects C/C++ to high
level languages such as Python, Ruby, Java, Tcl, PHP.  The steps are:

  1) Build an interface file for your code so SWIG knows what portions
    of the code should be available in the python modules.
  2) Run swig on the interface file to create a wrapped C file and the
    corresponding python module.
  3) Compile the wrapped C file and other C source files.
  4) Build a shared library using the object files.

There is a basic tutorial here:
  http://www.swig.org/tutorial.html


Q: I want to search for certain keywords in a monstrous text file.
   But for each "hit," I want to see, not just the line containing the
   word, but a few lines **preceding** it as well.  I think I could
   write a perl script to do this, but is there another way?

[[ Answers, Questions, and Tips Graciously Accepted ]]

Thomas J. Baring	ARSC Web Specialist	ph: 907-450-8619
Donald Bahls	ARSC User Consultant	ph: 907-450-8674
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020

Send comments and questions to the current editors using this Contact Form.

To subscribe to the e-mail edition of the ARSC HPC Users' Newsletter, send this message to: "majordomo@arsc.edu":
    subscribe hpc_users
    end
For help with majordomo, send this:
    help
    end
Leave the "subject" line of your message blank.

The ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter is archived and available by anonymous FTP to: ftp://ftp.arsc.edu/pub/publications/newsletters/

Arctic Region Supercomputing Center
PO Box 756020, Fairbanks, AK 99775 | voice: 907-474-6935 | email:

home | search | about | support | news | science | resources