ARSC T3D Users' Newsletter 91, June 14, 1996
pxfgetarg() - Handling Command Line Arguments in T3D Programs
[Don Morton, visiting ARSC from Cameron University, contributes this article. See his contact information, provided below.]
Many implementations of Fortran, particularly those in a Unix environment, allow for command-line argument passing when loading the executable. These functions, IARGC() and GETARG() are documented on Denali's man pages. In the following example, we demonstrate the use of these functions in a program which simply echos the arguments provided on the command line:
argtest1.f:
ccc argtest1.f
program argtest1
implicit none
integer numargs, ! Number of arguments on command line
& i ! Loop counter
character*80 argument ! Character string for storing an argument
external iargc
integer iargc
ccc Find out how many arguments there are
numargs = iargc()
ccc Print each of the arguments - Note that the 0th argument, following
ccc Unix tradition, is the name of the binary.
do i=0,numargs
call getarg(i, argument)
print *, 'Argument ', i, ': ', argument
enddo
end
The above works fine on most Unix platforms, including the Cray Y-MP, Linux PC's, SGI's, etc. However, if you try this on a T3D program, you'll run into a link error stating that getarg() is an unresolved function call. Denali's man page for getarg() implies that the function is implemented on all Cray Research systems, but this is in error. Tucked in the pages of a Cray Research Service Bulletin is the real scoop.
Cray states that the increased address space on the T90 and MPP systems requires a two-word Fortran Character Descriptor in some of the Fortran system interface routines. Thus, they can no longer reliably detect when Hollerith arguments are used in place of Fortran character arguments and vice versa, particularly in routines which have optional arguments (such as getarg(), which can be called with an optional third argument).
Cray has provided alternative interface routines that conform to the Posix Fortran 77 Language Interfaces standard. The alternative for getarg() is pxfgetarg(), and ipxfargc() is the Posix-compliant alternative to iargc(). So, the above program, written for portability would look like:
argtest2.F:
ccc argtest2.F
program argtest2
implicit none
integer numargs, ! Number of arguments on command line
& i ! Loop counter
character*80 argument ! Character string for storing an argument
#ifdef _CRAY-MPP
external ipxfargc
integer ipxfargc
ccc extra variables needed for calling pxfgetarg()
integer arglength, ierror
#else
external iargc
integer iargc
#endif
ccc Find out how many arguments there are
#ifdef _CRAY-MPP
numargs = ipxfargc()
#else
numargs = iargc()
#endif
ccc Print each of the arguments
do i=0,numargs
#ifdef _CRAY-MPP
call pxfgetarg(i, argument, arglength, ierror)
if (ierror .ne. 0) then
write(0,*) 'Problem with input argument ', i
stop
endif
#else
call getarg(i, argument)
#endif
print *, 'Argument ', i, ': ', argument
enddo
end
Several points concerning the above program:
- Note the .F extension which, on many systems (including denali) will cause the preprocessor to be invoked before compilation. On systems where this doesn't happen automatically, you'll need to run it through the C preprocessor first, with appropriate flags.
- The _CRAY-MPP macro is defined automatically when using the CRAY MPP compilers.
- pxfgetarg() is more functional than getarg(), allowing for more robust execution. Full details are available in the man pages.
=======================================================================
Don Morton Email: morton@arsc.edu
Visiting Scientist (summer) Voice: (907) 474-5507
Arctic Region Supercomputing Center Fax : (907) 450-8601
University of Alaska
Fairbanks, AK 99775
http://grizzly.cameron.edu/morton.html
=======================================================================
List of Posix-Compliant Routines
A list of system-interface routines for which Cray has created alternative, Posix-compliant versions follows. Man pages are available, and documentation may be found in the "Application Programmer's Library Reference Manual, publication SR-2165:Old Routine New Routine ----------- ----------- access() pxfaccess() chdir() pxfchdir() chmod() pxfchmod() chown() pxfchown() chroot() pxfchroot() creat() pxfcreat() getarg() pxfgetarg() getenv() pxfgetenv() iargc() ipxfargc() link() pxflink() open() pxfopen() rename() pxfrename() stat() pxfstat() uname() pxfuname() unlink() pxfunlink() utime() pxfutime()
Porting Heterogeneous Distributed Codes to the T3D - Part II
Last week (Newsletter #90), Don Morton discussed the master/slave paradigm for distributed programming, and contributed programs for a pair of cooperating processes. With these, he demonstrated the recommended method of porting PVM master/slave codes to the Y-MP/T3D.The method he used follows CRI's advice (see next article) of opening only one circuit between the master (on the Y-MP) and the entire group of slaves (on the T3D). The master communicates with the slave on PE0, and no other. The slave on PE0, in turn, is designated as special, and relays data and instructions to all of the other slaves. In a sense, this amounts to having two masters, one on the Y-MP and a second on PE0. Another view: the process on the Y-MP acts as an interface, or front-end, to a master/slave group on the T3D.
Another simpler approach is to treat the Y-MP process as the sole master, and open a pvm circuit from it to each and every slave. This follows the traditional master/slave paradigm. It may require less recoding than the "dual" master approach, and could be an easy first step in porting distributed codes to the Y-MP/T3D. Once a program following this method is working, the programmer could move some, or even all, of the master's duties to the slave on PE0.
The following codes show a Y-MP master coordinating a group of equivalent T3D slaves (compare with last week's examples). To run these, you must set the PVM environment variable, PVM_PE_LIST, to "all" before launching the PVM daemon. (See next article.)
master:
include 'fpvm3.h'
parameter (NUMPROCS=5, ISIZE=1000, NUMSLAVES=NUMPROCS-1)
dimension vec(ISIZE)
dimension list(0:NUMSLAVES) ! For storing TID's
ccc Enroll in PVM
call pvmfmytid(mytid)
print *, 'Hello World, I am the master: task ', mytid
ccc Spawn slaves
call pvmfspawn("slave", 0, '*', NUMSLAVES, list, info)
print *, 'Task ', mytid, ' spawned ', info, ' slaves...'
print *, 'TID List of Slaves:'
do i=0,info-1
print *, list(i)
enddo
ccc Initialize a vector for transmission to slaves
do i=1,ISIZE
vec(ISIZE) = real(i)
enddo
ccc Send it to all slave processes
msgtag = 1
call pvmfinitsend(PVMDATADEFAULT, info)
call pvmfpack(REAL8, vec, ISIZE, 1, info)
do i=0,NUMSLAVES-1
call pvmfsend(list(i), msgtag, info)
print *, 'Task ', mytid, ' sent data to ', list(i)
enddo
ccc Wait for ACK from all slaves
msgtag = 2
do i=0,NUMSLAVES-1
call pvmftrecv(-1, msgtag, 30, 0, info)
if (info .NE. 0) then
call pvmfbufinfo (info, nbytes, imsgtag, isender_tid, ierr)
print *, 'Master received ACK from slave: ', isender_tid
else
print *, 'Master timed out waiting for ACK from slave.'
goto 999
endif
enddo
ccc exit PVM
999 continue
print *, 'Task ', mytid, ' exiting...'
call pvmfexit(info)
end
slave:
include 'fpvm3.h'
parameter (ISIZE=1000)
dimension vec(ISIZE)
ccc Enroll in PVM
call pvmfmytid(mytid)
call pvmfgetpe(mytid, mype)
call pvmfparent(myparent)
print *, 'Slave, task ', mytid, ' PE ', mype,
+ ' my parent ', myparent, ' group instance ', inum
ccc Waiting for "data" from master
msgtag = 1
call pvmftrecv(myparent, msgtag, 30, 0, info)
if (info .NE. 0) then
call pvmfunpack(REAL8, vec, ISIZE, 1, info)
print *, 'Slave ', mytid, ' received data ...'
else
print *, 'Slave ', mytid, ' timed out waiting for data.'
goto 999
endif
ccc Send ACK to master
msgtag = 2
call pvmfinitsend(PVMDATADEFAULT, info)
call pvmfpack(INTEGER8, 1, 1, 1, info)
call pvmfsend(myparent, msgtag, info)
ccc Exits PVM
999 continue
print *, 'Task ', mytid, ' exiting...'
call pvmfexit(info)
end
Makefile:
all: master slave
slave: slave.f
TARGET=cray-t3d /mpp/bin/cf77 -I/usr/include/mpp -o slave slave.f
cp slave ${HOME}/pvm3/bin/CRAY/
master: master.f
TARGET=target cf77 -I/usr/include/pvm3 -o master master.f -lpvm3
Environment Variable 'PVM_PE_LIST'
PVM_PE_LIST tells the pvm daemon with which T3D PEs it must prepare to communicate. For example:
'unset PVM_PE_LIST' ## circuit to task spawned on PE0
## PVM daemon ALWAYS opens circuit to PE0.
'PVM_PE_LIST=0' ## circuit to PE0
'PVM_PE_LIST=2' ## circuit to PE0 and PE2
'PVM_PE_LIST=1,2,4,8' ## circuit to PEs 0,1,2,4, and 8
'PVM_PE_LIST=all' ## circuit to every spawned task
PVM_PE_LIST is ignored by programs run directly on the T3D (using mppexec). Only if your program uses the pvm daemon and calls "pvm_spawn" or "pvmfspawn" should you worry about it.
The following runs of the master/slave program given in today's Newsletter show the effect of different settings for PVM_PE_LIST. If a PE is not on the list, pvmfspawn doesn't return a valid task id number (TID) for that PE, and thus, the master can't communicate with it.
## Note: run under Korn shell.
denali$ PVM_PE_LIST=all
denali$ pvm
pvm> quit
denali$ master
Hello World, I am the master: task 262146
Task 262146 spawned 4 slaves...
TID List of Slaves:
393216
393217
393218
393219
Task 262146 sent data to 393216
Task 262146 sent data to 393217
Task 262146 sent data to 393218
Task 262146 sent data to 393219
Master received ACK from slave: 393219
Master received ACK from slave: 393218
Master received ACK from slave: 393216
Master received ACK from slave: 393217
Task 262146 exiting...
denali$ unset PVM_PE_LIST
denali$ pvm
pvm> quit
denali$ master
Hello World, I am the master: task 262146
Task 262146 spawned 4 slaves...
TID List of Slaves:
393216
1
1
1
Task 262146 sent data to 393216
Task 262146 sent data to 1
Task 262146 sent data to 1
Task 262146 sent data to 1
Master received ACK from slave: 393216
Master timed out waiting for ACK from slave.
Task 262146 exiting...
denali$ export PVM_PE_LIST=0,2
denali$ pvm
pvm> quit
pvmd still running.
denali$ master
Hello World, I am the master: task 262146
Task 262146 spawned 4 slaves...
TID List of Slaves:
393216
1
393218
1
Task 262146 sent data to 393216
Task 262146 sent data to 1
Task 262146 sent data to 393218
Task 262146 sent data to 1
Master received ACK from slave: 393218
Master received ACK from slave: 393216
Master timed out waiting for ACK from slave.
Task 262146 exiting...
We've noted several times that CRI discourages programmers from opening more than one circuit from daemon to T3D. Okay... here is the source, taken from 'docview.'
pvm.hence.50
PE communication
2.6.1.2
PEs on the CRAY T3D system communicate with the daemon and other PVM
tasks by using sockets. In theory, any PE can do so. But UNICOS
limits the number of open files per application and the number of open
sockets in the system. So, if a CRAY T3D application running on a
large number of PEs were to set up communications for each PE, it may
hit either or both of these limits.
Socket communications are very slow, especially compared to the cost
of communications between PEs. Because much of socket communication
is single-threaded in the agent process, the PVM daemon, or UNICOS
itself, the cost goes up as more PEs try to communicate at the same
time.
For these reasons, by default only PE0 establishes communications with
the daemon, and Cray Research suggests you consider using PVM in this
manner. However, you can specify additional PEs by setting the
PVM_PE_LIST environment variable, as follows:
setenv PVM_PE_LIST 0,4,8,12
setenv PVM_PE_LIST all
This environment variable must be set for both the PVM daemon pvmd3
and the application to read, and both must read the same value.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note
At present, PE0 always establishes communications with the daemon,
even if PE0 is not specified in PVM_PE_LIST. It is suggested that
PVM_PE_LIST specify PE0 if it is being used to ensure future
compatibility. It is possible that future releases may introduce other
mechanisms for controlling access to the daemon.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
List of Differences Between T3D and Y-MP
The current list of differences between the T3D and the Y-MP is:- Data type sizes are not the same ( Newsletter #5 )
- Uninitialized variables are different ( Newsletter #6 )
- The effect of the -a static compiler switch ( Newsletter #7 )
- There is no GETENV on the T3D ( Newsletter #8 )
- Missing routine SMACH on T3D ( Newsletter #9 )
- Different Arithmetics ( Newsletter #9 )
- Different clock granularities for gettimeofday ( Newsletter #11 )
- Restrictions on record length for direct I/O files ( Newsletter #19 )
- Implied DO loop is not "vectorized" on the T3D ( Newsletter #20 )
- Missing Linpack and Eispack routines in libsci ( Newsletter #25 )
- F90 manual for Y-MP, no manual for T3D ( Newsletter #31 )
- RANF() and its manpage differ between machines ( Newsletter #37 )
- CRAY2IEG is available only on the Y-MP ( Newsletter #40 )
- Missing sort routines on the T3D ( Newsletter #41 )
- Missing compiler allocation flags ( Newsletter #52 )
- Missing compiler listing flags ( Newsletter #53 )
- Missing MXMA routine on the T3D ( Newsletter #75 )
- Missing getarg() routine on the T3D ( Newsletter #91 )
A Call for Material
If you have discovered a good technique or information concerning the T3D and you think it might benefit others, please send it to the e-mail address above and it will be passed on through this newsletter.Current Editors:
E-mail Subscriptions:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669 Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678 Arctic Region Supercomputing Center University of Alaska Fairbanks PO Box 756020 Fairbanks AK 99775-6020
-
Subscribe to (or unsubscribe from) the e-mail edition of the
ARSC HPC Users' Newsletter.
-
Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.