Loader Directives

NOUSXMSG and USX in Particular

If you know that code contains unsatisfied external references (USXs), but you want it to link anyway, you must tell mppld to ignore them. Use one of the loader directives, NOUSXMSG or USX.

For example, MPICH 1.0.13 references fork and execvp, which don't exist in the T3D libraries. As noted in > Newsletter #95 , "cc" invokes mppld. The two USXs will be detected, and by default, in mppld, a USX is given a "WARNING" message level which results in a non-executable output.

If you give the loader the following directives, however, it will ignore the specified USXs, but still catch and treat any others as "WARNINGS":

NOUSXMSG=fork NOUSXMSG=execvp

Another option is to reduce the severity of a USX message. The following directive tells the loader to give any and all USXs the "CAUTION" message level, so, although you will be warned, the result will still be executable:

USX=CAUTION

You can also combine these directives. With the following settings, "fork" and "execvp" will be ignored (you will get no feedback about them); the loader will warn you, however, of any other USXs; but regardless of USXs, the output binary will be executable:

USX=CAUTION NOUSXMSG=fork NOUSXMSG=execvp

Once you determine which directives are necessary, there are a few ways to issue them:

1. List the directives in a file (e.g., "direc.list" in the current directory), and tell the loader to scan the file. The command line flag "-j" (for mppld) or "-i" (for mppldr) lets you specify a directive file.

   For example:

   cc prog.c -Wl"-j ./direc.list"

2. Give them on the command line using the "-D" loader flag.

   For example:

   cc prog.c -Wl"-DUSX=CAUTION"

3. List them, delimited by semi-colons, in the environment variables MPP_SEGDIR (for mppldr) or MPP_LDDIR (for mppld).

   For example:

   denali% export MPP_LDDIR="USX=CAUTION"

   denali% cc prog.c

As noted in > Newsletter #92 in the article on "mppfixpe," mppldr is invoked at run time EVERY time you run a plastic executable. The only way I know to pass directives to mppldr at run-time is via environment variables.

For instance, if you wanted to use "cc" to compile and link a plastic executable but you wanted to ignore, at both compile and run-time, the USXs, fork and execvp, you could do the following:

  denali%  export MPP_LDDIR="NOUSXMSG=fork;NOUSXMSG=execvp"
  denali%  cc prog.c
  denali%  export MPP_SEGDIR="NOUSXMSG=fork;NOUSXMSG=execvp"
  denali%  a.out -npes 4

Life is easier for Fortran programmers. The Fortran compiling systems invoke mppldr, so if you set MPP_SEGDIR for the compile/link phase, the same settings will be used at run-time:

  denali%  export MPP_SEGDIR="NOUSXMSG=fork;NOUSXMSG=execvp"
  denali%  f90 prog.c
  denali%  a.out -npes 4

Jobs blocked by PE configuration

At one point last Friday, mppview showed a "normal" T3D pattern, something like this:

 _________________________________________________________________________
 \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \
  \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\
 _________________________________________________________________________
 \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \ 
  \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\
 _________________________________________________________________________
 \ BBB      BBB      BBB     BBB       CCC      CCC      CCC      CCC     \
  \ BBB      BBB      BBB     BBB       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\
 _________________________________________________________________________
 \ BBB      BBB      BBB     BBB       CCC      CCC      CCC      CCC     \
  \ BBB      BBB      BBB     BBB       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\

Part    User    PID    Program  State  Flags    Shape- YZX   (base)   Elapsed
----  -------- ------ -------- ------ ------ ----------------------- ---------
        BBB     90285 a.out    Active B      32= 2x 2x 8  
        ...

A few minutes later, it looked something like this:

 _________________________________________________________________________
 \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \
  \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\
 _________________________________________________________________________
 \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \
  \ AAA      AAA       .        .       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\
 _________________________________________________________________________
 \ DDD      DDD       .        .       CCC      CCC      CCC      CCC     \
  \  .        .        .        .       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\
 _________________________________________________________________________
 \ DDD      DDD       .        .       CCC      CCC      CCC      CCC     \
  \  .        .        .        .       CCC      CCC      CCC      CCC     \
   \________________________________________________________________________\


Part    User    PID    Program  State  Flags    Shape- YZX   (base)   Elapsed
----  -------- ------ -------- ------ ------ ----------------------- ---------
       BBB      90285 a.out    Wait   B~p      32= 2x 2x 8  
       EEE      45257 a.out    Wait   I~p      8= 2x 1x 4  
       ... 

The 32 PE job in the NQS script which user "BBB" was running had completed, and before the next 32 PE job in "BBB's" script was initiated, user "DDD's" 8 PE job had started up, taking some of the PEs vacated by "BBB's" first job. Unfortunately, the shape of the available 32 PE block was then 2x 4x 4, and with the current T3D setup, 32 PE jobs require a shape of 2x 2x 8.

"BBB's" second job, being first in line waiting for PEs, then blocked all subsequent jobs, like "EEE's" little interactive job. I mistakenly thought that the problem was with "BBB's" job, and asked him to qdel it, which he did. But NQS then submitted another user's 32 PE job, which promptly became blocked, as well. The block was finally removed when "DDD" killed his 8 PE job. (My thanks to patient and helpful users.)

Once again, we've found a situation in which PEs remain unnecessarily idle.

Interactive T3D Newsletters (yeah, sure)

By the time this Newsletter slaps the virtual porches of our subscribers, ARSC's webmaster expects to have added new buttons to the bottoms of each of the archived issues. For the "point and click" crowd, you will be able to walk forward or backward through the issues or even search them for an interesting topic, say, "CRAFT."

A: {{ What's a convenient way to "vi" every file which contains a given 
      string, in the current working directory,   (E.g., You want to read
      every T3D Newsletter which mentions "CRAFT".)  }}

    vi `grep -l CRAFT *`   # Sent in, with variations, by two readers.
                           #   Be sure to use the backward slanting
                           #   single quotes, (`).  The output of the
                           #   quoted command is broken into space-
                           #   delimited words and substituted for
                           #   the original command.  Thus, the files
                           #   found by "grep" become arguments to "vi."


Q: Using "vi," how would you insert a "C" at the beginning of each
   line of text for a block of lines (i.e., comment out some Fortran code)?

[ Answers, questions, and tips graciously accepted. ]

Ed Kornkven	ARSC HPC Specialist	ph: 907-450-8669
Kate Hedstrom	ARSC Oceanographic Specialist	ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020

Subscribe to (or unsubscribe from) the e-mail edition of the ARSC HPC Users' Newsletter.

Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.