ARSC HPC Users' Newsletter 229, September 28, 2001

ARSC Fall 2001 Training

    Oct 10, 2001, 3:30-5pm:   User's Introduction to ARSC Supercomputers 
    Oct 17, 2001, 2-4pm:      Introduction to Unix 
    Oct 24, 2001, 2-4pm:      Productivity Tools 
    Dec 12, 2001, 2-4pm:      Visualization Case Studies 

Course descriptions and registration are available at:

"dmcp" and Group Sharing of Migrated Datasets

dmcp is a new command, installed on ARSC's Crays to simplify sharing of DMF migrated files between group members.

You won't put fellow group members over quota when you use dmcp .

dmcp offers an alternative solution to an old, vexing problem which exists because, while only the owner of a file can use dmput to migrate it off-disk, ANYONE with read access to a file can bring it on-disk where it counts against the owner's quota.

From "news dmcp," here's the description:

 dmcp copies a file. If the file is DMF migrated, dmcp will copy it
 without retrieving the original file to disk.
 The purpose of dmcp is to allow group members to share migrated files
 without impacting each other's disk quotas.
        dmcp [-f] from_file to_file
       A file in a local filesystem to which you have read access.
       This is the source file.
       The destination file.
        -f : forces overwriting to_file even if a file with that name
             already exists.  [Default: Does not overwrite.]
  1)  The syntax of dmcp is not equivalent to that of cp. It is 
      modeled after dmcopy.
  2)  dmcp works on local filesystems only.  
      Thus, when logged onto yukon, /allsys files are not accessible to
      dmcp. (/allsys is a chilkoot disk, NFS mounted on yukon.)  Remote
      files are accessible to cp, but if they are DMF migrated, cp will
      retrieve them to disk and they will impact the /allsys quota of the
      owner of the file.
  3)  It is undesirable to have multiple copies of a file in existence:
      there is the risk of them becoming inconsistent, and the extra
      storage can be costly.  Thus, you should consider removing files
      obtained with dmcp as soon as you're done with them.

Here's a scenario in which dmcp can be helpful: one group member has 50, 4 GB data files in his account, and they're all migrated to tape where they don't impact his 10 GB disk quota. Other group members, working independently on the same data set, retrieve and read one file per run, using Fortran OPEN and READ statements. Unable to dmput the files, the group members will put the owner of the files over quota on their third run, at which point all progress will stop.

In the past, groups at ARSC have emailed the owner of the files, telling them when to run dmput, or have tried the rather complicated solution, which runs dmput out of a cron job, described at:

> /arsc/support/news/hpcnews/hpcnews222/index.xml#qt

With dmcp , the owner of the files does nothing. Group members must preemptively dmcp a needed file (to TMPDIR, probably), point their Fortran OPEN statement to the copy, and after the run, rm the copy. This can all go in a qsub script.

Review ARSC's storage management policies to learn about your file system options, quotas, and DMF:

OMP_DYNAMIC Always Enabled Under UNICOS

The article on mixed-mode MPI/OpenMP codes in the last newsletter gave SGI Onyx and IBM SP examples, but never mentioned the Cray SV1. For those who were wondering, it was because the SV1 never spawned the expected number of OpenMP threads, and we didn't understand why.

Jim Long of ARSC gets the credit for ferreting out the answer:

Under UNICOS, OMP_DYNAMIC is _always_ enabled. From the CF90 Commands and Reference Manual, S-3901-35 :

2.3.16 OMP_DYNAMIC (UNICOS Systems Only)

The OpenMP Fortran API defines the OMP_DYNAMIC environment variable as one that can enable or disable the dynamic adjustment of threads available for execution of parallel regions. On UNICOS systems, however, the dynamic adjustment of threads is always enabled and cannot be disabled.

If a program calls the OMP_SET_DYNAMIC(3) library routine with an argument of .FALSE., intending to turn off dynamic adjustment of the number of threads, the library routine is ignored. The OMP_GET_DYNAMIC(3) routine always returns .TRUE..

Dynamic adjustment of threads in OpenMP is a runtime feature that allows the number of active threads to match the number of available processors. Although the OpenMP standard allows turning this feature off, it is not adjustable on UNICOS systems like chilkoot. It is implemented to avoid over-subscription of processors, leading to performance degradation. Overall system load is monitored and resources allocated accordingly.

OpenMP requires, however, that the requested number of threads be adjusted only during serial portions of the code. The number of threads allocated for a parallel region remains unchanged throughout that parallel region.

CUG Fall SGI Workshop Postponed

The CUG SGI Fall SUMMIT conference scheduled for Oct 22-24 in Denver has been postponed. See: for details.

Quick-Tip Q & A

A:[[ I use C, C++, and Fortran 90/95. I need to generate file names
  [[ which must be unique, to be used for temporary, or scratch files.
  [[ Any suggestions?

As the reader response shows, you can roll your own or use a library

Fortran programmers can use TEMPNAM or TMPFILE on some systems.  The
Fortran standard also includes:


for opening a scratch file in the directory named by the environment
variable, "TMPDIR". This file will have a unique name, and will be
automatically removed at the end of the run.

For C/C++ programmers, we have:

From Richard Griswold


For C and C++ on Unix systems, you can use the tmpnam() function to get
a unique file name.  If your system has it, you can use mkstemp()
instead since it will also open the file for you, ensuring that no one
else gets the temp same name and opens the file before you do.

Other variations of these two functions are are tempnam() and mktemp().

From Brad Chamberlain


In the ZPL compiler, we generate unique names using the process ID and
user ID for the compiler, as follows:

/* IN:  path = base pathname
        tail = tail end of filename
   OUT: name = complete filename */
  pid_t pid;
  uid_t uid;

  pid = getpid();
  uid = geteuid();

  sprintf(name, "%s/zc-%d-%d-%s", (path ? path : "."),
          pid, uid, tail);

If multiple filenames are required, an additional static integer variable
counter could be appended onto the string to uniquify each name created by
the program.

While this does not guarantee a unique filename, the only way a name can
be duplicated is for the same user to happen to get the same process ID
twice during compilation.  This is unlikely, especially since these files
are temporary and cleaned up after a compilation.  Files that are expected
to persist for arbitrary amounts of time would require a more coordinated
effort (probably using files on the file system itself) to ensure that
names didn't conflict.

Q: I have a wallop*** of text files.  It's easy to list those which
contain some word, for instance, "dungarees:"

  grep -i -l "dungarees" *.txt

However, I need the files which do NOT contain the word.  This command:

  grep -i -l -v "dungarees" *.txt

went berserk! It claimed that _every_ file does not contain dungarees,
which is false!  Does grep have some bug?  What should I do?

  ps: what IS the collective for files?  ("Wallop" just sounded good...)

[[ Answers, Questions, and Tips Graciously Accepted ]]

Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top