ARSC HPC Users' Newsletter 416, November 08, 2010



Greetings from the New Editor

[By Kate Hedstrom]

You’ve seen my ramblings here before and now I get to be an even bigger part of this newsletter. My primary job is working on an ocean circulation model with I/O in a NetCDF format. The model is running great, but dealing with all the input and output is a major headache!

If you’re wondering what’s on my mind these days, it’s Python and Perl. Python because some colleagues of mine are developing Python tools to handle everything from grid generation to making movies of model floats. (In this context, by "floats" I am referring to idealized point particles which are moved around by the currents in the model -- also called Lagrangian drifters.) I expect we’ll be putting these tools out on github when we’re ready, complete with a cmake build environment.

Then there’s Perl. I first learned Perl before there was a camel book, before there was Python. I have less immediate need for it, but I’m intrigued to find out there’s now a useful implementation of at least part of Perl 6 ready to download and install. It’s called Rakudo Star and it’s based on the Parrot project. One thing that isn’t yet in Rakudo is Perl Data Language (PDL) support, though it is promised in Synopsis #9. To find out more, check:

My husband and I also enjoyed listening to:

Another crazy project you might find interesting is described here:

John Graham-Cunningham is hoping to create Charles Babbage’s Analytical Engine, a computer designed back in 1837, but never built. If you think this is worth doing, you can help out with a donation.


Cmake Update

[By Kate Hedstrom]

The CMake article in the last issue is the result of sending a student ahead into the unknown. The decision to try CMake was based in part on my frustration when trying autotools with Fortran, described back in 2002 ( /arsc/support/news/hpcnews/hpcnews248/index.xml#article2 and /arsc/support/news/hpcnews/hpcnews249/index.xml#article1 ).

The CMake article generated a response from Jed Brown, rightly pointing out some omissions. His points:

1. You shouldn’t be setting compile flags because that reduces portability. Instead, CMake has a CMAKE_BUILD_TYPE variable which has options including Release and Debug. This is from the FAQ at :

Adapt the following commands in your CMakeLists.txt (this example sets the Release With Debug Information mode):

          "Choose the type of build, options are: None Debug Release
          RelWithDebInfo MinSizeRel." FORCE)

2. There’s no need to worry about the library being shared or not. Using simply:

  target_link_libraries(grid m gu)

should do the trick. However, Bob’s point is that we want to force the use of the shared library over the static one for loading into Python. One would have to try it to see if it worked.

3. One of the strengths of CMake over the alternatives is its ability to create config.h files for inclusion in your source code. Perhaps there will be yet another article in the future.

4. Most importantly, Bob was allowing the build to take place in his source directory. He has since been made to see the error of his ways. With CMake, you really, really want to build in a separate build directory. In fact, this is so strongly encouraged that CMake doesn’t provide a way to "make distclean" between differing builds. Instead, create a build directory, cd to it, then invoke cmake by pointing to the source directory:

     mkdir Build
     cd Build
     cmake path/to/sources

With this method, the Cache files and other generated files will be in the Build directory, which can be deleted for a fresh start.


Chugach and Pacman Updates

Status quo doesn’t last very long here at ARSC. We are continually working to keep our resources current for our users. This fall we also undertook a project to separate our machines according to our two main user communities, so that we will now allocate Chugach, our new Cray XE6, to our HPCMP projects, and Pacman and Midnight to our academic projects. At this writing, Pingo, the Cray XT5 is still running HPCMP work, but as soon as Chugach has completed acceptance testing, Pingo will be retired.

We hope to see Chugach generally available by the end of the year. It will provide a nice bump in processing power for us, with 728 compute nodes of dual 8-core AMD "Magny-Cours" processors for a total of 11648 processing cores, Cray’s new Gemini interconnect, and a feature Cray calls "Cluster Compatibility Mode" which will make the XE6 a better platform for commercial applications than the XT series. For those keeping score at home, with its 2.3 GHz processors, each of which can support four simultaneous floating point operations, the theoretical (guaranteed not to exceed) performance of Chugach will be

2.3 x 4 x 11648 = 107161.6 GFlops or about 107 TFlops

On the academic side, Midnight and Pacman are both getting pretty extreme makeovers. Midnight has been slimmed down to 256 X2200 (4-core) nodes and 35 X4600 (16-core) nodes for a total of 1584 compute cores. Future plans for Midnight include an update to its OS and other system software, tentatively in the first quarter of 2011. Pacman, on the other hand, is growing. By the time the dust settles this month, Pacman will boast 2032 compute cores with some interesting options for users:

  • 58 16-core compute nodes: (dual 8-core 2.3 GHz AMD Opterons), 64 GB memory per node (4 GB per core), 250 GB local disk
  • 40 12-core compute nodes: (dual 6-core 2.2 GHz Opterons), 32 GB memory per node (2.6 GB per core)
  • 4 Large memory nodes: Quad 8-core 2.3 GHz Opterons, 256 GB memory per node (8 GB per core), 1 TB local disk
  • 2 GPU nodes: dual 4-core 2.4 GHz Intel processors, 64 GB memory per node (8 GB per core), 2 M2050 nVidia Fermi GPU cards

These four node types can be selected by an option to the resource request to the Torque/Moab scheduler. For more information on the new Midnight and Pacman configurations, see midnight no longer available



Quick-Tip Q & A

A:[[ Some times I need to loosen up the permissions on a file so someone
  [[ else can access it. Of course, I use ’chmod’ to do this to the file and
  [[ possibly even the directories above the file. No matter how many times
  [[ this comes up, I sometimes fail to fully achieve my goal and need to
  [[ make a second effort. The problem is, the only way I can find out if I
  [[ failed is when the "someone else" tells me that they still can’t reach
  [[ the file.

[[ My question then is, is there a utility that will tell me if a specific [[ (or arbitrary) user can read/write/execute a given file?
# # From the editors #
This is a tough question and we got no responses that directly solve the problem. We did get a couple of replies that help cope with directory and file permissions management. Hopefully by having some tested approaches to changing permissions, setting them will be much less error-prone.
The answer to the question depends on a couple of issues: (1) What permissions are granted to the three classes of user (owner, group or everyone else); and (2) in which of these three classes does the user in question fit?
We can answer the first question with "ls -l" which shows not only file permissions, but the group associated with the file. The second question may be answered with either the "groups" command ("groups username") or the "id" command ("id username"). If the user is in the group of the file, then "group" permissions apply, otherwise "others" permissions do.
Finally, don’t forget that permissions must be set appropriately for the enclosing directories, and execute permission must be set if you want the user to be able to see the file in the directory. These little complexities are, we are sure, what prompted the question in the first place.
# # Hank Happ had this tip: #
Sometimes it is necessary to give group "write" permission to directories and files on machines so other "group" users can access your files. This also means giving the group "execute" permission on directories. Unfortunately, if you simply use the command "chmod -R g+wx", then all the files will have "execute" permissions, which you don’t want. The trick is to give only subdirectories execute permission.
Here is a simple solution. - Starting one level above the directory you want to give permission for, type "chmod -R g+w directoryname" to give write permission on that directory and all files and subdirectories beneath. - Give execute permission on that directory with "chmod g+x directoryname". - Then give execute permission on subdirectories only with "find directoryname -type d -exec chmod g+x {} \;"
# # Linux has a nice shortcut for this operation: #
The capital X in the command "chmod -R g+rwX directoryname" says to make the new permission executable if either (1) the file was already executable, or (2) the file is a directory.
# # And Tony Craig continued the theme of "getting the permissions set # correctly in the first place" with this reply: #
I don’t have an answer to the question above, but can provide some aliases that address a slightly different issue that collaborations always run into. That’s how do I quickly change permissions for a large tree of dirs and files and make sure everything is group (or something else) accessible.
I have the following aliases setup for my .cshrc shell. These aliases change all file or dir permissions below the location they are executed. These are extremely useful for setting proper permissions on both dir and file for a large tree.
alias pubdir ’find . -type d -exec chmod ugo+rx {} \; &’ alias pubfil ’find . -type f -exec chmod ugo+r {} \; &’ alias pubdirw ’find . -type d -exec chmod ug+w {} \; &’ alias pubfilw ’find . -type f -exec chmod ug+w {} \; &’ alias pubfilxw ’find . -type f -exec chmod ugo-w {} \; &’
If I want to make all files in a directory tree group readable, then I cd to the top of the tree and execute pubdir and pubfil and I’m done. I need to make sure both the directories are set to the proper permissions as well as the file. It’s easy to setup various aliases for different permission settings depending on need.
# # Wrap up from the editors #
One more item to mention along these lines is a Perl script that was submitted by Dale Clark, way back in Issue 220 of this newsletter. Take a look at: /arsc/support/news/hpcnews/hpcnews220/index.xml#qt We don’t have expiration dates here, so for this or any other questions, if you ever stumble across something helpful you are encouraged to send it in and we will do a follow-up.

Q: On Solaris there’s a cool command called "pargs" that can list the environment variables set for a running process.
host 13% pargs -e 28119 28119: -bash envp[0]: USER=myusername ... envp[12]: DISPLAY=localhost:15.0
There’s no pargs command on Linux. Is there a way to get the environment variables for a process I own on a Linux system?

[[ Answers, Questions, and Tips Graciously Accepted ]]

Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top