Fish User Guide (Cray XK6m-200)

Introduction

The Arctic Region Supercomputing Center (ARSC) operates a Cray XK6m-200  ( fish ) running the Cray Linux Environment version 4 (CLE4).

Fish is a resource dedicated to University of Alaska affiliated academic users performing non-commercial, scientific research of Arctic interest.

"Fish" Hardware Specifications

Fish cabinets image

The Cray XK6m-200 at ARSC consists of the following hardware:

  • 2 Login Nodes: "fish1.arsc.edu" and "fish2.arsc.edu"
    • One six core, 2.6 GHz AMD Istanbul processor per node
    • 16 GB of memory per node. 
  • 48 GPU Enabled Sixteen Core Compute Nodes
    • One sixteen core, 2.1 GHz AMD Interlagos Processor per node
    • 64 GB of memory per node (4 GB per core)
    • One nVIDIA Tesla X2090 GPU accelerator with 6GB RDDR5 memory
  • 32 Twelve Core Compute Nodes
    • Two six core, 2.6 GHz AMD Istanbul processors per node
    • 32 GB of memory per node (2.5 GB per core)
  • Cray Proprietary Gemini Interconnect
  • 20 TB Lustre $HOME file system

Operating System / Shells

The operating system on fish is the Cray Linux Environment version 4 (CLE4).  Under the CLE4 environment, the fish login nodes run 64-bit SUSE Linux.   The fish compute nodes run Compute Node Linux (CNL) - a lightweight operating system custom designed to improve resource availability and scalability for user jobs.  

The following shells are available on fish:

  • sh (Bourne Shell)
  • ksh (Korn Shell)
  • bash (Bourne-Again Shell) default
  • csh (C Shell)
  • tcsh (Tenex C Shell)

If you would like to change your default login shell, please contact User Support .

System News, Status, and RSS Feeds

System news is available via the news command when logged on to fish . For example, the command "news queues" gives news about the current queue configuration. System status and public news items are available on the web.

Storage

This system provides access to portable data storage directories, easily usable through environment variables. Please be familiar with the purpose and policies of each storage directory.

Available Software

Open source and commercial applications have been installed on the system in /usr/local/pkg.  In most cases, the most recent versions of these packages are easily accessible via modules.  Additional packages may be installed upon request.

Sample Code Repository ($SAMPLES_HOME)

The $SAMPLES_HOME directory on fish contains a number of examples including, but not limited to:

  • Torque/Moab scripts for MPI, OpenMP and Hybrid applications
  • Examples for Installed Commercial Software Packages and other installed applications.
  • Examples using common libraries

User Installed Software

This system provides a repository for third-party software packages which are installed by users but are not supported by ARSC.

Parallel Programming Models

Several types of parallelism can be employed on fish using different programming models and methods.

Programming Environments

Fish provides multiple compiling environments for different programming languages and compiler brands. The modules package is installed, which allows you to quickly switch between these different environments.

Compiling and Linking with PGI

Job Submission and Resource Accounting

Job submission is done through the Torque/Moab scheduler. Resources are allocated in CPU hours and are managed on a per-project basis.

Back to Top