ARSC HPC Users' Newsletter Quick-Tip Index

Select category: All Quick-Tips Programming Unix Utility Scripting Technical General

Symbol A B C D E F G H I J K L M N O P Q R S T U V W X Z
  • Symbol
  • .netrc fileauto-initialize ftp sessions and define ftp macros 273
  • /dev/stdinaccessing stdin as a file 338
  • [shell syntax for conditional tests, "/usr/bin/[" versus "/usr/bin/test" versus "[[" 348
  • A
  • AMD Opterondetermine the core a process is using 396
  • AutotoolsMakefile style advice 418
  • aliasesreader favorites 146 290 363
    using within scripts 342
  • apportioning K tasks among M PEs 132
  • arrunning ar command on 32 and 64 bit libraries on IBM systems 333
    using ar or ld to combine .o files and reduce over-sized argument string 300
  • argument list too longwhen the shell can't handle wildcard expansion 216 300
  • assign commandtranslating unformatted files 219
  • automobile engine heaterunplugging 156
  • awkchecking number of inputs per line 409
    examples 272 316 337 356 370 375 391 409
    extracting fields from ls output 356
  • B
  • BLACSglobal reductions with BLACS, coding from communication primitives 351
  • babelconnecting C/C++ to python 344
  • base conversionconverting numbers between bases in Unix 282
  • basename/dirname commandsreturn file name or directory path from a path/filename string 268
  • bashcommand completion in bash 353
    command line editing 397
    extracting the extension from a filename using pattern matching operators 347
  • batchsubmitting batch jobs in IRIX 249
  • batch removeusing Unix commands 179
  • bcevaluating math expressions found in a text file 285 406
  • bearsdefense against 197
  • blankets on your carwarming your car when it's cold out 334
  • blind directoriessafer file sharing 191
  • book reviewscientific books 320
  • bucket sortfaster than quicksort 141
  • C
  • CC vs Fortran -- default array indexing 154 155
    calling C from Fortran -- portable interface 260
    comment blocks for documentation systems 367
    commenting out commented code, or, how to nest "comments" 339
    finding path of an executable 183
    linking gridinfo, gridinit, and FFT libraries with C/C++ code 233
    path to executable 183
    pointer pointer 394
    pointer reference 159 394
    preprocessor directives with make 326
    pretty-printer 295
    programming style question, how would you choose the minimum 264
    question-mark colon construct 127
    retaining the precision of output doubles using format specifiers 361
    trapping floating point errors using gcc feenableexcept 376
    using realloc 394
    viewing arrays in totalview 145
  • C++confusion between "cpp" and "CC" commands 279
    copying values from a C style array to a STL vector 350
    linking to C libraries 308
    using the STL map class 399
    using the STL vector assign member function 350
    using the STL vector reserve member function 350
    using the stringstream class 373
  • C/C++calling from python 344 349
  • CDPATHjumping between directories 123
  • CPUdetermine the core a process is using 396
  • CVSalternative to diff 313
    version control and software maintenance 278
  • Catch-22 257
  • Co-Array Fortranarray bounds 154
  • Collectivescollective nouns for supercomputer related terminology 384
  • Compiler options-O3,aggress 166
    -g 166
    from the executable file, determine what compiler flags were used 368
  • Cray Fortran(/ ... /) syntax -- array constructor 157
    -Fmacro expansion 174
    -Oinlinefrom 207
    -Rbarray bounds checking 129
    -Vversion 149
    -eIimplicit none 129
    -rxreport on variable usage and cross-references 221
    -s cf77typestype compatibility with Fortran 77 174
    -static 202
    ALLOCATE statement -- testing for success 135
    Directive for unwinding loops 292
    performance comparison of DO-IF vs WHERE 205
  • cURLcommand-line retrieval of web pages 311
  • carpal tunnel syndromeavoiding 113
  • chgrp 192
  • chmod-R go+rXintelligently add execute bit 147
    1770setting the sticky bit 109
    changing entire path to a selected file, but not recursively 220
    g=x ~blind home directory 191
    group/world executable directories put at risk all files within 298
  • code-blindnesscures 271
  • col-bxconverting man pages 118
  • commusing the comm command to compare files 362
  • commasreformatting long number strings with comma separators 329
  • compilersdetermining version 149
  • core filesdeterminine origin of 200
    limiting size 121
    managing files from many processes 403
  • cp--parentspreserving parent directories (GNU cp) 377
    -i 187
    -pcopying modification time along with the file 231
  • cpuUNICOS /etc/cpu command for control over caching 209
  • cshfile name completion 176
    redirecting stderr and stdout 223
  • csh/tcshcommand line editing 397
    extracting the extension from a filename using variable modifiers 347
  • cutextracting columns of data from text files 272 337 356
  • D
  • DISPLAY environment variableIncorrect setting causes console session failure 243
  • DMFFile compression and affect on DMF migration 188
    How to dmput files owned by other members of a group 222
  • DoxygenFor code documentation 367
  • DuplicatesFinding duplicate files 392
  • date+%Y%m%dCreating file versions with a date stamp 310
    --dateTiming a series of jobs 393
    Generating a series of file names containing dates 398
  • date stampCreating file versions with a date stamp 310
  • dcdesktop calculator 167
  • debuggingfinding problem in one of many source files 275
  • deleting ^M characters in batch 139 287
  • deleting the file "-i" 102
  • development environmentsreader favorites 341
  • diamond as conductor 120
  • diff"modern" alternatives to diff (tkdiff, ediff, vimdiff, etc...) 313
    -D 124
    comparing a file to stdin 338
    comparing a remote file to a local file 338
    ndiff, numerical differences allowed and handled intelligently 378
  • dirname/basename commandsreturns directory path or file name from a path/filename string 268 401
  • dmgetmost efficient use 162
  • dotfilesmaintaining consistent dotfiles across multiple systems 312
  • du-minclude Cray DMF migrated files 164
    -skexcludes Cray DMF migrated files, under-estimates size of tar file 267
    -sktotaled disk usage in KBs 164
    -sktotal size of subdirectories 405
  • E
  • e-mailsending automatic vacation messages 143
  • e-mail from NQS 138
  • echofor shell preview of wild-card expansion 256
    in .bashrc causes scp failure 419
  • editorsfavorite development environments 341
    using text editors to copy/paste rectangular blocks 272
  • egrepadvanced filtering examples 161 168 337 389
  • emacsdiff-like features 313
    getting colors to work on any platforms 297
    tramp to access remote files 314
  • env/usr/bin/env searches PATH for binary 372
  • environment variablesappending to $PATH, $LM_LICENSE_FILE, etc. 245
    available to running NQS jobs 198
    show environment of a running process 417
    testing if set 319
  • escapescontrolling the interpretation of regular expressions 280
  • evalevaluate variable as a perl program 406
  • exsingle command line "ex" command prepends file A to file B 302
  • execUnix exec command, for redirecting stdout to file for entire session 388
  • exit statusof a pipeline 412
  • expand commandreplacing tabs with spaces 203
  • expectan expect script for ftp initialization 273
    timeout 411
  • explainCray's "explain" command prints details on many warnings/errors 293
  • exprdifferences 237
    using regular expression to extract a substring 347
  • eye strainreducing 106
  • F
  • FFTlinking gridinfo, gridinit, and FFT libraries with C/C++ code 233
  • FortranCYCLE and EXIT 248 249
    DO loop iterator exit value, formula for computing 325
    Fortran 90 INTEGER constants 215
    Fortran 90 MODULE replaces Fortran 77 COMMON 199
    Fortran 90 OPTIONAL subroutine args and effect on inlining 208
    Fortran 90 generic functions 215
    Fortran vs C -- default array indexing 154 155
    IO buffering problems caused out-of-order output, fixed with flush 386
    LOG10 vs ALOG10 158
    NAMELIST 213
    OPEN (RECL=1024) 144
    RESHAPE 228
    WRITE 144
    WRITE with implied DO incompatible with read with explicit DO 284
    calling C from Fortran -- portable interface 260
    calling from Python 421
    f90 type inconsistency 174
    free vs fixed format discussion 261
    header and include files 171
    intrinsic functions 158
    pretty-printer 295
    trapping floating point errors using gcc feenableexcept 376
    unformatted files and porting them 219
    using arbitrary array bounds 154 155
  • file commandprobe file type and other info 112 200 332
  • filesaccess time 178
    batch editing in place 126
    compression and its affect on DMF migration 188
    determining source or type of 112
    generating unique file names in C/C++/Fortran programs 229
    remote access 402
    sharing with your group without compromising home directory 191
  • find-ctimeto find recently changed files 405
    -execusing with grep 283
    -execusing with tar 331
    -exec and -typeusing -exec with chmod and -type 332
    -mminto find recently modified files (GNU find) 346
    -mtimeto find recently modified files 346
    -pruneprevent descending into subdirectory 181 187
    -sizefinding large files to help stay under quota 259
    -type mfinding migrated files 162
    examples 147 187 230 283 331 332 357 405
    preserving files which match a wildcard expression, deleting the rest 301
    removing core and mppcore files 107
  • ftpautomating ftp sessions with batch scripts and initialization macdef macros 273
    how to "more" remote and local files 105
    mget/mput a batch of files without prompting 277
  • G
  • gccinline assembly 396
  • gdbstack traces from many files 403
  • grep--signifies end of options 226
    -llisting file names 103
    -oShow only the part of a matching line that matches the pattern 358
    -vexcluding grep itself from ps output 224
    list files not containing certain text, using find -exec 230
    list lines before search pattern 345
    listing file names even when only one file searched 283
    searching for multi-line patterns 371
  • grmview-deqdetermining memory usage at runtime 134
    -l 134
  • groupgroup/world executable directories put at risk all files within 298
    what determines group ownership of a new file 192
  • gzipDetermining if a zipped file is okay without unzipping it 321
  • H
  • Haskellexamples 375 410
  • hexidecimalconverting decimal to hex and back in Unix 282
  • hidden characters in file names 180
  • I
  • ImageMagick for postscript to png image conversion 374
  • Initializing program parametersavoiding recompilation 213
  • implicit none 129
    adding implicit none and explicit declarations to legacy codes 221
  • inliningimpeded by Fortran 90 OPTIONAL arguments and one workaround 208
    inlinefrom required by Cray Fortran if source in separate files 207
  • interprocess communicationcommunicating Perl processes 407
  • J
  • jajob accounting command 190
  • K
  • kdestroyremoving Kerberos tickets 288
  • kerberoskerberos skew too great 189
    next token mode 169
  • keyboard keys"enlarging" a too-small key on the keyboard by remapping it 352
  • killaborting MPP jobs 125
  • krcpcopying directories recursively 343
  • ksh"_" environment variable 183
    CDPATH environment variable 123
    PS1, favorite settings for your shell prompt 349
    command line editing 397
    file name completion 176
    history file environment variable, HISTFILE 317
    integer keyword 354
    loops in ksh 354 370
    prompt, favorite settings for your shell prompt 349 424
    redirecting stderr and stdout 223
    typeset declaration 385
  • L
  • Linuxcommands for beginners 408
  • Loadlevelergetting number of nodes and procs 316
  • ldusing ld or ar to combine .o files and reduce over-sized argument string 300
  • librariesdetermining which used 168
    show which shared library each symbol in an executable is provided by 383
  • limit-d dlimlimiting size of core files 121
    setting maximum stack frame size when your code won't compile 202
  • listserverproblems unsubscribing 218
  • llsubmitstarting batch job in directory from which it was submitted 289
  • ln-spreventing core file creation 114
  • lock filecommunicating Perl processes 407
  • ls-Ssort by file size 405
    -boctal codes for nongraphic characters 180
    -dlisting directories only 391
    -qquestion marks for nongraphic characters 180
    -tsort by last time file was modified 405
    -usort by last time file was accessed 405
    colorized output 415
    when directory is not "executable" 244
  • ls *listing two subdirectory levels 128
  • lslppgetting XLF version numbers 318
  • M
  • MFLOP/Smeasuring on T3E 153
  • MPIMPI_Abort for aborting MPP jobs 125
    MPI_Bcast vs SHMEM_BROADCAST 195
    MPI_Comm_dup and creating your own MPI libraries 232
    MPI_Rsend, synchronizing with MPI_Recv 291
    displaying system information from an MPI program 399
    how to preface stdout with task number 255
    message tags, coordinating sends and recvs 210
    using PMPI to build custom MPI subroutine timers 382
  • MPI compilershow the flags the MPI compiler passes to the system compiler 390
  • Minneapolisbest restaurants 170
  • make.PHONY target 400
    Makefile style advice 418
    dependency for Makefile itself 111
    parallel 148
    passing macro definitions from "make" compile line through to compilers 297
    preprocessor directives with make 326
    unconditional make 400
  • man pagesadding personal man pages to search path 186
    stripping format chars 118
  • md5sumusing to find duplicate files 392
  • memorizationhow to remember numbers 305
  • memory for T3E compilergetting (or not) more of it 251
  • memory utilizationdetermining at run-time with ja command 190
  • merging source code versions 124
  • mkdir-pmaking complete tree 172
  • modules (Fortran 90)initializing variables 199
  • modules (UNICOS environment)"module load pghpf" command 151
    creating custom Cray Programming Environment modules 186
    listing compilers, libs, tools, etc. and versions 117
    trouble calling "module" in qsub script 206
  • mosquitosrepellents, traps, and head nets 250
  • mpirun-ffor complex argument lists 211
    timing a series of mpirun commands to evaluate performance changes 393
  • mpixx, mpicc, mpixlf90show the flags the MPI compiler passes to the system compiler 390
  • mppcore filesdeleting 107
    storing on /tmp 114
  • mppsize command 112
  • mppview-tdetermining memory usage at runtime 134
  • mppwidthfetching core count within a job 404
  • mv-i 187
    moving a directory into itself 225
    pitfalls 177
  • N
  • NQS daemon 175
  • NQS jobautomatically creating unique directory for output 269
    checking stdout/stderr while still running 217
    determining if checkpointed 140
    preventing rerun 119
    vanishes 138
  • NetCDFnetCDF Operators (NCO) 378
    testing the equivalence of netCDF files while allowing for differences in precision 378
    viewing netCDF data 327 378
  • New Years Computing Resolutions 236
  • ndiffdiff of files, but with numerical differences allowed and handled intelligently 378
  • nmshow which shared library each symbol in an executable is provided by 383
  • numeric constantsdeclaring in Fortran 90 to be of a particular type 215
  • numerical precisionnumerical precision and roundoff error summing real numbers 366
  • O
  • OBJECT_MODErunning ar command on 32 and 64 bit libraries on IBM systems 333
  • OS version numberdiscover using "uname" command 204
  • OpenMPPermissible C/C++ "for" loops 253
    SINGLE vs MASTER comparison 212
  • od-cviewing files 180 287 337
    -tdisplay binary data converted to standard types 365
  • optimization level defaultsCray, SGI, IBM compilers 235
  • P
  • PATT3E performance analysis tools 153
  • PATHlisting one path per line in a colon-delimited list of paths 265
  • PBSscript generation 395
    testing if PBS_O_WORKDIR set 319
  • PEvers (programming environment components) 117
  • PIPESTATUScapture pipeline exit status 412
  • Pigetting machine precision value for Pi in C/C++ and Fortran programs 263
  • parallel make 148
  • parallel random number generation 133
  • passwordsinvent and remember 131
    remembering 324
  • pathhow-compiledPathscale compiler option to determine, from the executable, what compiler flags were used 368
  • performance improvementgeneral advice 166
    timing a series of MPI jobs 393
  • perl".." (ranging operator) many nifty uses 163
    -eentering script on command line 163 265
    -p -iedit files in-place, assumed loop 126 208 274 304
    Algorithm::Permute module 410
    File::Basename module 401
    FindBin module 401
    Schwartzian transform 375
    Time::Local module 398
    alarm 411
    checking number of inputs per line 409
    evaluating math expressions found in a text file, and s///e 285
    examples 241 262 265 287 304 322 329 356 357 358 370 375 409
    extracting fields from ls output 356
    map 358
    recovering from an untar in the wrong place 357
    script for condensing a file 116
    slurp mode 371
  • perl regular expressionslook-ahead assertion 358
    minimal, or non-greedy, matching 262 358
  • perl xdumpread binary files and convert 365
  • permissionssetting for a directory tree 416
  • pghpfaccessing man pages 151
  • pgreplist processes that meet criteria 407
  • pipefailcapture pipeline failure status 412
  • postscript to png image conversionImageMagick, gs, gimp, Netpbm, pstopnm, and ps2png 374
  • preprocessingcpp examples 260 339
  • program memory usagehow to determine 134 137
  • ps-e option to show environment of a process 417
    -p $$to determine which shell I'm using 294
    extracting grep process from grepped output 224
  • python"/usr/bin/env python" searches PATH for python binary 372
    calling C/C++ from 344 349
    calling Fortran from 421
    evaluating math expressions found in a text file 285
    examples 375 392
    extracting matched patterns from lines of input 358
    glob module 385
    itertools module 410
    os.path.walk 392
    set type in python 362
  • Q
  • qalteraltering NQS requests at run-time 150
  • qdeldeleting a list of jobs 381
  • qsub-jjob log 140
    -l mpp_tMPP time 136
    -l p_mpp_tMPP time per process 136
    -nrno restart 119
    -rocheck stdout while job still running 217
    chaining a sequence of jobs 150
    specifying shell to use 206
    starting batch job in directory from which it was submitted 198
  • quotahow to locate all your big files so you can move or delete them 259
  • quotescontrolling the interpretation of regular expressions 280
  • R
  • readexample of shell read construct 356
  • redirecting stdoutUnix exec command, for redirecting stdout to file for entire session 388
  • remap keyboard"enlarging" a too-small key on the keyboard by remapping it 352
  • renamerenaming files 322 385
  • resetcommand for rescuing corrupted terminal session 361
  • rm-i and -fusing together 270
    deleting a file named "-i" 102
    disabling "rm *" (while otherwise allowing "*") 355 356
    preserving files which match a wildcard expression, deleting the rest 179 301
  • rmdir-pdeleting directory tree 172
  • roundoff errornumerical precision and roundoff error summing real numbers 366
  • rsynccheck for differences in a remote file 413
    copying files 423
    preserve file ownership and group settings during file transfer 364
  • rubypermutation library 410
  • S
  • SHMEMSHMEM_BROADCAST vs MPI_Bcast 195
    shmem_fence vs shmem_quiet 194
  • SV1disabling caching 209
  • SWIGconnecting C/C++ to python 344 349
  • SecurID Cardgenerating a sequence of valid codes w/out reentering PIN 196
    handling & storing 160
    obtainin a new PIN 201
  • Sourdough Sam Math contest 165
  • Stack frame size too largeSGI f90 warning 202
  • scp-pto preserve dates during transfer 314
    copying directories recursively 343
    echo command in .bashrc causes failure 419
    using remote environments with scp 340
  • scurvy, avoidingTips for impoverished students 369
  • search and replaceas smart as you want, across many directories and files 208
    removing unwanted octal and extended ASCII codes from text files 287
  • sedexamples 139 216 241 283 290 304 315 322 323 329 349
    search replace on slew of files 304
  • sequsing seq to generate a range of dates 398
    using seq with ksh 354
  • shellappending to the settings of existing environment variables, like PATH 245
    command line editing 397
    expansion of wildcard characters 216 225 256
    how to determine which shell I'm using 294
    prompt and PS1, favorite settings for your shell prompt 349
    sh script to reformatting number strings with commas 329
    shell aliases -- reader favorites 146
    syntax for conditional tests, "[" versus "[[" versus "test" 348
  • sort-nnumeric sort 405
    Unix sort examples 281 316 337 389
  • sort methods compared 141
  • sortingsorting when the key field can be pattern matched, but isn't normally delimited 375
    sorting words on one line 152
  • squeezing text 116
  • ssh-c arcfourfaster, but less secure, transfers 414
    known_hosts file 402
    piping input/output to/from 414
    searching a list of machines 402
    using a ssh configuration file to create machine aliases 353
  • statdetailed inode listins 356
  • static memory allocationdetermining 137
  • stderrpiping both stderr and stdout with csh and ksh 335
  • sticky bit 109
  • stringsUnix command 168
  • stty erase <BACKSPACE> 110
  • T
  • table of squaresusing recurrence to compute quickly 286
  • tabsreplacing with spaces 203
  • taradding selected files only 377
    archiving specific files only 331
    extracting specific files only 323
    moving everything accidentally untarred in the wrong directory somewhere else 357
    piping input/output to/from 414
    relative untar (remove leading "/'s") 184
    transferring directories between systems using tar with stdin and stdout 343
    using tar to preserve file ownership and group settings during file transfer 364
    why is tar file 3x expected size from "du -sk" 267
  • tcshcommand completion in tcsh 353
    command line editing 397
    disabling "rm *" with "set rmstar=1" 355
  • teepiping stderr and stdout to a file and displaying on the screen 335
  • terminalcolorized output 415
  • testshell syntax for conditional tests, "/usr/bin/[" versus "/usr/bin/test" versus "[[" 348
  • tips for lifefrom readers 185 306
    warming your car when it's cold 334
  • tips for studentsmilk crates 369
    paying bills 369
  • tmp and scratch purgers 177
  • totalviewexpanding fortran automatic arrays 243
    viewing C arrays 145
  • touch-rcopying file access and mod time 231
  • trtranslate command, example 140 152 316 337 356 375
  • U
  • Unixfinding path of an executable 183
    path to parent script 401
  • Unix commandsLinux commands for beginners 408
    giving arguments that start with "-" and could be construed as options 226
  • Unix vacation programautomatic e-mail reply 143
  • ualsextracting fields from ls output 356
    finding files newer than a specified age 377
  • ulimit-tset a hard limit on CPU time 411
  • ulpWhat's an ulp? Or, Unit of Least Precision 254
  • unameprint system info (hostname nodename release version hardware) 204
  • uniqUnix uniq command, examples 316 362
  • unique directory namegenerating automatically in scripts 269
  • V
  • version controlmaintaining software in collaborations 278
  • vi"!" and ":" commands, how to use registers to edit, save, and execute again 173 379
    "yank" from file 1 and "put" into file 2 252
    "|" jump to column 115
    :%s,/lib ,/usr/lib ,using characters other than "/" to delimit search/replace 266
    :%s/^M//gremoving CTRL-M characters 139
    :.,'m s/^/Ccommenting-out blocks of Fortran code 104
    :10,30s/.\{25\}/&|/inserting vertical lines 142
    :frenaming file buffer 193
    :rusing file insertion to prepend a file to current file 302
    ex commands 104 302 379
    finding cursor column 115
    generating and loading a list of files from grep 103
    noautoindentfixing staggering paste 122
    regular expresion to switch order of element within a line 274
  • vim"&" repeats last ":s" command 379
    autocmd, remember cursor position and other state between sessions 336
    making color schemes 328
    register ":" contains last ":" (or ex) command. "@:" executes it again 379
    replace 330
    setting highlight colors 328
  • vimdiffcheck for differences in remote file 413
  • W
  • WHERE construct in Fortran 90 205
  • web browsersobtaining URL of a frame in Netscape, IE, Lynx 227
  • wgetcommand-line retrieval of web pages 311
  • what commandprobing executable 108 168
  • X
  • xargs-lusing with find and grep 283
    examples 283 301 304 322
    preserving files which match a wildcard expression, deleting the rest 301
  • xlf-qreport -qlistloop, optimization, and other listings 309
    -qsuffixIBM Fortran compiler, recognizing .f90 files as Fortran 90 246 303
    -qsuffixRunning the preprocessor on files with arbitrary suffix 303
    getting XLF version numbers 318
  • Z
  • zshcommand completion in zsh 353
    command line editing 397
    disabling "rm *" 355
    loops in zsh 354
    prompt 424

 

Newsletter Topic Index

 

Back to Top