ARSC HPC Users' Newsletter 419, March 29, 2011

Advanced vim part II

[by Tom Baring]

Part one of this series appeared in issue #370 , September 2007, offering these VIM tips:

  • Cursor movement perfection
  • Combine cursor movement with change/delete/pipe operations
  • Use regular expressions
  • Let VIM handle repetitive tasks

Here’s one more skill for improving your VIM efficiency:

  • Use VIM’s built-in line editing (command-line mode) operations

For many of us, vi and VIM are a huge improvement over editors which make you grab the mouse. In VIM, you move the cursor with efficient keystrokes, but the need for any cursor movement at all can be reduced by using the command line.

Rather than moving the cursor and performing some action, you can often simply type a command to produce the same result, saving yourself keystrokes and improving your odds of doing exactly what you want without mistakes.

My hope with this article is to get you started with this new approach to editing (which is actually an old approach, if you were ever "privileged" to use TECO).

Background

When you’re in VIM "normal-mode" (AKA "command-mode") and hit the colon key, you enter the under-used "command-line mode" and can enter "ex" commands. Every VIM user already uses these:


  :q[uit]
  :w[rite]
  :s[ubstitute]

And probably these:


  :r[ead]
  :f[ile]
  :g[lobal]
  :!

But how about these?


  :ya[nk]
  :d[elete]
  :pu[t]
  :co[py]
  :m[ove]
  :fo[ld]
  :foldc[lose]
  :foldo[pen]
  :i[nsert]
  :a[ppend]
  :ma[rk]
  :j[oin]

The functionality of these operations is available in normal-mode, or can be constructed, so why bother with line editing? For many editing problems, they result in:

  1. Less typing
  2. More control
  3. More fun

Examples

Example 1 : copying a line to the current cursor location.

Your cursor is at line 457. Near the top of the screen you see a line (400) that you want to copy to your current location.

Normal-mode solution:

  1. Move cursor to line 400: Hjjjjjj or 400G or 30kkkkkk
  2. Yank the line: yy
  3. Move cursor back to 457: Lkkkkkkk or 457G or 33jjjjj
  4. Put the line: p

Command-line solution (Drum-roll first, please)


  :400t.

Example 2 : moving a block of text.

Your cursor is on line 111. You decide to move lines 134-143 to line 120.

Normal-mode solution:

  1. Move the cursor to line 134: 134G or jjjjjjjjjjjjjjjjjjjjjjj
  2. Delete from the cursor to line 143: d143G or d10d (hoping you didn’t miscount)
  3. Move cursor to line 120: 120G or kkkkkkkkkkkkkk
  4. Put the deleted block: p (hoping you didn’t accidentally delete anything else along the way, overwriting your saved text)

Command-line solution:


  :134,143m120

The two big advantages to command-line solutions:

  1. You don’t have to move the cursor.
  2. You can examine the command before committing it! In normal-mode, much of the command is invisible as you enter it: the movement commands, the delete buffer, etc. In command-line mode, it’s right in front of you.

How to do line editing in VIM

The first thing to learn is that all command line operations work on either one line or a range of lines. There are several ways to specify a line, and they’re all useful in different situations.

A LINE specification is

  1. one of the predefined, special lines: "." is the current cursor line "$" is the last line in the file (you can also specify an offset relative to "." or "$" as a line specification. E.g., "$-3" or ".+12" or ".-12".)
  2. an explicit line number, e.g., "120". (Enter ":se[t] nu[mber]" to show line numbers.)
  3. a mark. (In normal-mode, hit m followed by a letter to set a mark at the cursor position.

    E.g, "ma" sets mark "a." On the command line use ":ma[rk]". E.g., ":120ma a" places mark "a" on line 120--without moving the cursor.)

  4. the line on which a search operation finds a match. (E.g., :/pattern/)

A RANGE specification is

  1. a comma delimited line specification. E.g.:

    :120,130 (lines 120-130 inclusive) :.,$ (current cursor line to end of file) :’a,’b (mark "a" to mark "b") :/Apple/,/Zebra/ (Next occurance of "Apple" to next occurance of "Zebra")

  2. the predefined, special range:

    "%" is the range 1,$ (i.e., the entire file)

ARGUMENTS to command-line commands

The commands I’m recommending take either a line address, a register name, a mark name or nothing as their arguments. What are these things?

  • Line specification: One line specified as described above.
  • Register name: When you yank or delete a range, the contents of that range are stored in a buffer, called a register. In addition to the unnamed (default) register you have 26 named registers, a-z. I suggest always yanking/deleting into a named register so you’re less likely to overwrite it. In MacVIM, buffers are shared between all tabs in a window, so you can easily yank a couple things from one file (one tab) and put them in another.
  • Mark name: You can set up to 52 unique marks: a-z and A-Z.

Command-Line Commands

Syntax notes:

  • Items in square brackets are optional.
    • The default for [range] is the current cursor line.
    • The commands have the given abbreviations, e.g., "ya" works for "yank"
    • The default for [register name] is, well, the default register
  • Items in curly braces are required.

COMMANDS: Explanations and Examples

Yank :[range]ya[nk] [register name]

In normal-mode, whenever you delete anything, with "x" or "d", the thing deleted overwrites the unnamed register. Many times, I’ve yanked a block of text, but before putting it in its new location, I’ve managed to overwrite it by accident. Thus, I’ve learned to always yank to a named buffer. E.g.,:


      :.,122ya k

Best of all, it’s not necessary to move the cursor first. Just specify both ends of the range, wherever they are:


      :12,13 ya x 
      :$-4,$ ya y 
      :’a,’b ya z

Delete :[range]d[elete] {address}

With delete, it’s even more important to use a named register because (obviously) you’ve deleted the thing, making it harder to recover if you overwrite the default register.


       :98,789d k

Deleting the current line:


       :d k    
       :.d k 

Put :[line]pu[t] [register name]

You can put the contents of a register at the current cursor position:


       :pu k  
       :.pu k

Or anywhere:


       :333pu k 
       :’a pu k 

You can put the default register at the current location:


       :pu

Or at the next occurance of "Zebra"


       :/Zebra/pu

Copy :[range]co[py] {address} or :[range]t {address}

Copy a single line to another location:


      :$-5co81

Note, ":t" is an alias for copy, and saves one character of typing:


      :11,22t33

You can copy a range of lines inside itself, if needed:


      :11,33t22

Move :[range]m[ove] {address}

Example:


      :11,22m33

Fold :[range]fo[ld]

"Folding" hides a range of lines. Say you want to see and edit a file’s header and footer simultaneously, hiding everything in between:


      :20,$-20 fold

How about hiding everything between the opening and closing BODY tag in an HTML file:


      :/<body>/,/<.body>/ fold

Foldopen :[range]foldo[pen]

Fold definitions persist, even after you "open" them to see what’s inside. You can open all folds in a range with foldopen:


      :%foldo
      :100,200 foldo

Foldclose :[range]foldc[lose]

Similarly, close all folds in a range:


      :%foldc

(Note, that there are other commands for erasing fold definitions.)

Mark :[line]ma[rk] {register}

Set mark "a" at the current cursor position:


      :ma a

A nice thing about "mark" is that it doesn’t move the cursor:


      :111ma a
      :/apple/ma a
      :/Apple/ma A
      :/zebra/ma z
      :/Zebra/ma Z

Join :[range]j[oin]

Rather than moving the cursor and hitting JJJJJJJJJJJJJJ or guessing "15J", use command-line joing:


      :234,256j

Global :[range]g[lobal]/pattern/ command

I don’t get fancy with global, maps, or search and replace, strongly preferring to record and replay complicated combinations (see issue 370). That said, "global," has its place. "Global" selects every line in a range which matches a pattern, and then executes an ex command on those lines. Here are some examples:

Change "T3E" to "HPC" but only on lines, from the current cursor line to the end of the file, containing the text, "Users’ Newsletter."


      :.,$g/Users\’ Newsletter/s/T3E/HPC/g

Copy every line containing "Users’ Newsletter" to the end of the file.


      :g/Users\’ Newsletter/t$

Delete all empty lines:


      :g/^$/d

Shift matching lines, with "global" with "mark" and "copy," using multiple ex commands. This shifts every line containing BOBBY up four lines:


      :g/BOBBY/mark k 
 move ’k-4

Help :h[elp] [command or topic]

Help on "mark." Command-line and then normal-mode versions:


      :h :ma
      :h m

Help on ranges, :foldopen, and :global:


      :h cmdline-ranges
      :h :foldopen
      :h :global

Help main page:


      :h

Other Commands...

I’ve listed most of the commands I find useful, but there are many more:


    :h ex-cmd-index

Miscellany

Executing multiple commands

The "|" serves as a separator between ex commands, so you can type several at once if you know what you want. E.g.:


      :33,40t$ 
 47,50t$ 
 33,50d

      :set noignorecase 
 %s/Apple/apple/gc 
 %s/Zebra/zebra/gc 
 set ignorecase

Repeating a command-line ex command

As described in issue 370, VIM stores the last command in a special register aptly named ":", and you can execute the contents of any register with "@". Thus, you can easily re-execute your last command line ex command:

In normal-mode, repeat the previous ex command:


      @:

In command-mode, repeat the previous ex command:


      :@:

Stop escaping forward slashes!!!

The pattern delimiter in "global" and "substitution" is NOT restricted to the forward slash ("/"). I always use a comma (",").

So, instead of this mess (to change /usr/local/bin to /usr/bin):


      :g/\/usr\/local\/bin/s/\/local//

I type this:


      :g,/usr/local/bin,s,/local,,

The delimiter can be any non-alphanumeric character except ’\’, ’"’ or ’|’, so use whatever you like, e.g.:


      :g!/usr/local/bin!s!/local!!
      :g#/usr/local/bin#s#/local##

If your "personal" delimiter character appears in the pattern, escape it:


      :s,The time is come\, the walrus said,Let’s,

Next Steps...

The way I suggest you master VIM is to print a detailed reference card, like this, and keep it handy:

http://tnerual.eriogerg.free.fr/vim.html

Most of the command-line commands I describe in this article aren’t on the card, so write them in the margins.

Then, whenever you’re ready for a stretch, look at the card, pick any command or feature you find intriguing, learn it and use it till it’s second nature. Then pick another... and another... ad infinitum...

Document Freedom Day

For anyone who has ever had trouble reading files in some closed-source format, there is a Document Freedom Day.

The fourth annual one is March 30, 2011. Check out http://documentfreedom.org/2011/ for details.

Still more git

[by Kate Hedstrom]

If you were foolish enough to be learning git only from my rantings, you would not yet have heard about tags and git bisection. It is time to remedy that situation.

Tags

Tags are a way to provide a more user-friendly name for a particular revision. You can always refer to revision numbers by their SHA1 number, but you might want to have a better way to label the code used for a particular simulation. To tag the currently checked out code:


% git tag version_2.3

You can also tag a particular revision:


git tag stable-1 f3a7081fb9

You would then be able to see your tags like:


% git tag
stable-1
version_2.3

You can later access code using:


% git checkout version_2.3

You can add a message to a tag with the "-a" argument:


% git tag -a testing-1 -m ’my first tag string’

and then see that string later with:


% git show testing-1

One thing about tags is that they only apply to your current repository. Pushing changes, then downloading them on another system will make the code corresponding to the tagged version visible elsewhere, but not the tag itself. The same code base on one machine gives:


mg57 116% git tag -l
NEP5_45
testing-1

and this on another:


332% git tag
nep5_run38

I’ve clearly missed tagging several runs between 38 and 45!

Bisection

Say you know you had a working version of the code a couple months ago. Then you bring in two months worth of updates from somewhere, including conflict resolution. You find that at the end of this, something doesn’t work. What to do? Assuming that there are a number of discrete revisions, you can ask git to help you narrow down which change it was that created difficulties for you. Here’s how:


% git bisect start
% git bisect good nep5_run38
% git bisect bad

This is saying the current version is bad, while that at tag nep5_run38 is good. If you don’t have tags, the SHA1 numbers work as well. Say things look like:


            HEAD
             

           a123
             

           b3452
             

           73f8g
             

          nep5_run38

Git will check out the b3452 code and ask you to check it. Once it’s been checked, tell it either "git bisect good" or "git bisect bad". It will then check out a123 or 73f8g, accordingly. You will then need to check that one and report on it, until you are down to just one step between a "good" and a "bad" case.

When I was forced to try out the bisection, I ran into one pitfall. Not all codes compiled off the bat so I made some local changes. Because git wants to check out new codes after a "git bisect good/bad", it won’t do it when there are local modifications. Make sure you do your test, then "git reset --hard", then "git bisect good/bad". Make sure "git status" is clean before your report!

Once you get your answer about which change to focus on, clean up the bisect business with "git bisect reset".

Some material shown here comes from the Git Community Book: http://book.git-scm.com/

If git bugs you, know that you’re not the only one. You might check out http://reprog.wordpress.com/2010/05/10/git-is-a-harrier-jump-jet-and-not-in-a-good-way/ and the two follow-on articles for some interesting tips about the good, the bad and the ugly in git.

Quick-Tip Q & A


A.[[ After I added an "echo" statement to the .bashrc of a certain machine,
  [[ I was no longer able to scp to it.  When I tried to scp to that
  [[ machine, I would see the echo and nothing else, and the prompt returned
  [[ without any indication of failure.  Why did an echo statement in my
  [[ .bashrc prevent scp from working?

# # We received this tip from reader Samy Bara: #
Old known issue, see https://bugzilla.redhat.com/show_bug.cgi?id=20527 .
# # Elaborating just a bit, especially for readers who don’t use web browsers: #
The scp command reads from ~/.bashrc. It is a known issue that if the ~/.bashrc generates output, it breaks scp. The referenced RedHat ticket was closed with status "WONTFIX" and a note that this bad behavior is the fault of OpenSSH.

Q: I’m setting up a new ocean simulation and we have an idea for how to handle land-fast ice. It requires reading in the land-fast ice from some observations. My problem is that the model grid is in lat/lon and the model reads netCDF files while the observations are in various GIS formats. Available formats are GeoTiff, ESRI shapefiles, and Microsoft Access Database. Any advice on dealing with these?

[[ Answers, Questions, and Tips Graciously Accepted ]]


Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top