Detail oriented

Ed Fine's blog.

How VMware's Ovftool Password Handling Gave Me a Headache

ovftool is a command-line utility from VMware that lets one do useful things with VMs on ESXi and vSphere remote systems.

I had installed ovftool and was trying to use it copy a VM between two ESXi servers, based on this useful post from virtuallyGhetto. For various reasons, it’s often a better idea to use ovftool for copying VMs than by just using scp on the raw files.

Immediately, I ran into a WTF? moment.

$ ovftool vi://root@
Enter login information for source vi://
Username: root
Password: *******
Error: Could not lookup host: root

Error: Could not lookup host: root???

This confused the living daylights out of me. This has nothing at all to do with looking up a host.



And the answer is… drum roll…

Locators. At least, the URI-flavored ones. A locator points to different resource types like VMs or hosts.

When ovftool gets a URI, it’s more or less of the form protocol://username:password@host:port/ or protocol://username@host:port/

(protocol can be one of the standard schemes like https or file, or VMware-specific ones like vi or vcloud.)

If ovftool gets a URI without the password - which I would imagine most security conscious people would prefer - it quite sensibly prompts for a password and captures it without displaying it.

At this point, it appears that ovftool forms the full URI - including password - and uses that to authenticate with the remote system.

You can see where this is going.

The ovftool PDF manual clearly notes (but not clearly enough, in my view):

Encoding Special Characters in URL Locators

When you use URIs as locators, you must escape special characters using %
followed by their ASCII hex value.  For instance, if you use a “@” in your
password, it must be escaped with %40 as in vi://foo:b%40r@hostname, and a
slash in a Windows domain name (\) can be specified as %5c.

Now I get it.

  1. ovftool captures the password from stdin and does not urlencode it.
  2. ovftool forms the URI with the unencoded password, and does not check it for validity.
  3. ovftool uses the URI to contact the remote system.
  4. The malformed URI is interpreted such that the user name - root in this case - is considered to be the remote system’s host name.
  5. The connection attempt dies with the oh-so-misleading message, Error: Could not lookup host: root.
  6. I get a headache.

This hypothesis can be proven as follows.

  • urlencode the failing password;
  • Feed the encoded password to ovftool at the Password: prompt;
  • Profit!!

What I think ovftool does wrong

  1. ovftool violates The principle of Least Astonishment. When a tool accepts a password for input, it is expected that the tool does any necessary transformations to it prior to using it. ovftool must urlencode the password if it obtains it via a password prompt.
  2. ovftool fails to check that the URI is well-formed. The characters that must be urlencoded in the various parts of a URI are well-known, and it should be fairly easy to test this.
  3. A more minor, yet valid quibble, is that ovftool echoes asterisks once the password has been entered, which is ok, except that it echoes the same number of asterisks as the number of characters in the password. Exact password length can give a useful clue to a would-be attacker (who would need to be looking over your shoulder, but still, it’s so easy to avoid this mistake).

A simple workaround

A simple workaround is to urlencode the password yourself on the command line. If you have access to a clipboard-copy-paste utility, the entire thing can be done without displaying the password.

Let’s say that your password is my@random pass/+?$#%^&*()-_+={}[]\|;:'”,/?.

This code snippet will prompt for the unencoded password, urlencode it, and put it into the clipboard, ready for pasting. The advantage of using this over the various command-line urlencoding utilities - that might or might not be available - is that Python is available just about everywhere these days. If not, there’s always Perl.

python -c 'import urllib; import getpass; print(urllib.quote_plus(getpass.getpass()))' | $CLIPUTIL

Clipboard utilities

Possible values of $CLIPUTIL include

  • Windows 10: clip (or redirect to /dev/clipboard in older versions)
  • macOS: pbcopy
  • Linux (X): xclip -selection c

Git Submodule Cheat Sheet

git submodule is powerful, error-prone, and often confusing unless it’s used pretty much daily.

Having a cheat sheet can be pretty useful, so here you go.

This cheat sheet is based on a post by Christophe Porteneuve (many thanks).

Erlang's Internal Data Representation

I find it interesting how the Erlang BEAM engine represents data in memory.

Quick look: An Erlang list

This is the in-memory layout of the Erlang list "phi".

[112, 104, 105]

How BofA Froze My Account for Supporting My Children

My open Dear Jane letter to Bank of America

Dear Bank of America,

Your MSB department believes I am a wicked money launderer or something, so you froze my personal account. If I hadn’t been paying attention, it could have been very nasty indeed, but I drained my accounts and left you only $27.08 in the one you subsequently froze. Don’t spend it all at once.

The Myth of the Fungible Resource

There is a pervasive myth amongst software development management types: the existence of the fungible resource.

A fungible resource is a human cog that managers and executives believe they can drop into a business machine to replace an existing cog. In other words, when developers part ways with a company, many managers believe that their replacements can quickly get up to speed and take over where their predecessors left off.

Pale Blue Dot

Let’s put things in perspective: we’re not all that in the grand scheme of things.