I've wanted to write this up for quite a while. Haven't quite got around to it mostly for not knowing how to start. So instead of agonising over an introduction, I'll just paste IRC logs instead.
#vim on Freenode, on a subject of whether terminals should encode the Alt modifier key by high-bit mask, or Escape prefix:
You -always- want alt-sends-escape. Anything about high-bits just breaks UTF-8 and is always broken.
alt==highbit => it is impossible to distinguish UTF-8 input from Alt-modified input. Cannot be done. All input is broken. Fail. Go home
alt==esc => it is possible to distinguish Escape prefix from Alt-modified input by using timing information. Not perfect, sometimes false hits, but
generally works.
False positives happen if you type too quickly on a possibly-laggy connection, and multiple 1-byte writes coaless into a single 2-byte read buffer. False negatives ought not happen if your terminal is working nicely, but could be possible in buggy terminals (hello ConnectBot + Hackers' Keyboard), or if e.g. network MTU issues start cutting your buffers in weeeeird places.
I recommend 20msec.
20 is good because the human eye/brain will fudge over intervals shorter than 20 msec.. if two things happen within 20msec, or something that you expect to happen, happens within 20msec after, you'll perceive it as instant.
So a 20msec escape timeout means your brain doesn't notice that 20msec delay after you press the Escape key, before it hits back to Normal mode again. But 20msec is looooads of time for vim to wait for the trailing letter that might just be coming after the ESC byte if it was in fact an Alt- modified key.
Incidentally, this is the style used by my libvterm terminal emulator library, and libtermkey terminal key event input library.
