Alt modifier - high-bit or escape?

I've wanted to write this up for quite a while. Haven't quite got around to it mostly for not knowing how to start. So instead of agonising over an introduction, I'll just paste IRC logs instead.

#vim on Freenode, on a subject of whether terminals should encode the Alt modifier key by high-bit mask, or Escape prefix:

You -always- want alt-sends-escape. Anything about high-bits just breaks UTF-8 and is always broken.

alt==highbit => it is impossible to distinguish UTF-8 input from Alt-modified input. Cannot be done. All input is broken. Fail. Go home
alt==esc => it is possible to distinguish Escape prefix from Alt-modified input by using timing information. Not perfect, sometimes false hits, but generally works.

False positives happen if you type too quickly on a possibly-laggy connection, and multiple 1-byte writes coaless into a single 2-byte read buffer. False negatives ought not happen if your terminal is working nicely, but could be possible in buggy terminals (hello ConnectBot + Hackers' Keyboard), or if e.g. network MTU issues start cutting your buffers in weeeeird places.

I recommend 20msec.

20 is good because the human eye/brain will fudge over intervals shorter than 20 msec.. if two things happen within 20msec, or something that you expect to happen, happens within 20msec after, you'll perceive it as instant.

So a 20msec escape timeout means your brain doesn't notice that 20msec delay after you press the Escape key, before it hits back to Normal mode again. But 20msec is looooads of time for vim to wait for the trailing letter that might just be coming after the ESC byte if it was in fact an Alt- modified key.

Incidentally, this is the style used by my libvterm terminal emulator library, and libtermkey terminal key event input library.

Extended colour support, terminals, and ECMA-48

tl;dr summary: Terminal authors - please accept CSI 38:5:123 m as well as anything else, to set extended colours. NOTE THE COLON. Many terminals these days are starting to support extended colour modes; specifically things like 256 colours. They're all doing it wrong.

CSI 38;5;123 m

No no no no no. That is NOT what you think. It does not select colour 123 from palette 5. CSI arguments are separated by semicolons. This sets three unrelated SGRs; 38, 5, and 123. 38 sets a foreground colour to .. er.. nothing in particular. 5 is blinking mode, 123 has no defined meaning to my knowledge. All the other following are exactly identical:

CSI 5;38;123 m
CSI 38 m CSI 5 m CSI 123 m

ECMA-48 already defines a perfectly good way for parameters to take sub-parameters. It's the colon. The correct way to encode this concept is

CSI 38:5:123 m

Why does this matter? It matters for parsers not to have to understand what is going on. Consider

CSI 3;38:9:5:4:3:2:1;11;1

This is equivalent to

CSI 3 CSI 38:9:5:4:3:2:1 CSI 11; CSI 1

I.e. I have no idea what palette 9 is, but I can definitely parse out the contents of this SGR from the others, knowing exactly where it ends. I don't have to "just know" that palette 9 happens to take 5 parameters. The CSI encodes this. My own terminal emulator library, libvterm, already understands these colon-based arguments. It parses them correctly. This is important for other palettes that don't take just one value. For example, the RGB, RGBA, CMY, and CMYK palettes. It's vital to be able to parse out these sub-arguments from the single SGR 38 or 48. Standards exist for a reason, people. Please use them.

---

Edit 2012/12/20: actually, I have recently learned that xterm now supports this in its correct form as well, since xterm patch 282.