Tickit version 0.31

(mostly a copy of the mail to the tickit-dev mailing list)

A lot of stuff happening lately. And also I haven't written one of these for ages. I won't go into every detail, but here's a rundown of the most interesting parts:

• Rect/RectSet are now C library based (0.26)

  Fairly simple, no surprises here. C implementation means it's available in C and other languages, and probably a bit faster in actual use.

• New Term event binding API (0.26)

  Rather than a single on_key/on_mouse/etc..., there is now just a list of possible event handlers. Event handling subs don't have to be restricted to a single event; each is registered with a bitmask. This is done to more closely match the C API.

  The existing Perl API of having a single on_* handler for each event type is still supported, by wrapping the newer API.

• All windows are now FLOAT windows (0.28)

  As was first suggested in 0.23, all the windows now use the new float logic. The previous environment variable has now been removed. This hopefully shouldn't actually affect anything as it's been the default for a while now, but does simplify the code internals.

• $win->close and no more weak references (0.28)

  Using weak references and relying on DESTROY works OK in some circumstances in Perl, but won't scale to C and other languages, and still makes for tricky logic. To this end, I've removed all the weaken()ing and replaced it with an explicit ->close method to remove a window. This also makes it much more robust in nontrivial cases.

  This change is mostly of interest to container widget developers, or in more dynamic long-lived programs.

• Tickit::Style (0.29)

  This one's the big main one of the list; in fact so bit I'll probably write another mail. In summary; we now have something of a first attempt at being able to separate out style from widget implementations, in a way that's easy to add to application- or user-specific style files. More on this later.

• Tickit::Pen now comes in mutable and immutable forms (0.30)

  Since most pens don't get mutated, and Tickit::Style performs better with cached pens, I've split the idea of a Pen into mutable and immutable types. Tickit::Style returns immutable pens, so widgets shouldn't attempt to mutate them.

  Tickit::Pen->new itself still returns a mutable pen for now, but in the future this may change; code that specifically wants a mutable or immutable pen should use the appropriate subclass.

• Tickit::Pen changes to support upcoming Tickit::RenderContext (0.31)

  A few small changes that allow the new Tickit::RenderContext to work better.

Also some changes in the underlying libtickit C library:

• Generic string/integer value termctl operations

  Primarily provided to let the xterm driver set the window title, etc..., but the general idea is something similar to ioctl(), so we don't have to extend the API a thousand times just to add lots of little options for specific terminals.

• Split xterm/TI-based driver model

  To support more specific options in future, and also to give a better (or more accurate) terminfo-based driver. There are now two drivers, selected by the $TERM environment variable, so the xterm-specific things can be done nicely, and still arrange for the generic terminfo driver to work.

  This also allows for other terminal-specific drivers in future, in case we find those useful. Perhaps a Win32 console one too.

Net::Async::HTTP::Server

Recently, I've been working on a way to serve HTTP with IO::Async. For a while it's been possible to use HTTP as a client with Net::Async::HTTP, so it seemed only natural to provide a server named Net::Async::HTTP::Server.

I've tried to design the interface fairly similar to some of the other server modules, most notably Net::Async::FastCGI. This allows for similar code to be written one way or the other, for responding via HTTP or FastCGI.

use Net::Async::HTTP::Server;
use IO::Async::Loop;
 
use HTTP::Response;
 
my $loop = IO::Async::Loop->new();
 
my $httpserver = Net::Async::HTTP::Server->new(
   on_request => sub {
      my $self = shift;
      my ( $req ) = @_;
 
      my $response = HTTP::Response->new( 200 );
      $response->add_content( "Hello, world!\n" );
      $response->content_type( "text/plain" );
 
      $req->respond( $response );
   },
);
 
$loop->add( $httpserver );
 
$httpserver->listen(
   addr => { family => "inet6", socktype => "stream", port => 8080 },
   on_listen_error => sub { die "Cannot listen - $_[-1]\n" },
);
 
$loop->run;

As with Net::Async::FastCGI the distribution also includes modules to support hosting a PSGI application via Plack. The Plack::Handler module in particular allows the use of Net::Async::HTTP::Server directly from the plackup commandline:

plackup -s Net::Async::HTTP::Server --listen ":8080" application.psgi

It's still in somewhat of an early state - while it supports sending streaming responses (as required by PSGI) it doesn't yet support streaming receiving of requests. That shouldn't be too hard to add, but so far I haven't found a need to add it yet.

Alt modifier - high-bit or escape?

I've wanted to write this up for quite a while. Haven't quite got around to it mostly for not knowing how to start. So instead of agonising over an introduction, I'll just paste IRC logs instead.

#vim on Freenode, on a subject of whether terminals should encode the Alt modifier key by high-bit mask, or Escape prefix:

You -always- want alt-sends-escape. Anything about high-bits just breaks UTF-8 and is always broken.

alt==highbit => it is impossible to distinguish UTF-8 input from Alt-modified input. Cannot be done. All input is broken. Fail. Go home
alt==esc => it is possible to distinguish Escape prefix from Alt-modified input by using timing information. Not perfect, sometimes false hits, but generally works.

False positives happen if you type too quickly on a possibly-laggy connection, and multiple 1-byte writes coaless into a single 2-byte read buffer. False negatives ought not happen if your terminal is working nicely, but could be possible in buggy terminals (hello ConnectBot + Hackers' Keyboard), or if e.g. network MTU issues start cutting your buffers in weeeeird places.

I recommend 20msec.

20 is good because the human eye/brain will fudge over intervals shorter than 20 msec.. if two things happen within 20msec, or something that you expect to happen, happens within 20msec after, you'll perceive it as instant.

So a 20msec escape timeout means your brain doesn't notice that 20msec delay after you press the Escape key, before it hits back to Normal mode again. But 20msec is looooads of time for vim to wait for the trailing letter that might just be coming after the ESC byte if it was in fact an Alt- modified key.

Incidentally, this is the style used by my libvterm terminal emulator library, and libtermkey terminal key event input library.

Project Updates - Tickit window scrolling, Circle scroll indicators

Some progress on a few of my projects:

Tickit::Window now supports scroll and scrollrect even if the window partly covered by other floating windows. With the expose-after-scroll behaviour turned on (soon to be default in a later version once I know it works nicely), it always returns true, and tries to scroll as many regions of the screen as it can, queueing expose operations for the areas that needed it.

A good example of this working can be seen in the newly-updated Tickit::Widget::Scroller, which now supports a scroll position indicator, a small floating window in the top or bottom right corner.

Another example can be seen as the latest feature in Circle::FE::Term. Scrolling a channel window gives an indicator in the bottom right corner, which also includes a count of new lines of content added since scrolling, if there are any.

The Past, The Present and The Future - LPW2012

For those who are interested, here are the slides of the talk I gave about Futures at LPW2012 this year:

The Past, The Present and The Future

Pangoterm - config and scrollback

I've been working more on pangoterm, and it now has two important features that bring it much closer to being a real usable terminal.

The first is that it now supports a lot more configuration by command-line, and also the same things by a configuration file. Support for a config file isn't something I personally have directly needed so far, mostly because the compiled-in defaults are already what I'd like. But it's something other people have been asking for, and this helps support other users too, so it's nice to have.

I also found quite a neat way to implement it too, requiring just one line of macro to declare the existence of a configuration setting and give its default value and help description. This then creates a normal variable that can be read in the usual way:

CONF_INT(scrollback_size, 0, 1000, "Scrollback size", "LINES");
...
pt->scroll_size = CONF_scrollback_size;

The other main feature that's now in place is scrollback buffer. Now, whenever text disappears off the top of the screen it is saved, and can be recalled again by scrolling the mouse wheel or using the Shift-PageUp and Shift-PageDown keys. A small custom-drawn indicator shows the scroll position without needing to consume any screen space in the normal state of not being scrolled.

I also found and fixed an almost embarrassingly-bad bug around the redrawing code. When erasing cells, it used to make one cairo call for every cell to be erased, leading to many calls whenever an entire line had to be cleared. Now it composes multiple blank cells with the same pen, in the same way as it composes multiple cells of text; and is faster when scrolling partial lines, and so on.

While I've already been using it quite a lot lately to run the vim I use to edit its own source code, for some tasks I've still been using xterm. But not any more - with the addition of scrollback to pangoterm, I find I'm now using it for everything. It's become my default terminal both for home and work use.

libvterm / pangoterm performance improvements

I recently made a couple of small but drastic improvements to libvterm and pangoterm's performance, when scrolling long output. Using my not-so-scientific test of taking a ~500KB text file and running:

$ time cat file

This technically measures the amount of time it takes cat to write the data into the TTY, but since the buffer is a fixed 4KiB in size, it also measures the time that pangoterm takes to read all but the final 4KiB of the file, which is under 1% of the file. I managed to obtain the following timings. Before I started:

real    0m4.206s
user    0m0.000s
sys     0m0.036s

By optimising libvterm's moverect buffer operation with memmove() (revision -r511):

real    0m2.035s
user    0m0.000s
sys     0m0.048s

With pangoterm deferred updates, that delay re-rendering of the screen until all the PTY data has been read, or every 20 msec (revision -r482):

real    0m0.358s
user    0m0.004s
sys     0m0.016s

It's now well over 10 times faster than it used to be. Ohh.. and it beats xterm. By quite a bit.

real    0m2.294s
user    0m0.000s
sys     0m0.036s