Event loops and Jenga; or 24 Advent Calendar Events in One Go

There are many event loops systems in Perl. Do they play together?

I was thinking about this recently, at my LPW2010 talk about IO::Async. In the hackathon the following day, I managed to write IO::Async::Loop::POE; a way to run IO::Async atop POE.

So I started thinking further; if you can run one event loop system on top of another, how high can we stack them? Can we build a tower, putting each atop the previous, growing taller. Each new layer we try to add would start to get harder, more difficult, increasing the chances the whole thing came crashing down. Sortof like a Jenga tower.

So what would a Perl event loop Jenga tower look like?

My attempt looks like this: (326 lines, jenga.pl)

The output looks something like this:

$ perl jenga.pl
AnyEvent resolved 127.0.0.1:80
Glib reads Hello world!
POE reads Hello world!
POE resolved 127.0.0.1:80
IO::Async reads Hello world!
AnyEvent reads Hello world!
AnyEvent listener accepted
POE listener accepted
IO::Async resolved 127.0.0.1:80
AnyEvent connected received
POE connected received
IO::Async listener accepted
IO::Async connected received
Glib child exited 0
POE child exited 0
IO::Async child exited 0
AnyEvent child exited 0
Glib timer
POE timer
IO::Async timer
AnyEvent timer
^CIO::Async SIGINT
AnyEvent SIGINT
POE SIGINT
Stopping...

That's 24 events. Count them. It combines Glib, POE, IO::Async and AnyEvent. It performs a basic filehandle read, a child process watch, and a timed wait in each of these four systems. Because Glib lacks signal watching, only the other three perform this. The other three are also used to perform name resolution, socket listening, and socket connecting.

Everyone seems to be doing Advent Calendar blogs this year. 24 daily posts, each showing one small thing. Someone suggested I should write a Perl Event systems advent calendar. So perhaps here, consider this to be one. Except it has 24 windows all in one go.

As it turns out, it's possible to make this tower a little higher. There's a module to run Event beneath Glib; that is, it replaces the core polling function of Glib to use Event instead. And I suspect it may just about be possible to run Tk on Glib, and the POE on Tk.

At some point in the new year, I have some plans to turn this one-program script into a more useful resource of examples and translations. The Rosetta Stone for Unix provides a cross-reference for looking up Unix concepts between different systems. I feel that a similar attempt at Perl event loops could be quite useful too.

Perl - CPS - version 0.11

CPS is a Perl module that provides several utilities for writing programs in Continuation Passing Style.

In a nutshell, CPS is a method of control flow within a program, which uses values passed around as continuations as a replacement of the usual call/return semantics. In Perl, this is done by passing CODE references; calling a function and passing in a CODE reference to be invoked with the result, rather than having the function return it. While at first this doesn't seem to be very useful, most of its power comes from the observation that the function doesn't need to invoke its continuation immediately; if it performs some IO operation or similar, it can perform this in an asynchronous fashion, invoking its continuation later. This style of coding is often associated with nonblocking or asynchronous event-driven programming. It is typical of such event systems as IO::Async.

A typical problem with implementing CPS control flow, is that all of the usual Perl control-flow mechanisms are built for immediate call/return semantics, where the use of CPS gets in the way. The CPS module provides utility functions for implementing control flow in a continuation passing style, by providing a set of functions that replace the usual Perl control-flow keywords. For example the Perl control structure of a foreach loop

foreach my $frob ( @frobs ) {
   my $wibble = mangle( $frob );
   say "Mangled $frob looks like $wibble";
}
say "All done";

becomes a call to CPS::kforeach

use CPS qw( kforeach );

kforeach( \@frobs, sub {
   my ( $frob, $knext ) = @_;
   kmangle( $frob, sub {
      my ( $wibble ) = @_;
      say "Mangled $frob looks like $wibble";
      $knext->();
   } );
}, sub {
   say "All done";
} );

We haven't really gained anything by doing this though. If the process of mangling a frob involves some IO tasks, perhaps talking to some remote server, then we'll spend most of our time waiting for it to respond when we could be sending multiple requests and waiting on their responses. We could likely save some time by running them concurrently.

I gave a talk at LPW2010 about Continuation Passing Style and CPS, the slides of which are available here.

After discussing CPS and IO::Async at LPW, I was talking with mst about his IPC::Command::Multiplex module. He came up with the idea for another control-flow function; a combination of kpar and kforeach, which I called kpareach. In use it looks exactly like kforeach, except that it starts the loop body for each item all at the same time, in parallel, rather than waiting for each one to finish before invoking the next.

This is a new addition to CPS version 0.11, which is now on CPAN. It is also one of the first new control-flow structures that doesn't have a direct plain-Perl counterpart; a demonstration of the usefulness of CPS in event-driven programming.

General Updates

I have no big specific updates today. Instead, a list of lots of little things I've been working on:

• IO::Socket::IP now has preliminary non-blocking connect support (version 0.05_003). This isn't quite a perfect solution because of blocking name resolvers, but see also Net::LibAsyncNS below.


• Created a new CPAN dist, wrapping libasyncns, called Net::LibAsyncNS. This allows a simple way to asynchronise name resolver lookups.


• Have fixed a few bugs in IO::KQueue, relating to dodgy handling of Perl scalars in the udata field. Some memory leak bugs still exist, but I believe these to be the kernel's fault. See below.


• Spent some time on Freebsd-hackers@ arguing about kqueue and managing user data pointers. Long story short I believe kqueue API itself is missing a feature, making generic wrapping of it impossible from any high-level language, or properly by C libraries also.


• Both talks I submitted for LPW2010 were accepted; on the subjects of CPS and IO::Async.


• Net::Async::HTTP now has SSL support and can stream response body content as it arrives, rather than waiting for the whole response (version 0.08).

That's all for a quick update, but I may write about any or all of these topics in more detail later...