Perl - Sentinel - version 0.02

I have just uploaded the first (non-development) release of Sentinel; version 0.02.

It provides a little helper function that creates lvalues, suitable for lvalue typed accessors that want to run real code on assignment (such as for type checking or coercion, or update triggers), rather than just update a scalar.

use Sentinel;
 
sub foo :lvalue
{
   my $self = shift;
   sentinel get => sub { return $self->get_foo },
            set => sub { $self->set_foo( $_[0] ) };
}

It makes the following two lines equivalent

my $obj = Some::Object->new;

$obj->set_foo( 100 );
$obj->foo = 100;

This is rare among my modules, more or less the first thing I've written mostly as a result of ranting about it on #perl, rather than because I actually wanted it. My argument kept being that if anyone does want an lvalue accessor, it's trivially easy to write one given some function sentinel as in this example, and that actually implementing the sentinel function itself isn't hard; it's a small piece of obvious XS magic.

So here it is.

It has to cheat somewhat on versions of Perl before 5.14, because of the way lvalue context isn't properly as powerful as it now is. Long story short - there may be unit-test failures on some older versions of Perl; but I can't tell yet because the CPAN Testers web frontend is still down, so I can't see the smokers. If anyone sees any version-related failures, please do let me know.

---

Edit 2011/11/29: By the power of crowdsourced smoke testing, it appears this does work stably across many Perl versions - thanks all who've informed me.

Perl - Term::TermKey - version 0.09

Last week saw a new version of Term::TermKey (0.09), and also the underlying libtermkey (0.9). This contains a fairly small new feature, giving control of the way that EINTR is handled.

Version 0.8 added graceful handling of EINTR to restart IO operations rather than fail with an error. This had unfortunate knock-on effects for the Perl-level wrapping of it, because of the deferred nature of Perl's safe signals. On a signal (such as the not-so-unlikely SIGWINCH) a flag would be set, but the termkey_waitkey(3) operation would be restarted, not returning back to Perl's control until a keypress event was actually received. This upset programs that wish to respond to SIGWINCH and redraw the terminal to the new size.

Version 0.9 of libtermkey now therefore has a new flag, TERMKEY_FLAG_EINTR whose presence makes the blocking IO operations (termkey_waitkey(3) and termkey_advisereadable(3)) to return a new result code, TERMKEY_RES_ERROR after which the caller can inspect the value of errno, to observe an EINTR.

Again because of Perl's safe signal handling, the Perl wrapping of libtermkey has to always enable this flag, so it can invoke the $SIG{WINCH} signal handler, for example. The Term::TermKey module therefore now always sets TERMKEY_FLAG_EINTR on the underlying TermKey instance, and emulates the presence or absence of this flag at the Perl level, by optionally restarting its IO operation, or itself returning TERMKEY_RES_ERROR.

An unfortunate bug here in the emulation and hiding of this flag from the Perl level means that Term::TermKey 0.09 fails to correctly read the TermKey flags back out of the underlying object. In particular it fails to be able to check on the presence of TERMKEY_FLAG_UTF8 that libtermkey itself may have enabled, after detecting a UTF-8 locale. This causes Tickit's unit tests to break with the familiar "Wide character in syswrite at ..." error.

This bug has now been fixed in the source code repository, and will be present in the next version, 0.10. Tickit 0.10 also has an independent fix for the same bug, by using Perl's ${^UTF8LOCALE} instead of reading the TermKey flags back out again.

Also upcoming in libtermkey 0.10, will be some Solaris portability fixes, and a new canonicalisation flag, which turns a TERMKEY_SYM_DEL key into TERMKEY_SYM_BACKSPACE, for those terminals that send DEL on Backspace.

IO::Async and AnyEvent

I've recently been working on a new IO::Async loop implementation, IO::Async::Loop::AnyEvent. Like the Glib and POE loops, this one uses another event system as its underlying implementation; AnyEvent.

What makes this one a little different and noteworthy, is that AnyEvent claims not to be an event system as such, but rather a compatibility layer on top of event systems. I have so far resisted writing this particular loop implementation on the grounds that, since AnyEvent just applies a cross-compatibility layer on top of some other existing event system anyway, IO::Async might as well use that underlying event system directly. In practice however this doesn't quite seem to work out all the time; existing code exists and is already written. Sometimes that existing code already works using AnyEvent directly, making it harder to drop a small section of IO::Async-based code inside it.

In that situation, an individual module/function/etc.. can construct itself an IO::Async::Loop::AnyEvent, to allow that component to use IO::Async-based functionallity, while still interacting correctly with the rest of the AnyEvent-based program.

It is not primarily intended that this module be used as the basis of an entire program; mostly because a neater solution for mixed IO::Async+AnyEvent exists in the form of AnyEvent::Impl::IOAsync.

There are still some minor issues to deal with currently; most notably that because each of IO::Async and AnyEvent could use the other as an event source, there are some circularity problems when AnyEvent picks AnyEvent::Impl::IOAsync to use, when IO::Async::Loop::AnyEvent loads it.

Deep recursion on subroutine "AnyEvent::Impl::IOAsync::io" at /home/leo/src/perl/IO-Async-Loop-AnyEvent/blib/lib/IO/Async/Loop/AnyEvent.pm line 103.

Hmmmmm.... A little work there still needed :)

Perl - Term::Terminfo - version 0.06

I've just uploaded a new version of Term::Terminfo. In brief; this is a small wrapper around the terminfo database, and can be used to enquire about properties of a given terminal. For annoying historic reasons, each of these properties is known by at least two names; its short "capname", a two or three letter code often found in the actual file on disk, and a longer "varname", the name of a variable in the curses C library, which stores its value for the current terminal.

This latest version, 0.06, adds a whole duplicate set of methods - varname accessors. Prior to 0.06, the properties were only accessible using their short capnames.

It's probably best to use the longer varnames anyway. They are more self-documenting, and a little more future-proof against the admittedly-remote possibility of new variables being added in the future.

I'm also planning to support unibilium in a later version. This is a standalone terminfo-parsing library, which is useful for reading terminfo data without linking against the full curses library. This is especially useful when trying to build a replacement for curses...

XS beats Pure Perl

Someone reported some test failures trying to install Tickit, which seemed to be related to shortcomings in Text::CharWidth. The latter seems to have very poor unit test coverage on itself, so the failures didn't appear during its installation, only when Tickit::Utils was tested against it. On initial inspection I wondered if Text::CharWidth simply wasn't using wcswidth(3) correctly, and whether I should get around to my plan of rewriting bits of Tickit::Utils in XS instead for performance, as well as work around this bug.

This turned out to be quite a good idea. Implementing cols2chars() and chars2cols() in XS instead of Perl makes them at least 10 times faster. I tested it on four strings; two ASCII and two Unicode; a long and a short of each:

Calls/sec		PP	XS	Ratio
chars2cols	along	48685	406504	834.97%
chars2cols	ashort	72674	704225	969.02%
chars2cols	ulong	37341	387596	1037.99%
chars2cols	ushort	52966	714285	1348.57%
cols2chars	along	16350	403255	2466.39%
cols2chars	ashort	58685	649350	1106.50%
cols2chars	ulong	13561	362318	2671.76%
cols2chars	ushort	50556	632911	1251.90%

In fact, some cases it turns out to be 24 times faster.

I haven't looked into too much detail on why, but I suspect a large amount of the reason is to do with the way the XS functions primarily walk along the internal UTF-8 representation of the strings, counting bytes, characters, and columns as they go, and returning the appropriate count(s) when the required. The pureperl implementation doesn't have direct access to the byte offsets, so only has character numbers to work to. The frequent character-to-byte or byte-to-character conversions at all the boundaries between the functions result in multiple UTF-8 byte skip counting steps along the string each time a function is entered or left, generally slowing it down.

As to the original test failure, it turned out to be entirely unrelated lack of locale support in the platform's libc. The XS implementations fail there in the same way. But having implemented the above improvements, I decided to leave them in anyway.

XS faster than Pure Perl; who'd have thought it?

Perl - IO::Async - version 0.41

Wow; seems I haven't been writing these posts as much as I should be. I last wrote about version 0.34, and just now I've uploaded version 0.41 to CPAN. Quite a bit changed between then and now, here's a rundown of the most important bits:

• Added IO::Async::FileStream. This behaves like a read-only IO::Async::Stream, but reads its data from a regular file on the filesystem. Like tail -f, or File::Tail, it watches the file for changes in size, and follows appended data.


• Added IO::Async::Function. This is a true IO::Async::Notifier subclass to represent an asynchronous block of code; which was previously handled by DetachedCode. Being a true Notifier subclass gives it many advantages, and should provide a better base to build other things on. IO::Async::Resolver is now a subclass of Function, not DetachedCode.


• Added IO::Async::Process. This is another IO::Async::Notifier subclass, this time to represent an external process.


• Support encoding parameter in IO::Async::Stream. This allows the Stream to handle text in some encoding, rather than simply raw bytes.


• Support first_interval parameter to IO::Async::Timer::Periodic. If supplied, this will be the first wait time when the timer is started. In particular, if it is zero the timer's first invocation will happen immediately.


• Allow Loop->listen to be extended via extensions, similar to ->connect. Such extensions as IO::Async::SSL are already set up to make use of this.


• Distribution now uses Test::Fatal rather than Test::Exception. The former is smaller and simpler, whereas the latter relies on clever tricks to hack on caller(), which sometimes breaks some setups.


• Added convenient method in IO::Async::Loop for handling socket addresses; extract_addrinfo. This recognises strings for common networking constants; 'inet', 'inet6' and 'unix' as socket families, 'stream', 'dgram' and 'raw' as socket types. These convenient forms are also recognised by the ->connect and ->listen methods.


• Now prefers to use the IPv6 support functions found in Perl 5.14's Socket module; only falling back to using Socket::GetAddrInfo in earlier perls.

Sleep Sorting with IO::Async and CPS

There's a bit of a silly meme going around lately; an implementation of a sorting algorithm that works by many parallel sleeps. So I thought I'd have a go at it from an IO::Async perspective.

use CPS qw( kpareach );
use IO::Async::Loop;

my $loop = IO::Async::Loop->new;

kpareach( \@ARGV,
   sub {
      my ( $val, $k ) = @_;
      $loop->enqueue_timer(
         delay => $val,
         code  => sub { print "$val\n"; goto $k },
      );
   },
   sub { $loop->loop_stop },
);

$loop->loop_forever;

This produces such output as:

$ perl sleepsort.pl 3 8 4 2 6 5 7 1
1
2
3
4
5
6
7
8

It's a little more verbose than I'd like it though. I have been pondering creating a CPS::Async module, which would provide some more simple CPS-like versions of the usual blocking primatives we're used to, like sleep and sysread. Using this new hypothetical module would lead to something looking like:

use CPS qw( kpareach );
use CPS::Async qw( ksleep );

kpareach( \@ARGV,
   sub {
      my ( $val, $k ) = @_;
      ksleep( $val, sub {
         print "$val\n";
         goto $k;
      } );
   },
   sub { CPS::Async::stop },
);

CPS::Async::run;

This would be a small wrapper around IO::Async, providing CPS-style functions that turn into method calls on some implied IO::Async::Loop object; similar to for example, the AnyEvent::Impl::IOAsync, which exposes an AnyEvent API using IO::Async.

I have plenty of other projects to keep me amused currently, but I'll sit it on the back burner in case something interesting happens that way...