2020 Perl Advent Calendar - Day 9

<< First | < Prev | Next >

Yesterday we saw some ways to write concurrent asynchronous code which waits on a few different tasks to complete. Sometimes we want to do the same thing multiple times concurrently but with different data each time. Often it's the case that each item can be processed independently of the others, so it makes sense to try to do several at once.

One approach here is to simply await a future to process each individual item inside a regular foreach loop. This will only work on one item at once, so we won't make use of concurrency.

## A poor idea for iterating a list ##
use Future::AsyncAwait;

foreach my $item (@ITEMS) {
    await PROCESS($item);
}

Another idea is to use map to apply an asynchronous function to every item in the list, and thus start all the items at once, then wait on all those futures using needs_all. This may end up being too concurrent - if the list contained many thousand items we might do too many at once and overload whatever external service or system we are talking to.

## Another poor idea ##
use Future::AsyncAwait;

await Future->needs_all(
    map { PROCCESS($_) } @ITEMS
);

A better middle-ground between these two extremes was introduced in the original blog series on day 18, in the form of the Future::Utils::fmap collection of helper functions. The basic idea of fmap is that it is a future-aware equivalent of Perl's map operator. The fmap function is given a block of code which is expected to return a future, and a list of items. It invokes the code block once for each item in the list, collecting up and waiting on the returned future values, until all the items are done. fmap returns a future to represent the entire operation, which will complete with the results of each individual item.

The fmap family actually contains three individual functions, which all operate in the same basic manner. The difference between them is in how they handle return values from each individual item block - fmap_concat can handle an entire list from each item and concatenates all the results together for its overall result, fmap_scalar expects exactly one result per item, and fmap_void does not collect up any results at all; running the code block simply for its side-effects.

Since fmap expects a future-returning function and itself returns a future it is also idea for use with async/await syntax. It can be invoked in an await expression and passed an async sub to operate on.

This somewhat-paraphrased example uses the GET method of a Net::Async::HTTP user agent object to concurrently fetch and JSON-decode a collection of data from multiple API endpoints of some remote service.

use feature 'signatures';
use Future::AsyncAwait;
use Future::Utils qw( fmap_scalar );
use JSON::MaybeUTF8 qw( decode_json_utf8 );

use Net::Async::HTTP;
my $ua = Net::Async::HTTP->new;
...

my @urls = ...

my @data = await fmap_scalar(async sub ($url) {
    my $response = await $ua->GET($url);
    return decode_json_utf8($response->content);
}, foreach => \@urls, concurrent => 16);

A few things should be noted about this example. First is that the async sub syntax is used explicitly to create an asynchronous function to pass as the first argument to the fmap_scalar function. Second is the use of the concurrent parameter, telling the function how many items to keep running concurrently. Finally, the list of items has to be passed in an array reference rather than a plain flat list.

These facts all come about because the fmap functions are just plain Perl functions and not special syntax, as opposed to the real syntax provided by the async and await keywords. Whereas these two keywords were inspired by a whole collection of other languages which have all adopted it as a standard pattern, there is not much existing design on the problem of bounded concurrency map-style syntax. This particular area remains a matter of ongoing design and discussion. Thoughts welcome ;)

<< First | < Prev | Next >

2020 Perl Advent Calendar - Day 8

<< First | < Prev | Next >

We've now had a good look at a number of situations involving asynchronous code which does one thing at a time. Back on day 3 we noted that an important use-case for asynchronous code is the ability to do multiple actions concurrently and wait for results from all of them at once. We saw a way to achieve that, by starting multiple operations at once by calling multiple asyncronous functions, then using await on each returned future in turn.

This is a sufficiently important and frequent pattern when dealing with asynchronous code and futures, that in the original advent series, day 13 introduced a constructor method, Future->needs_all, which helps this. It takes a list of futures and yields a new future, which will complete only when all of its components have completed successfully; or alternatively, will fail when any one of them has failed. Since this constructor yields a future, when using async/await syntax we can simply await on it in order to suspend until all of the components are ready.

For example, this method from a hardware chip driver needs to write to two distinct control registers in order to change the chip's configuration. It can do this most efficiently by issuing write commands to both of them individually, then waiting for them both to complete, by using the ->needs_all constructor.

use feature 'signatures';
use Future::AsyncAwait;

async sub change_config(%changes)
{
    ...
    await Future->needs_all(
        $self->write_register(REG_CTRL1, $ctrl1),
        $self->write_register(REG_CTRL3, $ctrl3),
    );
}

We can also use this structure to obtain values. When a ->needs_all future completes, it will yield a list of results by concatenating the result lists from each of its individual components. The chip driver makes use of this when reading back the configuration, by issuing two read commands for each of the control registers, and awaiting the result of both together.

use Future::AsyncAwait;

async sub read_config
{
    my ($ctrl1, $ctrl3) = await Future->needs_all(
        $self->read_register(REG_CTRL1),
        $self->read_register(REG_CTRL3),
    );
    ...
}

If a failure occurs in any of the component futures, needs_all will re-throw that failure. In effect, it acts as if we had in fact performed an await expression once on every one of the individual futures. It acts as if we had written

## A less well-written form of the above example ##
use Future::AsyncAwait;

async sub read_config
{
    my $f1 = $self->read_register(REG_CTRL1);
    my $f3 = $self->read_register(REG_CTRL3);
    
    my $ctrl1 = await $f1;
    my $ctrl3 = await $f3;
}

When waiting for more than one future like this it is preferrable to use a structure like needs_all rather than individual await expressions. Having multiple await expressions means that the containing async sub has to be resumed and suspended again each time one of them makes progress, before it finishes them all. Having a single await on the combined future only has to suspend and resume once. Results and errors are still handled just as they would be as if multiple awaits were used.

If there are more than just a few concurrent tasks to perform, there can be even better ways to express this. We will take a look at another approach tomorrow.

<< First | < Prev | Next >

Awaiting The Future

Introduction

Various articles I have previously written have described Futures and their use, such as the Futures Advent Calendar. In this article, I want to present a new syntax module that greatly improves the expressive power and neatness of writing Future-based code. This module is Future::AsyncAwait.

The new syntax provided by this module is based on two keywords, async and await that between them provide a powerful new ability to write code that uses Future objects. The await keyword causes the containing function to pause while it waits for completion of a future, and the async keyword decorates a function definition to allow this to happen. These keywords encapsulate the idea of suspending some running code that is waiting on a future to complete, and resuming it again at some later time once a result is ready.

use Future::AsyncAwait;

async sub get_price {
    my ($product) = @_;

    my $catalog = await get_catalog();

    return $catalog->{$product}->{price};
}

This already reads a little neater than how this might look with a ->then chain:

sub get_price {
    my ($product) = @_;

    return get_catalog()->then(sub {
        my ($catalog) = @_;

        return Future->done($catalog->{$product}->{price});
    });
}

This new syntax makes a much greater impact when we consider code structures like foreach loops:

use Future::AsyncAwait;

async sub send_message {
    my ($message) = @_;

    foreach my $chunk ($message->chunks) {
        await send_chunk($chunk);
    }
}

Previously we'd have had to use Future::Utils::repeat to create the loop:

use Future::Utils qw( repeat );

sub send_message {
    my ($message) = @_;

    repeat {
        my ($chunk) = @_;
        send_chunk($chunk);
    } foreach => [ $message->chunks ];
}

Because the entire function is suspended and resumed again later on, the values of lexical variables are preserved for use later on:

use Future::AsyncAwait;

async sub echo {
    my $message = await receive_message();
    await delay(0.2);
    send_message($message);
}

If instead we were to do this using ->then chaining, we'd find that we either have to hoist a variable out to the main body of the function to store $message, or use a further level of nesting and indentation to make the lexical visible to later code:

sub echo {
    my $message;
    receive_message()->then(sub {
        ($message) = @_;
        delay(0.2);
    })->then(sub {
        send_message($message);
    });
}

# or

sub echo {
    receive_message()->then(sub {
        my ($message) = @_;
        delay(0.2)->then(sub {
            send_message($message);
        });
    });
}

These final examples are each equivalent to the version using async and await above, yet are both much longer, and more full of the lower-level "machinery" of solving the problem, which obscures the logical flow of what the code is trying to achieve.

Comparison With Other Languages

This syntax isn't unique to Perl - a number of other languages have introduced very similar features.

ES6, aka JavaScript:

async function asyncCall() {
  console.log('calling');
  var result = await resolveAfter2Seconds();
  console.log(result);
}

Python 3:

async def main():
    print('hello')
    await asyncio.sleep(1)
    print('world')

C#:

public async Task<int> GetDotNetCountAsync()
{
    var html = await
        _httpClient.GetStringAsync("https://dotnetfoundation.org");

    return Regex.Matches(html, @"\.NET").Count;
}

Dart:

main() async {
  var context = querySelector("canvas").context2D;
  var running = true;    // Set false to stop game.

  while (running) {
    var time = await window.animationFrame;
    context.clearRect(0, 0, 500, 500);
    context.fillRect(time % 450, 20, 50, 50);
  }
}

In fact, much like the recognisable shapes of things like if blocks and while loops, it is starting to look like the async/await syntax is turning into a standard language feature across many languages.

Current State

At the time of writing, this module stands at version 0.22, and has been the result of an intense round of bug-fixing and improvement over the Christmas and New Year break. While it isn't fully production-tested and ready for all uses yet, I have been starting to experiment with using it in a number of less production-critical code paths (such as unit or integration testing, or less widely used CPAN modules) in order to help shake out any further bugs that may arise, and generally evaluate how stable it is becoming.

This version already handles a lot of even non-trivial cases, such as in conjunction with the try/catch syntax provided by Syntax::Keyword::Try:

use Future::AsyncAwait;
use Syntax::Keyword::Try;

async sub copy_data
{
    my ($source, $destination) = @_;

    my @rows = await $source->get_data;

    my $successful = 0;
    my $failed     = 0;

    foreach my $row (@rows) {
        try {
            await $destination->put_row($row);
            $successful++;
        } catch {
            $log->warnf("Unable to handle row ID %s: %s",
                $row->{id}, $@);
            $failed++;
        }
    }

    $log->infof("Copied %d rows successfully, with %d failures",
        $successful, $failed);
}

Known Bugs

As already mentioned, the module is not yet fully production-ready as it is known to have a few issues, and likely there may be more lurking around as yet unknown. As an outline of the current state of stability, and to suggest the size and criticality of the currently-known issues, here are a few of the main ones:

Complex expressions in foreach lose values

(RT 128619)

I haven't been able to isolate a minimal test case yet for this one, but in essence the bug is that given some code which performs

foreach my $value ( (1) x ($len - 1), (0) ) {
    await ...
}

the final 0 value gets lost. The loop executes for $len - 1 times with $value set to 1, but misses the final 0 case.

The current workaround for this issue is to calculate the full set of values for the loop to iterate on into an array variable, and then foreach over the array:

my @values = ( (1) x ($len - 1), (0) );
foreach my $value ( @values ) {
    await ...
}

While an easy workaround, the presence of this bug is nonetheless a little worrying, because it demonstrates the possibility for a silent failure. The code doesn't cause an error message or a crash, it simply produces the wrong result without any warning or other indication that anything went wrong. It is, at time of writing, the only bug of this kind known. Every other bug produces an error message, most likely a crash, either at compile or runtime.

Fails on threaded perl 5.20 and earlier

(RT 124351)

The module works on non-threaded builds of perl from version 5.16 onwards, but only on threaded builds 5.22 onwards. Threaded builds of 5.20 or earlier all fail with a wide variety of runtime errors, and are currently marked as not supported. I could look into this if there was sufficient interest, but right now I don't feel it is a good use of time to support these older perl versions, as compared fixing other issues and making other improvements elsewhere.

Devel::Cover can't see into async subs

(RT 128309)

This one is likely to need fixing within Devel::Cover itself rather than Future::AsyncAwait, as it probably comes from the optree scanning logic there getting confused by the custom LEAVEASYNC ops created by this module. By comparison, Devel::NYTprof can see them perfectly fine, so this suggests the issue shouldn't be too hard to fix.

Next Directions

There are a few missing features or other details that should be addressed at some point soon.

Core perl integration

Currently, the module operates entirely as a third-party CPAN module, without specific support from the Perl core. While the perl5-porters ("p5p") are aware of and generally encourage this work to continue, there is no specific integration at the code level to directly assist. There are two particular details that I would like to see:

• Better core support for parsing and building the optree fragment relating to the signature part of a sub definition. Currently, async sub definitions cannot make use of function signatures, because the parser is not sufficiently fine-grained to allow it. An interface in core Perl to better support this would allow async subs to take signatures, as regular non-async ones can.

  A mailing list thread has touched on the issue, but so far no detailed plans have emerged.

• An eventual plan to migrate parts of the suspend and resume logic out of this module and into core. Or at least, some way to try to make it more future-proof. Currently the implementation is very version-dependent and has to inspect and operate on lots of various inner parts of the Perl interpreter. If core Perl could offer a way to suspend and resume a running CV, it would make Future::AsyncAwait a lot simpler and more stable across versions, and would also pave the way for other CPAN modules to provide other syntax or semantics based around this concept, such as coroutines or generators.

local and await

Currently, the suspend logic will get upset about any local variable modifications that are in scope at the time it has to suspend the function; for instance

async sub x {
    my $self = shift;
    local $self->{debug} = 1;
    await $self->do_work();
    # is $self->{debug} restored to 1 here?
}

This is more than just a limit of the implementation, however as it extends to fundamental questions about what the semantic meaning of such code should be. It is hard to draw parallels from any of the other language the async/await syntax was inspired by, because none of these have a construct similar to Perl's local.

Recommendations For Use

Earlier, I stated that Future::AsyncAwait is not fully production-ready yet, on account of a few remaining bugs combined with its general lack of production testing at volume. While it probably shouldn't be used in any business-critical areas at the moment, it can certainly help in many other areas.

Unit tests and developer-side scripts, or things that run less often and are generally supervised when they are, should be good candidates for early adoption. If these do break it won't be critical to business operation, and should be relatively simple to revert to an older version that doesn't use Future::AsyncAwait while a bugfix is found.

The main benefit of beginning adoption is that the syntax provided by this module greatly improves the readability of the surrounding code, to the point that it can itself help reveal other bugs that were underlying in the logic. On this subject, Tom Molesworth writes that:

Simple, readable code is going to be a benefit that may outweigh the potential risks of using newer, less-well-tested modules such as this one.

This advice is similar to my own personal uses of the module, which are currently limited to a small selection of my CPAN modules that relate to various exotic pieces of hardware. Many of the driver modules related to Device::Chip have begun to use it. A list of modules that use Future::AsyncAwait is maintained by metacpan.

I am finding that the overall neatness and expressiveness of using async/await expressions is easy justification against the potential for issues in these areas. As bugs are fixed and the module is found to be increasingly stable and reliable, the boundary can be further pushed back and the module introduced to more places.

---

This article is adapted from one that was originally written in two parts for the Binary.com internal tech blog - part 1, part 2.

I would also like to thank The Perl Foundation whose grant has enabled me to continue working on this piece of Perl infrastructure.

Async/await in Perl - control flow for asynchrony

I've decided that my Future::AsyncAwait CPAN module is sufficiently non-alpha that I've started migrating a few of my less critical code into using it. I thought I'd pick a few of the Device::Chip drivers for various kinds of chip, because they're unlikely to be particularly involved in anyone's real deployment code, as really I only wrote those to test out some ideas on the chips before writing microcontroller code in C for them. These seemed like good candidates to begin with.

Here's an example of a function in the previous version, using Futures directly. The code had lots of syntactical noise, some ->then chaining and the Future::Utils::repeat loop not looking like a regular foreach loop. You can just-about read what's going on but it's clear there's a lot of machinery noise getting in the way of really understanding the code.

By rewriting all the logic using await expressions inside an async sub we arrive at a version that much closer resembles the sort of thing you'd write in straight-line synchronous code. In reading it you can just skim over the awaits while looking at it and read it like synchronous code.

A question you might begin to ask at this point is why I'd choose to implement this particular set of syntax or semantics, of the various possibilities for how to manage asynchronous control flow. Aside from its general neatness and applicability to Futures (which I've already worked with at length), there's one key reason: The async/await syntax here is the exact same thing as implemented in Python 3, ES6, C#5, Dart, even Rust is currently considering adopting it Yes, it's nice to have a good concurrency model built into the language, but it's considerably stronger if it's the same as the consensus among a variety of other languages too.

Some language references for them:

Python	Tasks and coroutines
JavaScript	async function
C#	Asynchronous Programming
Dart	Dart Language Asynchrony Support

If the four quite semantically-different languages of Python, JavaScript, C# and Dart can all come to the same idea, then maybe it has merit. I honestly think that given a few years, async/await could become as ubiquitous as if or while loops, to the level of "well obviously our language has that". This is why I wanted to steal it into Perl. In ten years time it might look as foolish for a language not to have an async/await construct, as it does today for it not to have a try/catch or a switch.

Ah... more on that subject another day perhaps ;)

Kinds of Invocation in Event-Reflexive Programming

<< First | < Prev

In the previous post I introduced the idea of Event-Reflexive programming, and discussed the first use-case I had for it; driving the user provisioning system at an ISP. I said this story would continue in chronological order.

A couple of years into this job, I felt I had learned Perl enough to do what surely pretty-much any Perl developer does at this time. Yes, I decided to write an IRC bot. It's one of those rites of passage that every developer goes through at some point. Of course, even this early in my programming career I had already seen several dozen terrible attempts at this, so I was quite determined to ensure mine wouldn't suffer quite as many of those mistakes. In my head, of course, I knew I wouldn't suffer many mistakes because I, of course, was armed with Event-Reflexive Programming.

I ended up with, I thought, the most amazing (it wasn't), the most powerful (it wasn't) and the most flexible (it wasn't) IRC bot the world had ever seen (it isn't). However, in the process of building it I had expanded on the original concept of event-reflexivity considerably.

In the previous post I introduced the most basic two forms of invoking the plugins in an event-reflexive system, run_plugins and run_plugins_reverse. In the course of developing this IRC bot, I found it necessary to create a number of other variations on this basic theme.

Recalling the original two functions, both of these simply execute action hooks defined by the plugins. Neither of them returns an interesting result. What I found while implementing the IRC bot was that as well as merely requesting that work be performed, I was also using the event-reflexive core to abstract out a number of query-like operations - such as abstracting away the specific mechanism of database used to store information about registered users. At this point, the event-reflexive core needs a way to pose a question to the list of plugins, and return an answer as soon as one has been provided:

sub ask_plugins {
  my ( $query, @args ) = @_

  foreach my $plugin ( @plugins ) {
    next unless $plugin->can( $query );
    my $ret = $plugin->$query( @args );
    return $ret if defined $answer;
  }

  return undef;
}

Perl being Perl, it's only a short matter of time before we want a list-valued return from some of these queries. And once we're returning a list, we're not restricted to returning the result from a single plugin - we can run them all:

sub ask_plugins_list {
  my ( $query, @args ) = @_;

  my @ret;
  foreach my $plugin ( @plugins ) {
    next unless $plugin->can( $query );
    push @ret, $plugin->$query( @args );
  }

  return @ret;
}

The final and most interesting invocation function was called scatter_plugins. This being written years before I had encountered the concept of Futures, it was initially written with a complex combination of additional code reference arguments, before I managed to neaten it up somewhat with the creation of Async::MergePoint. I won't give the implementation here, but the point of this particular call was to account for the fact that some plugin actions are going to be asynchronous, and only return a result later. What we'd like to do is start all the operations concurrently, then await their eventual completion before continuing.

These days, I would instead implement this operation using Future. In fact, at this point a case could be made for implementing all of them using Future. If the entire event-reflexive core was based on futures, then trivially it will cope with any synchronous or asynchronous kind of work environment (due to the universal suitability of futures). If we currently set aside our previous question of plugin ordering, and assert that ordering doesn't matter, then all the remaining operations besides reverse can be expressed on top of a single idea:

sub _call_all_plugins {
  my ( $method, @args ) = @_;

  return map {
    my $plugin = $_;
    $plugin->$method( @args );
  } grep { $_->can( $method ) } @plugins;
}

sub run_plugins_concurrently {
  Future->needs_all( _call_all_plugins( @_ ) )
        ->then_done( "" ); # return an empty result
}

sub ask_plugins_concurrently {
  Future->needs_any( _call_all_plugins( @_ ) )
}

sub ask_all_plugins {
  Future->needs_all( _call_all_plugins( @_ ) )
}

In fact at this point our previous idea of scatter_plugins becomes totally redundant - the universal expressiveness of Futures has allowed this to be expressed even simpler. But this incredibly simple implementation has come at a cost - we've lost the sequential lazy-evaluation nature of ask_plugins. Additionally, whatever stop-on-error semantics we might have wanted out of run_plugins have been lost.

Perhaps instead we decide we need ordering, at least in some cases. This brings to mind some additional invocation functions, that themselves are also wrappers around a single common idea:

use Future::Utils qw( repeat );

sub _call_each_plugin {
  my ( $reverse, $while, $method, @args ) = @_;

  repeat {
    my ( $plugin ) = @_;
    $plugin->$method( @args );
  } foreach => [ grep { $_->can( $method ) }
                 $reverse ? reverse(@plugins) : @plugins ],
    while => $while;
}

sub run_plugins_sequentially {
  _call_each_plugin( 0, sub { 1 }, @_ );
}

sub run_plugins_sequentially_reverse {
  _call_each_plugin( 1, sub { 1 }, @_ );
}

sub ask_plugins_sequentially {
  _call_each_plugin( 0, sub { not shift->get }, @_ );
}

Keeping in mind my first question from the previous post, on the subject of ordering between plugins, this motivates a choice of second questions:

If ordering guarantees are not required, are the concurrent invocation functions given above sufficient to express any order-less possibly-asynchronous operation in an event-reflexive system?
If ordering is required, are the additional sequential invocation functions sufficient to express any ordered possibly-asynchronous operation?

Next >

Futures advent day 24

Day 24 - Futures compared to Callbacks

It would seem at first glance that futures provide similar benefits to managing control flow by callbacks. However, they provide several advantages in comparison.

When performing a sequence of many operations using callbacks, the ever-increasing nesting nature of the callback functions leads to an ugly indenting pyramid look in the source code.

FIRST_CB( $arg1, sub {
  SECOND_CB( $arg2, sub {
    THIRD_CB( $arg3, sub {
      FINISHED()
    });
  });
});

Because futures are connected together using the return value of a function, not through a value passed into it, they can avoid this mess and remain at a fixed indentation level. This also allows, for example, a new stage to be added between existing stages without upsetting the indentation of the following code; making neater diff output in revision control systems, and giving less chance of a merge conflict when branching.

FIRST_F( $arg1 )->then(sub {
  SECOND_F( $arg2 )
})->then(sub {
  THIRD_F( $arg3 )
})->then(sub {
  FINISHED()
});

Moreover, many other shapes of control flow start to look much more like their synchronous counterparts, precisely because they are linked together using the return values out of the individual units and require no other values to be passed in.

Possibly the most simple example of concurrent control flow is a two-way merge case, where two operations are started concurrently waiting for the result of both before continuing. Using callbacks this would need to be solved by each callback storing its result in a variable they both lexically capture, and checking in each whether both results have been provided.

my $one_result; my $two_result;

ONE_CB( sub {
  $one_result = shift;
  if( defined $two_result ) {
    FINISHED($one_result, $two_result);
  }
});
TWO_CB( sub {
  $two_result = shift;
  if( defined $one_result ) {
    FINISHED($one_result, $two_result);
  }
});

Immediately two issues come to light here. First is the repeated FINISHED code - if that were itself a further chain of operations with callbacks, this would be impossible (or at least very tedious) to repeat twice, and of course gets much worse beyond two concurrent branches. Secondly, we are testing the results for definedness - maybe undef is a perfectly valid result from each function. In that case we'd have to track two further variables to simply remark whether each operation has completed:

my $one_done; my $one_result;
my $two_done; my $two_result;

ONE_CB( sub {
  $one_result = shift;
  $one_done++;
  if( $two_done ) { ... }
});
...

This example of course only handles the success case. Imagine how much more complex the code would be if each function took two code references, one for success and one for failure, and additionally returned some kind of operation ID that would be used to cancel the operation in progress if it was no longer required. This would now need eight lexically captured variables, adding much more boilerplate control-flow noise to the code. Moreover, now there are more variables being shared among code blocks, it creates the possibility that strong reference cycles remain long after the operation has finished, failed, or been cancelled that retain an object in memory long after it was required. It may end up looking something like (and keep in mind this is the most simple case of two concurrent operations and a single "afterwards"):

my $one_done; my $one_result; my $one_failed; my $one_id;
my $two_done; my $two_result; my $two_failed; my $two_id;

my $finished = sub {
  undef $one_id; undef $two_id;
  FINISHED();
};

$one_id = ONE_CB(
  sub { $one_result = shift; $one_done++;
        $finished->() if $two_done; },
  sub { $one_failed++;
        TWO_CANCEL($two_id) if !$two_done; undef $two_id;
        FAILED() },
);
$two_id = TWO_CB(
  sub { $two_result = shift; $two_done++;
        $finished->() if $one_done; },
  sub { $two_failed++;
        ONE_CANCEL($one_id) if !$one_done; undef $one_id;
        FAILED() },
);

By comparison, the Future needs_all constructor neatly wraps up all this implicit behaviour, removing the control- and data-flow noise from the code, and much more concisely expressing its intent.

Future->needs_all(
  ONE_F(), TWO_F(),
)->then(sub {
  my ( $one_result, $two_result ) = @_;
  FINISHED($one_result, $two_result);
})->else(sub {
  FAILED();
})->get;

---

So, there we have it. In the past 24 posts we have seen how Futures can neatly express all the various kinds of control-flow logic we typically find in a Perl program, and also express the additional shapes of code we find useful when working with asynchronous and concurrent programming. This neatness ultimately comes from the fact that a Future object is a first-class value representing the operation itself, and being first-class comes the ability to combine it with others to produce new first-class values to represent combinations of this operation with others.

Futures allow the control- and data-flow structure of a program to be inherently expressed together, describing the dependency relationships between individual operations. Both successful results and failures are automatically propagated up from the atomic units that create them, through the various layers of logic up towards the topmost level of the program. Actions in progress can be abandoned when no longer required, causing a graceful cancellation of the activity that had been pending up until that time.

Futures change state from pending to complete when they are provided with a result, meaning that when they become ready they already have the results stored in them. This makes for convenient control-flow that coincides with data-flow; ensuring that the result of an operation is passed to the next operation in the sequence at the time it is executed. This convenient pairing of control- and data-flow stands in contrast to the split nature of other kinds of concurrency control, such as callback functions or locks and mutexes, which generally only manage the flow of control and require other techniques like lexical variables shared between multiple closures to provide the data flow. Such sharing of mutable state between domains of concurrency is the source of many kinds of concurrency bug which cannot happen with Futures.

In summary, Futures provide a useful abstraction to build all kinds of program logic on top of, whether it is initially intended to be asynchronous or not. Middle-level library modules especially will benefit from using Futures to express intent and combine actions together, as they will then automatically be able to make use of asynchronous and concurrent abilities of the base layers they are built from, without having to expressly depend on those being present.

<< First | < Prev

Futures advent day 23

Day 23 - Additional Benefits of Futures

Beyond simply being able replicate regular perl control-flow styles, building program logic on top of Futures has many additional benefits.

The primary benefit is of course the ability to work asynchronously, allowing the concurrency of being able to start multiple operations and wait for them all to succeed. We have seen this with the tree-forming needs_all, needs_any and wait_any constructor methods, and the fmap utility.

my $f = Future->needs_all(
  ONE(), TWO(), THREE(),
);

my ( $one, $two, $three ) = $f->get;

my $f = fmap1 {
  FUNC($_),
} foreach => [ @VALUES ], concurrent => 10;

my @results = $f->get;

Because Futures represent an operation in progress they are an ideal place to provide cancellation logic, allowing the consumer of the would-be result to abandon it and declare it no longer useful. This can be done explicitly by calling the cancel method.

sub PROCESS_REQUEST {
  my ( $req ) = @_;
  my $f = GET_RESULT( $req->PARAMS );

  $f->on_done(sub {
    $req->REPLY( @_ );
  });
  $req->ON_CLIENT_CLOSE(sub {
    $f->cancel;
  });

  return $f;
}

A failed Future provides an analog to a thrown exception, causing an entire chain or tree of operations to be abandoned and propagating back up towards the caller until a suitable error-handling block is found. In addition however, a failed Future can provide a full list of values as well as a single string. This allows error handlers to be much more fine-grained in their ability to distinguish different types of error.

my $f = GET("http://my-site-here.com/")
  ->else_with_f(sub {
    my ( $f, $failure, $op ) = @_;
    # may be           http, $request, $response
    if( $op eq "http" and $_[3]->code == 500 ) {
      say "Server is unavailable";
      return Future->new->done( $HOLDING_PAGE );
    }
    return $f;
  });

Middleware library functions can easily be built on top of basic actions implemented by futures and providing more of their own. When writing and testing sub libraries it becomes a simple matter to use these futures within the unit-tests themselves as a way to mock out responses from lower levels of logic in order to test the library code in isolation. For example, if we wish to unit-test a middleware function that uses an HTTP user agent to fetch a page, parse it, and return the page title we can provide a simple tiny user agent wrapper that just returns a new future, and does nothing else:

my $resp_f; my $url;
sub GET {
  ( $url ) = @_;
  return $resp_f = Future->new;
}
...

Our unit test can then drive the behaviour of that "user agent", as well as testing the function's results:

...
my $f = get_page_title( \&GET, "http://my-site-here.com/" );

isa_ok( $f, "Future" );
is( $url, "http://my-site-here.com/" );
ok( defined $resp_f, 'Response future created' );
is( !$f->is_ready, '$f is not yet ready' );

$resp_f->done( HTTP::Response->new(
  200, "OK", [ Content_type => "text/html" ],
  "<html><head><title>My title</title></head><body /></html>",
));

ok( $f->is_ready, '$f is now ready after HTML response' );
is( scalar $f->get, "My title", '$f->get returns title' );

Because we have a future to represent both sides of the function (its caller and the inner GET function it uses to fetch the page content) we have been able to easily test the function in the middle. The unit test script itself at various times takes on the role either of the outside caller or the inner HTTP user agent, and is able to easily interleave the two to ensure a neat test.

<< First | < Prev | Next >

---

Edit 2013/12/29: Updated to use else_with_f

Futures advent day 22

Day 22 - Equivalence of Control Flow

Over the past twenty or so posts we have seen many examples of control-flow structures using Futures to help write possibly-asynchronous, possibly-concurrent code in simple, neat ways which mirror the regular kinds of synchronous control flow that Perl already provides. You may by now have come to the conclusion that it is possible to duplicate any kind of control-flow logic using futures.

Simple sequencing of one operation then the next is done using then:

FIRST(); SECOND(); THIRD();

FIRST() ->then(sub { SECOND() }) ->then(sub { THIRD() });

Sequencing is interrupted by thrown exceptions, which may be caught using else:

try { FIRST(); SECOND(); } catch { CATCH(); };

FIRST() ->then(sub { SECOND () }) ->else(sub { CATCH() });

Conditional execution can be performed by using a regular if inside a then_with_f sequence, to return either a new or the original future:

FIRST(); if( $COND ) { SECOND(); }

FIRST() ->then_with_f(sub { my ( $f ) = @_; if( $COND ) { return SECOND() } return $f; });

Repeated loops such as do {} while can be implemented using Future::Utils::repeat():

do { BODY() } while( $COND )

repeat { BODY(); } while => sub { $COND };

A pre-condition while {} loop is a little trickier because it still needs a future to return if the loop body doesn't execute at all. The simplest way is simply to skip the repeat call if it isn't required:

while( $COND ) { BODY(); }

!$COND ? Future->new->done() : repeat { BODY(); } while => sub { $COND };

A foreach {} loop can also be written using repeat:

foreach my $VAR ( @LIST ) { BODY(); }

repeat { my $VAR = shift; BODY(); } foreach => [ @LIST ];

The values generated by a map {} call can be created using Future::Utils::fmap:

my @values = map { BODY(); } @LIST;

my $value_f = fmap { BODY(); } foreach => [ @LIST ];

In every case here, the future version of the control flow structure yields a future, which of course can then be combined inside other structures as required:

do { FIRST(); foreach ( @LIST ) { SECOND($_); } if( $C ) { THIRD(); } } until( $HAPPY )

repeat { FIRST() ->then(sub { repeat { SECOND(shift); } foreach => [ @LIST ] }) ->then_with_f(sub { my ( $f ) = @_; if( $C ) { return THIRD(); } return $f; }); } until => sub { $HAPPY };

<< First | < Prev | Next >

---

Edit 2013/12/29: Updated to use then_with_f

Futures advent day 21

Day 21 - Implementing Timeouts

Suppose now we have a function called TIMEOUT(), similar to SLEEP() except it returns a future that will fail at some later time. We can combine this with another future using the wait_any combination function, to create a timeout. Like needs_any and needs_all, this combination function takes a list of futures and returns a future representing their combination. In this case the combination will complete as soon as any of the individual futures completes, regardless of success or failure. At this point, any other pending futures are cancelled.

my $f = Future->wait_any(
  GET( "http://my-site-here.com/might-be-slow" ),
  TIMEOUT( 20 ),
);

my $page = $f->get;

Here we are using wait_any to add timeout behaviour to a page get operation. If the HTTP operation either succeeds or fails before the 20 seconds are up then $f will complete with the same result or failure, and the timeout will be cancelled. Alternatively, if it still has no result after 20 seconds then the timeout future will fail, causing the overall future $f to fail, and the HTTP operation will be cancelled. In this way we can easily add timeout behaviour to any future-returning function, without every operation individually having to implement timeouts of any kind.

<< First | < Prev | Next >

Futures advent day 20

Day 20 - Automatic Cancellation

A few days ago we saw two similar ways to combine a list of futures into a single future; needs_all and needs_any. In both of these combinations it can happen that the combined future has its result determined before all of its components are ready - needs_all if any component fails, or needs_any if any component is successful. In that case there is no need to continue running the remaining operations, as their outcome cannot further influence the combination.

Yesterday we saw how futures support a cancel operation which cascades back up the tree of operations, causing individual components of an action to be cancelled, and generally all the operations halted. Naturally, both needs_all and needs_any will do this when they are cancelled, causing all of their still-pending components to be cancelled. But additionally, both will cancel remaining pending futures automatically when their result is already known. needs_all will cancel any futures that remain pending when it has to fail because a component failed, and needs_any will cancel any that remain when it already has a result.

For example, recalling day 13 where we first saw needs_all:

my $f = Future->needs_all(
  GET( "http://my-site-here.com/page1" ),
  GET( "http://my-site-here.com/page2" ),
  GET( "http://my-site-here.com/page3" ),
);

my @pages = $f->get;

As was mentioned before, if any of these page fetches fails then the overall operation will fail too, and the get method will throw this exception. What wasn't mentioned before is that since at this point any other pending fetch operations have no further bearing on the result of the combined future, there is no need to continue running them. In this situation needs_all will cancel them.

<< First | < Prev | Next >

Futures advent day 19

Day 19 - Cancellation

As a future is a first-class object that represents an ongoing operation or activity, it serves as an ideal place to interact with that activity. As well as waiting for success or failure of this operation, we can also cancel it midway through by using the cancel method.

my $f = repeat {
  my ( $prev ) = @_;
  SLEEP($prev ? 10 : 0)->then( sub {
    GET("http://my-site-here.com/try-a-thing");
  });
} until => sub { $_[0]->get->code == 200 };

$SIG{INT} = sub { $f->cancel };

$f->get;

Here we have started an HTTP GET operation in a repeat loop, hoping to eventually achieve a success. If in the meantime the user gets bored and hits Ctrl-C, the SIGINT handler cancels the future representing the repeat loop. This will cause it to not continue executing another attempt, and also cascades the cancel call into its currently-running attempt. This cancel call continues to cascade down towards the individual basic futures that the entire operation is composed of. The behaviour of the then chain, for example, depends on how far the operation has progressed; in this case cancelling either the SLEEP or the GET.

The basic futures provided by event systems (such as the SLEEP call) can use the on_cancel method to register a code block to call if the future is cancelled. This could perhaps stop the event that they would use to implement the timer behaviour.

sub SLEEP {
  my ( $delay ) = @_;

  my $f = Future->new;

  my $id = EventSystem->timed_event(
    $delay, sub { $f->done }
  );

  $f->on_cancel( sub {
    EventSystem->stop_timed_event( $id );
  });

  return $f;
}

This cascading of cancellation requests allows future-based code to easily support cancelling partially-complete operations without having to implement the logic to track progress and direct the request appropriately. Simply provide on_cancel handlers for the basic future operations that make the overall activity and the request will be handled appropriately.

<< First | < Prev | Next >

Futures advent day 18

Day 18 - Using fmap Concurrently

The main difference between these seemingly-similar utilities of repeat and fmap is that repeat is intended for performing a given action repeatedly where each attempt may in some way depend on the result of the previous, whereas fmap is intended for performing a given action independently across a given list of items. Because these attempts are independent, there is no requirement to run just one of them at once. As we are able to perform actions asynchronously and concurrently using futures, it makes sense to allow fmap to do this. This is done by passing a number to the concurrent argument of fmap.

my $f = fmap {
  my ( $id ) = @_;
  GET("http://my-site-here.com/items/$id")
} foreach => [ 1 .. 200 ],
  concurrent => 10;

my @pages = $f->get;

The fmap utility can then start as many concurrent HTTP GET operations as we have asked for, 10 in this case, and tries to keep this number of them running as they complete; starting another each time. When finally all of them are complete, the returned $f itself then completes, giving the results as before.

Because we are now running multiple operations concurrently, it could be the case that the ten concurrently-running items complete in a different order than the order the IDs appeared in the original list. fmap takes care of this, ensuring it returns results from the $f->get call in the original input order regardless. It behaves similar to a perl map {} operator, concatenating the results of each individual attempt. Because it can't know in advance how many results will come back and in which order, to achieve this concatenation it has store the results of each call into a list of array references, and flatten it at the end when it is returned. Often this is unnecessary as we know each attempt will only yield a single reply - in this case we can use the more efficient fmap1, which takes exactly one result from each attempt.

my $f = fmap1 {
  my ( $id ) = @_;
  GET("http://my-site-here.com/items/$id")
} foreach => [ 1 .. 200 ],
  concurrent => 10;

my @pages = $f->get;

In other situations we may not even need to return any results at all, and are using it simply to iterate a block of code concurrently (rather than using repeat). For that situation, fmap_void is similar again, but does not bother to collect results at all; when it completes it yields an empty list from its future.

<< First | < Prev | Next >

Futures advent day 17

Day 17 - Returning a List of Results

We have now seen how Future::Utils::repeat can create control structures to repeatedly run a piece of code returning a future, until some ending condition occurs. When it finishes, the result of the repeat future yields the value of the final attempt it tried.

Sometimes though, we want to run an operation repeatedly and collect up all the results it yielded, rather than just the final one. If we have a list of things to iterate on and perform an operation on each we may wish to gather all the results from this. To do that we can use Future::Utils::fmap.

my $f = fmap {
  my ( $id ) = @_;
  GET("http://my-site-here.com/items/$id")
} foreach => [ 1 .. 200 ];

my @pages = $f->get;

Similar to the cases of repeat, here the returned future once again represents the overall operation of running the loop, and will only complete once the loop has finished running. Each item in the foreach list is given to each call to the code block inside. However, instead of the future yielding just the last result, all of the results are collected up and returned in a list.

<< First | < Prev | Next >

Futures advent day 16

Day 16 - Iterating Over a List

Yesterday we looked at the repeat utility, and how it can form a future representing a repeated sequence of attempts to perform some action, when each attempt returns a future to represent it. That example used the result of each attempt to decide if the overall operation should be considered a success, or to have another go.

Sometimes, the number of times to run a loop is known in advance, or at least does not depend on the result, because we wish instead to iterate over items of some given input data. To perform this we can use repeat like a perl foreach {} loop, taking a list of items from an array reference, and invoking the action block once for each item in the list. As before, it returns a future which this time will complete once the input list is exhausted and the final item's attempt has completed.

my %items = ...;

my $f = repeat {
  my ( $key ) = @_;
  PUT("http://my-site-here.com/items/$key",
    $items{$key}
  )
} foreach => [ keys %items ];

$f->get;

Because a foreach list is provided, the repeat function passes the body code successive values from it on each iteration. We don't need to supply a while or until condition here, because the loop knows to terminate when the list is exhausted.

<< First | < Prev | Next >

Futures advent day 15

Day 15 - Performing an Action Repeatedly

So far in this series we have been performing future-based actions that might succeed, or might fail, and combined them up into larger operations of success or failure. One fairly standard way of handling failures in application logic is by attempting to retry. We couldn't simply put a future get call inside a perl while {} loop, because then the loop itself would block until success and we couldn't do anything else at the same time.

Instead, we can use a function from the Future::Utils package called repeat. This takes a block of code which returns a future, and returns a future representing the operation of repeatedly calling that code and waiting for its future to finish, until some ending condition is satisfied. This condition is specified by a second block of code given as either the until or while arguments.

use Future::Utils qw( repeat );

my $f = repeat {
  POST("http://my-site-here.com/new", $form)
} until => sub { $_[0]->get->code == 200 };

my $page = $f->get;

Each time around the loop we will attempt to POST to the resource. When this HTTP operation finishes the until condition is tested. In this example, we are looking for a 200 OK response from HTTP. If we get any other response, we'll try again. The entire operation is represented by the future returned in $f. When we eventually get a response that is deemed acceptable, the overall future $f will receive its result.

In this rather simple example we'll retry each attempt immediately, with no limit on the number of retries. A more practical example would apply a delay between each call, and limit the number of retries. Let us now suppose we have a sleep-like function, DELAY, which returns a future that will complete some number of seconds later.

my $retries = 5;

my $f = repeat {
  my ( $prev ) = @_;

  DELAY($prev ? 10 : 0)
    ->then( sub {
      POST("http://my-site-here.com/new", $form)
    })
} while => sub { $_[0]->get->code != 200 and
                 $retries-- };

my $page = $f->get;

Here we keep a count of the number of retry attempts, and stop retrying once we run out of them. We can implement the pause between retries by asking the DELAY function for a pause of either 0 or 10 seconds, depending on whether there had been a previous failure (this being indicated by the presence or absence of the previous future being passed in to the code body).

<< First | < Prev | Next >

Futures advent day 14

Day 14 - Waiting for Alternatives

Yesterday's look at needs_all was our first look into an actual concurrent use-case where we can really benefit from the asynchronous nature of a Future, to combine multiple page requests concurrently and wait for them all to respond.

Today we look at a similar function, needs_any, which also takes a list of individual futures and returns a new future to represent their combination. In this instance, the new future will complete the first time any of its components completes successfully, or will fail once all of its individual components have failed.

my $f = Future->needs_any(
  GET( "http://uk.my-site-here.com/cache/a" ),
  GET( "http://de.my-site-here.com/cache/a" ),
  GET( "http://us.my-site-here.com/cache/a" ),
);

my $resp = $f->get;

Here we have started three different HTTP GET operations from three different servers, in the hope that whichever one is closest will respond first, thus making the overall future return the page. If that server happened to return an error, this will be ignored while there are still other alternatives available. The overall future will only yield an error if all the servers do. This gives us an easy way to attempt a variety of strategies to provide an answer to a given question.

<< First | < Prev | Next >

Futures advent day 13

Day 13 - Waiting For Multiple Futures

Yesterday we took our first look at some actually-asynchronous uses of a Future, by taking a look at Net::Async::HTTP, but so far in this series we haven't actually seen anything that properly makes use of a Future, that could not be done just as easily synchronously.

Now we can take our first look at some code that uses the asynchronous nature of an HTTP user agent to fetch multiple resources concurrently. By using the Future->needs_all constructor we can create multiple futures to fetch individual pages, then combine them together into a single future which we can then wait on by calling get.

my $f = Future->needs_all(
  GET( "http://my-site-here.com/page1" ),
  GET( "http://my-site-here.com/page2" ),
  GET( "http://my-site-here.com/page3" ),
);

my @pages = $f->get;

Here we have created three individual futures representing an HTTP page GET, and wrapped them all in a single wrapper future which will complete when all of its components are complete. The get call on this future then returns a list composed of the result of each individual future. Because we know each one only returns the HTTP::Response object itself, we can just get these as a list. If any of the individual HTTP GET futures fails, the overall combined future will fail too. This means we can neatly handle any exceptions that happen.

When using a properly asynchronous HTTP user agent we are now able to perform these multiple GETs concurrently. Each call to GET starts the fetch operation, which can now all complete concurrently eventually returning their results. This is our first real example of futures providing neat concurrency-enabling code.

<< First | < Prev | Next >

Futures advent day 12

Day 12 - Asynchronous Await

Up until now in these posts I have been deliberately vague on the subject of how the HTTP GET function actually works. All I have implied is that it takes a page URL and returns a Future that will eventually yield an HTTP::Response containing the resource. As I said right back in the first post on day 1, Futures work just fine if every result is in fact a synchronous return. Thus, we could choose to implement an entirely synchronous version of this function using LWP::UserAgent (and taking care not to confuse its get method with the unrelated Future one):

use Future;
use LWP::UserAgent;
my $ua = LWP::UserAgent;

sub GET
{
  my ( $url ) = @_;
  
  Future->call( sub {
    Future->wrap( $ua->get( $url ) )
  });
}

Here we have used Future->wrap to conveniently create a future to contain the result of a successful call to the UserAgent's get method (remember, this is HTTP GET and unrelated to the future get method). This call is itself wrapped in a Future->call block to ensure that if the UserAgent throws an exception, this will be wrapped in a failed future.

Alternatively, if we wanted some level of asynchronous behaviour, because we wish to perform multiple concurrent actions, or mix this with other code, we could instead use Net::Async::HTTP which already provides a GET method having the semantics we want:

use IO::Async::Loop;
use Net::Async::HTTP;

my $loop = IO::Async::Loop->new;
my $http = Net::Async::HTTP->new;
$loop->add( $http );

sub GET
{
  my ( $url ) = @_;
  return $http->GET( $url );
}

This one is implemented internally by Net::Async::HTTP returning a subclass of Future provided by IO::Async itself. This subclass understands how to wait for futures that are not yet ready, by invoking the containing loop until the result is available. Other event systems can be similarly catered for by subclassing Future to provide a suitable await method, which is used by get. We could even, if we were inclined towards threads, implement a subclass of Future which used some kind of thread-based synchronisation and communication to await the result being supplied by code running in a different thread.

Because they both conform to the interface of "returning a Future", either of these above implementations of GET are suitable for any of the examples we have seen so far, or will see in the next examples to come. So too would any other implementation that provides this interface. Because of this we find that Futures provide a powerful way to write the intermediate layers of processing and "business logic" in application libraries, which can remain agnostic on such low-level details as what event system is being used, or even if one is being used at all.

<< First | < Prev | Next >

Futures advent day 11

Day 11 - Transforming Results or Failures

While we are on the subject of convenient shortcuts to neaten up certain kinds of Future result handling, another common pattern that arises is the case of an immediate return; either a then block that returns an immediate result based on the given values, or an else block that returns an immediate failure based on the given one. In each of these cases the transform method allows us to write this more conveniently.

sub GET_title
{
  my ( $url ) = @_;

  GET_checked($url)->transform(
    done => sub { get_page_title( $_[0] ) },
  );
}

If the GET_title is successful then the code passed in the done argument is invoked with the result as the argument list, and whatever it returns is used as the result of the future that transform returned. This is a little shorter and more convenient than the functionally-equivalent then block returning an immediate future.

We can similarly use transform on the failure message to create a different message, perhaps with more details in it.

sub GET_title
{
  my ( $url ) = @_;

  GET_checked($url)->transform(
    done => sub {
      get_page_title( $_[0] )
    },
    fail => sub {
      my $message = shift;
      "Cannot get title of $url - $message", @_;
    }
  );
}

Because the failure may contain other values giving more context, we need to be careful to preserve them. This is done most easily by shifting the message, so the remaining values appear in @_.

<< First | < Prev | Next >

Futures advent day 10

Day 10 - Conditional Chaining

Over the past few days we have seen uses of then and else to chain sequences of code together. Sometimes the code in a then or else block will inspect its given arguments, and decide that it doesn't in fact want to perform any other action so just returns a new Future containing the same values again.

Instead of this, we can use the methods then_with_f and else_with_f, which are variations of then and else which pass the code block the actual Future object they are invoked on in addition to the result or failure list it contains. The code can then either construct a new Future containing different results, or just return that one directly.

sub GET_checked
{
  my ( $url ) = @_;

  GET( $url )->then_with_f( sub {
    my ( $f, $resp ) = @_;

    return $f if $resp->code =~ m/^[23]../;

    return Future->new->fail($resp->code." ".$resp->message, $resp);
  });
}

This is not only neater and clearer to read, but is also more efficient because it doesn't need to create yet another Future object just to contain the same result that the one it was invoked on already had. Equally, an else_with_f can neaten up the way we sometimes simply propagate a failure if we decide not to handle it.

my $f = GET_checked("http://my-site-here.com/a-page")
  ->else_with_f( sub {
     my ( $f, $failure, $response ) = @_;

     return $f if $response->code != 500;

     return Future->new->done(
       HTTP::Response->new( 200, "OK", [],
         "Server is down, but have some fluffy kittens instead")
     );
  });

<< First | < Prev | Next >

---

Edit 2013/12/29: Updated for then_with_f and else_with_f