Introduction
Various articles I have previously written have described Futures and their use, such as the Futures Advent Calendar. In this article, I want to present a new syntax module that greatly improves the expressive power and neatness of writing Future-based code. This module is Future::AsyncAwait.
The new syntax provided by this module is based on two keywords, async and await that between them provide a powerful new ability to write code that uses Future objects. The await keyword causes the containing function to pause while it waits for completion of a future, and the async keyword decorates a function definition to allow this to happen. These keywords encapsulate the idea of suspending some running code that is waiting on a future to complete, and resuming it again at some later time once a result is ready.
use Future::AsyncAwait;
async sub get_price {
my ($product) = @_;
my $catalog = await get_catalog();
return $catalog->{$product}->{price};
}
This already reads a little neater than how this might look with a ->then chain:
sub get_price {
my ($product) = @_;
return get_catalog()->then(sub {
my ($catalog) = @_;
return Future->done($catalog->{$product}->{price});
});
}
This new syntax makes a much greater impact when we consider code structures like foreach loops:
use Future::AsyncAwait;
async sub send_message {
my ($message) = @_;
foreach my $chunk ($message->chunks) {
await send_chunk($chunk);
}
}
Previously we'd have had to use Future::Utils::repeat to create the loop:
use Future::Utils qw( repeat );
sub send_message {
my ($message) = @_;
repeat {
my ($chunk) = @_;
send_chunk($chunk);
} foreach => [ $message->chunks ];
}
Because the entire function is suspended and resumed again later on, the values of lexical variables are preserved for use later on:
use Future::AsyncAwait;
async sub echo {
my $message = await receive_message();
await delay(0.2);
send_message($message);
}
If instead we were to do this using ->then chaining, we'd find that we either have to hoist a variable out to the main body of the function to store $message, or use a further level of nesting and indentation to make the lexical visible to later code:
sub echo {
my $message;
receive_message()->then(sub {
($message) = @_;
delay(0.2);
})->then(sub {
send_message($message);
});
}
# or
sub echo {
receive_message()->then(sub {
my ($message) = @_;
delay(0.2)->then(sub {
send_message($message);
});
});
}
These final examples are each equivalent to the version using async and await above, yet are both much longer, and more full of the lower-level "machinery" of solving the problem, which obscures the logical flow of what the code is trying to achieve.
Comparison With Other Languages
This syntax isn't unique to Perl - a number of other languages have introduced very similar features.
ES6, aka JavaScript:
async function asyncCall() {
console.log('calling');
var result = await resolveAfter2Seconds();
console.log(result);
}
async def main():
print('hello')
await asyncio.sleep(1)
print('world')
C#:
public async Task<int> GetDotNetCountAsync()
{
var html = await
_httpClient.GetStringAsync("https://dotnetfoundation.org");
return Regex.Matches(html, @"\.NET").Count;
}
Dart:
main() async {
var context = querySelector("canvas").context2D;
var running = true; // Set false to stop game.
while (running) {
var time = await window.animationFrame;
context.clearRect(0, 0, 500, 500);
context.fillRect(time % 450, 20, 50, 50);
}
}
In fact, much like the recognisable shapes of things like if blocks and while loops, it is starting to look like the async/await syntax is turning into a standard language feature across many languages.
Current State
At the time of writing, this module stands at version 0.22, and has been the result of an intense round of bug-fixing and improvement over the Christmas and New Year break. While it isn't fully production-tested and ready for all uses yet, I have been starting to experiment with using it in a number of less production-critical code paths (such as unit or integration testing, or less widely used CPAN modules) in order to help shake out any further bugs that may arise, and generally evaluate how stable it is becoming.
This version already handles a lot of even non-trivial cases, such as in conjunction with the try/catch syntax provided by Syntax::Keyword::Try:
use Future::AsyncAwait;
use Syntax::Keyword::Try;
async sub copy_data
{
my ($source, $destination) = @_;
my @rows = await $source->get_data;
my $successful = 0;
my $failed = 0;
foreach my $row (@rows) {
try {
await $destination->put_row($row);
$successful++;
} catch {
$log->warnf("Unable to handle row ID %s: %s",
$row->{id}, $@);
$failed++;
}
}
$log->infof("Copied %d rows successfully, with %d failures",
$successful, $failed);
}
Known Bugs
As already mentioned, the module is not yet fully production-ready as it is known to have a few issues, and likely there may be more lurking around as yet unknown. As an outline of the current state of stability, and to suggest the size and criticality of the currently-known issues, here are a few of the main ones:
Complex expressions in foreach lose values
(RT 128619)I haven't been able to isolate a minimal test case yet for this one, but in essence the bug is that given some code which performs
foreach my $value ( (1) x ($len - 1), (0) ) {
await ...
}
the final 0 value gets lost. The loop executes for $len - 1 times with $value set to 1, but misses the final 0 case.
The current workaround for this issue is to calculate the full set of values for the loop to iterate on into an array variable, and then foreach over the array:my @values = ( (1) x ($len - 1), (0) );
foreach my $value ( @values ) {
await ...
}
While an easy workaround, the presence of this bug is nonetheless a little worrying, because it demonstrates the possibility for a silent failure. The code doesn't cause an error message or a crash, it simply produces the wrong result without any warning or other indication that anything went wrong. It is, at time of writing, the only bug of this kind known. Every other bug produces an error message, most likely a crash, either at compile or runtime.
Fails on threaded perl 5.20 and earlier
(RT 124351)The module works on non-threaded builds of perl from version 5.16 onwards, but only on threaded builds 5.22 onwards. Threaded builds of 5.20 or earlier all fail with a wide variety of runtime errors, and are currently marked as not supported. I could look into this if there was sufficient interest, but right now I don't feel it is a good use of time to support these older perl versions, as compared fixing other issues and making other improvements elsewhere.
Devel::Cover can't see into async subs
(RT 128309)This one is likely to need fixing within Devel::Cover itself rather than Future::AsyncAwait, as it probably comes from the optree scanning logic there getting confused by the custom LEAVEASYNC ops created by this module. By comparison, Devel::NYTprof can see them perfectly fine, so this suggests the issue shouldn't be too hard to fix.
Next Directions
There are a few missing features or other details that should be addressed at some point soon.
Core perl integration
Currently, the module operates entirely as a third-party CPAN module, without specific support from the Perl core. While the perl5-porters ("p5p") are aware of and generally encourage this work to continue, there is no specific integration at the code level to directly assist. There are two particular details that I would like to see:
- Better core support for parsing and building the optree fragment relating to the signature part of a sub definition. Currently, async sub definitions cannot make use of function signatures, because the parser is not sufficiently fine-grained to allow it. An interface in core Perl to better support this would allow async subs to take signatures, as regular non-async ones can.
A mailing list thread has touched on the issue, but so far no detailed plans have emerged.
- An eventual plan to migrate parts of the suspend and resume logic out of this module and into core. Or at least, some way to try to make it more future-proof. Currently the implementation is very version-dependent and has to inspect and operate on lots of various inner parts of the Perl interpreter. If core Perl could offer a way to suspend and resume a running CV, it would make Future::AsyncAwait a lot simpler and more stable across versions, and would also pave the way for other CPAN modules to provide other syntax or semantics based around this concept, such as coroutines or generators.
local and await
Currently, the suspend logic will get upset about any local variable modifications that are in scope at the time it has to suspend the function; for instance
async sub x {
my $self = shift;
local $self->{debug} = 1;
await $self->do_work();
# is $self->{debug} restored to 1 here?
}
This is more than just a limit of the implementation, however as it extends to fundamental questions about what the semantic meaning of such code should be. It is hard to draw parallels from any of the other language the async/await syntax was inspired by, because none of these have a construct similar to Perl's local.
Recommendations For Use
Earlier, I stated that Future::AsyncAwait is not fully production-ready yet, on account of a few remaining bugs combined with its general lack of production testing at volume. While it probably shouldn't be used in any business-critical areas at the moment, it can certainly help in many other areas.
Unit tests and developer-side scripts, or things that run less often and are generally supervised when they are, should be good candidates for early adoption. If these do break it won't be critical to business operation, and should be relatively simple to revert to an older version that doesn't use Future::AsyncAwait while a bugfix is found.
The main benefit of beginning adoption is that the syntax provided by this module greatly improves the readability of the surrounding code, to the point that it can itself help reveal other bugs that were underlying in the logic. On this subject, Tom Molesworth writes that:
Simple, readable code is going to be a benefit that may outweigh the potential risks of using newer, less-well-tested modules such as this one.
This advice is similar to my own personal uses of the module, which are currently limited to a small selection of my CPAN modules that relate to various exotic pieces of hardware. Many of the driver modules related to Device::Chip have begun to use it. A list of modules that use Future::AsyncAwait is maintained by metacpan.
I am finding that the overall neatness and expressiveness of using async/await expressions is easy justification against the potential for issues in these areas. As bugs are fixed and the module is found to be increasingly stable and reliable, the boundary can be further pushed back and the module introduced to more places.
This article is adapted from one that was originally written in two parts for the Binary.com internal tech blog - part 1, part 2.
I would also like to thank The Perl Foundation whose grant has enabled me to continue working on this piece of Perl infrastructure.
