Futures advent day 18

Day 18 - Using fmap Concurrently

The main difference between these seemingly-similar utilities of repeat and fmap is that repeat is intended for performing a given action repeatedly where each attempt may in some way depend on the result of the previous, whereas fmap is intended for performing a given action independently across a given list of items. Because these attempts are independent, there is no requirement to run just one of them at once. As we are able to perform actions asynchronously and concurrently using futures, it makes sense to allow fmap to do this. This is done by passing a number to the concurrent argument of fmap.

my $f = fmap {
  my ( $id ) = @_;
  GET("http://my-site-here.com/items/$id")
} foreach => [ 1 .. 200 ],
  concurrent => 10;

my @pages = $f->get;

The fmap utility can then start as many concurrent HTTP GET operations as we have asked for, 10 in this case, and tries to keep this number of them running as they complete; starting another each time. When finally all of them are complete, the returned $f itself then completes, giving the results as before.

Because we are now running multiple operations concurrently, it could be the case that the ten concurrently-running items complete in a different order than the order the IDs appeared in the original list. fmap takes care of this, ensuring it returns results from the $f->get call in the original input order regardless. It behaves similar to a perl map {} operator, concatenating the results of each individual attempt. Because it can't know in advance how many results will come back and in which order, to achieve this concatenation it has store the results of each call into a list of array references, and flatten it at the end when it is returned. Often this is unnecessary as we know each attempt will only yield a single reply - in this case we can use the more efficient fmap1, which takes exactly one result from each attempt.

my $f = fmap1 {
  my ( $id ) = @_;
  GET("http://my-site-here.com/items/$id")
} foreach => [ 1 .. 200 ],
  concurrent => 10;

my @pages = $f->get;

In other situations we may not even need to return any results at all, and are using it simply to iterate a block of code concurrently (rather than using repeat). For that situation, fmap_void is similar again, but does not bother to collect results at all; when it completes it yields an empty list from its future.

<< First | < Prev | Next >