2020 Perl Advent Calendar - Day 25

<< First | < Prev

Bonus Day!

Over this blog post series we have built up to the post on day 24, which explains that all of what we've seen this series is available and working in Perl, right now. It is the Perl we can write in 2020. All of this has been possible because of the custom keyword feature which was first introduced to Perl in version 5.14.

When Perl gained the ability to support custom keywords provided by modules it started down the path that CPAN modules would experiment with new language ideas. Already a number of such modules exist, and it is likely this idea will continue to develop. What new ideas might turn up in the next few years, and will any of them evolve to become parts of the actual core language?

Here's a collection of some thoughts of mine. Some of these can be implemented in CPAN modules, in the same way as the four modules we've already seen this series. Other ideas however go beyond what would be possible via keywords alone, and stray into the realm of ideas that really do need core Perl support.

Match/case Syntax

Perl 5.10 added the smartmatch operator, ~~. I think we can mostly agree it has not been the success many had been hoping for. Its rules are complex and subtle, and there's far too many of them to remember. Furthermore, it still doesn't express the most basic question of whether basic scalar comparisons for equallity are performed as string or number tests. For example, is the expression 5 ~~ "5.0" true or false? I honestly don't know and the fact I'd have to look it up in a big table of behavior suggests that the thing has failed to achieve its goal.

Yet still we are left without a useful syntax to express control-flow dispatch based on comparing a given value to several example choices - a task for which many languages use keywords switch and case. I have already written to Perl5 Porters with my thoughts on a design I have nicknamed "dumb match", in response to this. The basic idea of dumb match is to make the programmer write down their choice of operator to be used to compare the given value with the various alternatives.

match($var : eq) {
    case("abc") { ... }
    case("def") { ... }
    case("ij", "kl", "mno") { ... }  # any of these will match

Here the programmer has specifically requested the eq operator, so we know these are stringy comparisons. Alternatively they could have requested any of

match($var : ==) {
    # numerical comparisons
    case(123) { ... }
    case(456) { ... }

match($string : =~) {
    # regexp matches
    case(m/^pattern/)    { ... }
    case(m/morestring$/) { ... }  # only the first match wins

match($obj : isa) {
    # object class matches
    case(IO::Handle) { ... }

Type Assertions

Various people have in various times written about or designed all sorts of variations on a theme of a "type system" for Perl. I have written reactions to some of those ideas before.

The idea I have in mind here is less a feature in itself, and more a piece of common ground for several of the other ideas, though it may have applications to existing pieces of Perl syntax. Common to several ideas is the need to be able to ask, at runtime, whether a given value satisfies some classification criteria. People often bring up thoughts of assertions like "this is a string" or "this is an integer" at the start of these discussions, but that isn't really within the nature or spirit of what Perl's value system can answer. Instead, I think any workable solution would be written in terms of the existing kinds of comparisons.

Perl 5.32 added the isa operator - a real infix operator that asks if its first operand is an object derived from the class given by its second.

if($arg isa IO::Handle) {

This is certainly one kind of type assertion. I could imagine a new keyword, for the sake of argument for now lets call it is*, which can answer similar yes/no questions on a broader category of criteria. It is likely that the righthand side argument would have to be some sort of expression giving a "type constraint", though exactly what that is I admit I don't have a neat design for currently.

*: Yes, I'm aware this operator choice would interfere probably with Test::More::is. Likely a solution can be found somehow, either by a better naming choice, better parser disambiguation, or a lexical feature guard.

It may be the case that generic type constraints can be constructed with an arbitrary Perl expression to explain how to test if a value meets the constraint:

type PositiveNumber is Numeric where { $_ > 0 };

While in general that would be the most powerful system, it may not lead to a very good performance for several of the other ideas here, so I am still somewhat on the fence about this sort of detail. Because I don't have a firm design on this yet, for the rest of this post I'm just going to give examples using the isa operator instead. But any of the examples or ideas would definitely apply to a more generalised type constraint operator or system, whenever one came to exist.

In any case, once a generic is operator exists for testing type constraints, it feels natural to allow that in match/case syntax too:

match($value : is) {
    case(PositiveNumber) { ... }
    case(NegativeNumber) { ... }

In addition it would be wanted in function and method signatures:

method exec($code is Callable)

And also object slot variables:

class Caption
    has $text is Textual;

Multiple Dispatch

Another idea that comes once you have assertions is the idea of hooking that into function dispatch itself. Some languages give you the ability to define the same-named function multiple times, with different kinds of assertion on its arguments, and at runtime the one that best matches the given arguments will be chosen. There are usually many rules and subtleties to this idea, so it may not ultimately be very suitable for Perl, but if a constraint system did exist then it would be relatively simple to write a CPAN module providing a multi keyword to allow these.

multi sub speak($animal isa Cow) { say "Moo" }

multi sub speak($animal isa Sheep) { say "Baaah" }

Naturally this syntax ought to be implemented in a way that means it still works with method and async as well, allowing us to just as easily

async multi method speak_to($animal isa Goose)
    await $self->say("Boo", to => $animal);

Signature-like List Assignment

Perl 5.20 introduced signatures, which can be imagined as a neatening up of the familiar syntax of unpacking the @_ list into variables. In some ways the following two functions could be considered identical:

sub add_a
    my ($x, $y) = @_;
    return $x + $y;

sub add_b($x, $y)
    return $x + $y;

This does however brush over a few more subtle details of signatures. Firstly, signatures are more strict on the number of values they receive vs. how many they were expecting. While this is a useful feature, it seems odd that Perl now lacks any syntax for performing a list unpack and checking that it has exactly the right number of elements in any situation other than the arguments from function entry.

For that task, I could imagine an operator maybe spelled := which acts exactly the same as a signature on a function:

my ($x, $y) := (1, 2);

my ($x, $y) := (1, 2, 3);  # complains about too many values
my ($x, $y) := (1);        # complains about not enough values

Of course, there's more to signatures than simply counting the elements. Signatures permit a default value to be used if the caller did not specify it; we could allow that too:

my ($one, $two, $three = 3) := (1, 2);

If signatures gain features like type assertions then it seems natural to apply them to the signature-like list assignment operator as well, allowing that to check also:

my ($item isa Item, $group isa Group) := @itemgroup;

If key/value unpacking of named arguments arrives then that too would be useful for unpacking a hash:

my (:$height, :$width) := %params;


The slot variables introduced by Object::Pad are written the same as regular lexical variables. I have for a while wished them to be distinct from regular lexicals, so they stand out better visually. The $: syntax can easily be made available, allowing them to be written with that instead:

class Point
    has $:x = 0;
    has $:y = 0;
    method describe($name) {
        say "Hello $name, this point is at ($:x, $:y)";

I accept this is a much more subjective idea than most of the other features. Personally I find it helps to visually distinguish object slots, now that they don't have such notation as $self->{...} to remind you.

True Core Implementations

As earlier mentioned, some of these ideas can be implemented as CPAN modules (those introduced by new keywords), but others (such as the := operator) would require core Perl support. It would also be nice to see some of the more established and stable CPAN keyword modules implemented in core Perl as true syntax as well.

It would be great if, in 2025, we could simply

use v5.40;    # or maybe it will be use v7.x by then

try { ... }
catch ($e) { ... }

class Calculator {
    method add($x, $y) { ... }

Having these available to the core language would hopefully mean that a lot more code would more quickly adopt them as features. While these things are all available as CPAN modules, and work even on historic Perl versions as far back as 5.16 from 2012, it seems that some people don't want to make use of such syntax features unless they are provided by the core language itself. Moving the implementation into core may help for other reasons too, such as efficiency of operation, or allowing them to do yet more abilities not available to them while they are third-party modules.

All in all, it's something we can hope for over the next five years...

<< First | < Prev


2020 Perl Advent Calendar - Day 24

<< First | < Prev

Over the course of this blog post series we have seen a number of syntax-providing modules from CPAN. Each of them sets out to neaten up some specific kind of structure often found in Perl code.

  • Future::AsyncAwait aims to neaten up asynchronous code flow, replacing older techniques like ->then method chaining and helper functions from Future::Utils by replacing them with regular Perl syntax.
  • Syntax::Keyword::Try brings the familiar try/catch pattern for handling exceptions, replacing more manual techniques involving eval {} blocks and inspecting the $@ variable.
  • Object::Pad provides an entire set of syntax keywords for managing classes of objects, allowing stateful object-oriented code to be neatly written without the risk of things like hash key collsions on $self->{...}.

Each one of these allows writing shorter, neater code that has less "machinery noise". With fewer distractions in the code it becomes clearer to see the detail of the specific situation the code is for. With less code to write there's less opportunity to introduce bugs.

Moreover we have seen that these syntax modules can be combined together, used in conjunction to allow even greater benefits. We saw on day 4 that try/catch control flow works within async sub, on day 22 that object methods can be marked as asynchronous with async method, and on day 23 we explored how the dynamically assignment syntax can be combined with objects, asynchronous functions, and even both at the same time.

The various code examples we've seen over the past 22 days or so have been written using these syntax modules, and also make use of Perl's signatures feature, and other things where possible, all to help in this regard. The shorter neatness that comes from not needing to write the line (or two) of code to unpack the function's arguments from @_ (and maybe the $self method invocant as well) removes yet another distraction and potential source of errors.

In summary: This series has been about what it feels like to write Perl code in the year 2020 - it has been about 2020 Perl. This is a language just as flexible and adaptable as Perl has ever been, yet still capable of any of the modern techniques common to other languages, which perhaps even the Perl of five or ten years ago was lacking in - neat function arguments, asynchronous control, exception handling, and syntax for object orientation. With all these new abilities, 2020 has been a great year for writing Perl code.

<< First | < Prev


2020 Perl Advent Calendar - Day 23

<< First | < Prev | Next >

For today's article, I'd like to take a look at yet another of my syntax-providing CPAN modules, Syntax::Keyword::Dynamically. This provides a single new keyword, dynamically. To quote its documentation:

Syntactically and semantically it is similar to the built-in perl keyword local, but is implemented somewhat differently to give two key advantages over regular local:
  • You can dynamically assign to lvalue functions and accessors.
  • You can dynamically assign to regular lexical variables.

This is important to us when working with Object::Pad because of the way slot variables work. Within a method body a slot looks like a regular lexical variable. This means that Perl's regular local keyword refuses to interact with one. If we want to assign a new value temporarily, only for the duration of one block of code and have it restored automatically afterwards, we must use dynamically instead.

For example, both Syntax::Keyword::Dynamically and Object::Pad contain a copy of a unit test which asserts that their interaction works as expected.:

has $value = 1;
method value { $value }

method test
    is $self->value, 1, 'value is 1 initially';

        dynamically $value = 2;
        is $self->value, 2, 'value is 2';

    is $self->value, 1, 'value is 1 finally';

If instead we were to try this using core Perl's local it fails to compile:

        local $value = 2;
$ perl -c example.pl
Can't localize lexical variable $value at ...

When a variable is dynamically assigned a new value inside an asynchronous function it has to be swapped back to its original value while that function is suspended, and its new value put back when the function resumes. This may have to happen several times before the function eventually returns. The way that dynamically is implemented means it is supported by Future::AsyncAwait and can detect the times it needs to swap values back and forth.

There is also a unit test which checks this interaction in both Syntax::Keyword::Dynamically and Future::AsyncAwait:

my $var = 1;

async sub with_dynamically
    my $f = shift;

    dynamically $var = 2;

    is $var, 2, '$var is 2 before await';
    await $f;
    is $var, 2, '$var is 2 after await';

my $f1 = Future->new;
my $fret = with_dynamically( $f1 );

is $var, 1, '$var is 1 while suspended';

is $var, 1, '$var is 1 after finish';

Given these three modules are now known to be working nicely in each of the three pairwise combinations, you might wonder if all three can be combined at once - can you dynamically change the value of an object slot during an async method? The answer is still yes.

All three of these module distributions contain a copy of a unit test which checks this behaviour:

class Logger {
    has $_level = 1;

    method level { $_level }

    async method verbosely {
        my ( $code ) = @_;
        dynamically $_level = $_level + 1;
        is $self->level, 2, 'level is 2 before code';
        await $code->();
        is $self->level, 2, 'level is 2 after code';

my $logger = Logger->new;

my $f1 = Future->new;
my $fret = $logger->verbosely(async sub {
    is $logger->level, 2, 'level is 2 before await';
    await $f1;
    is $logger->level, 2, 'level is 2 after await';

is $logger->level, 1, 'level is 1 outside';


is $logger->level, 1, 'level is 1 finally';

Each of these syntax modules has provided something useful on its own, but as we have seen both yesterday and today they can be combined with each other to provide even more useful behaviours. It is easily possible to create CPAN modules that operate together to extend the Perl language with new syntax and semantics, and have those extensions work and feel every bit as convenient and powerful as all of the native syntax built into the language.

<< First | < Prev | Next >


2020 Perl Advent Calendar - Day 22

<< First | < Prev | Next >

We started off this advent calendar series looking at the async/await syntax provided by Future::AsyncAwait, and the way that functions can be marked as async. More recently we have been looking at the class and object syntax provided by Object::Pad, such as syntax to provide named methods. Some of you may be wondering whether these two things can be combined; whether methods can be marked as being asynchronous. The answer is yes.

The way that these two modules are implemented means that they can coƶperate on how functions are parsed. The end result is that a method can be declared using the combined keywords async method and it behaves exactly as expected. Namely, that $self and the class's slot variables are available within the code, it returns a future-wrapped value, and permits the await keyword.

For example, back on day 6 we saw an example of await with a //= shortcircuit expression to optionally wait for a read operation to fill a cache on an object, implemented with a $self->{...} key inside async sub. At the time I said that the example was slightly reworded from the original code. That is because in reality, the code is implemented using the combination of async and method:

use Object::Pad;
use Future::AsyncAwait;

class Device::Chip::TSL256x extends Device::Chip;


has $_TIMINGbytes;

async method _cached_read_TIMING ()
    return $_TIMINGbytes //= await $self->_read(REG_TIMING, 1);

In fact, almost every post after that also had some code taken from modules that are implemented using async method. In each case, the real code was in fact shorter and more concise than the posted example because it did not have to start with the my $self = shift; line initially, and could use the shorter slot variables instead of hash key accesses on $self->{...}.

These two syntax modules - either individually or in combination - are able to greatly neaten a lot of common code patterns. To see just how much they provide here is what the method above might have been written if neither syntax module was used:

sub _cached_read_TIMING
    my $self = shift;

    return Future->done($self->{TIMINGbytes})
        if defined $self->{TIMINGbytes};
    return $self->_read(REG_TIMING, 1)->then(sub {
        ($self->{TIMINGbytes}) = @_;
        return Future->done($self->{TININGbytes});

In this version of the code it is far less obvious to see the flow of the logic. The caching behaviour of the TIMINGbytes field is harder to see, hidden by the various machinery of the future return value and ->then chaining. Additionally, the $self->{TIMINGbytes} field is referred to four times here - each one being just a hash key, and thus prone to typoes. Sure there are techniques to help detect such problems with classical Perl hash-based objects (such as locked hashes), but those all detect runtime attempts to actually touch the fields; none of them are able to point out problems at compiletime.

Such an error would be detected at compiletime using an Object::Pad-based slot variable:

has $_TIMINGbytes;

async method _cached_read_TIMING {
    return $_TININGbytes //= await $self->_read(REG_TIMING, 1);
$ perl -c example.pl
Global symbol "$_TININGbytes" requires explicit package name
  (did you forget to declare "my $_TININGbytes"?) at ...

By the way, did anyone spot the typo on the long example code above? I didn't, the first time I wrote it... ;)

<< First | < Prev | Next >


2020 Perl Advent Calendar - Day 21

<< First | < Prev | Next >

So far we've been looking at features of some syntax modules that are relatively well-established - Future::AsyncAwait has a couple of years of production battle-testing against it, and even Object::Pad's basic class features have been found to be quite stable over the past six months or so. For today's article I'd like to take a slightly different direction and take a look at something much newer and still under experimental design.

Some object systems which use inheritance to create derived classes out of base ones (including the base system in Perl itself) support the idea that a given class may have multiple bases. This is called Multiple Inheritance. Iniitally it may sound like a useful feature to have, but in practice trying to support it makes implementations of object systems more complicated, and can lead to situations where the choice of correct behaviour is non-obvious, or in some cases conflicting with what may seem sensible. Situations get especially complicated if the same partial class appears multiple times in the inheritance hierarchy leading up to a given class.

For this reason most modern object systems, including Object::Pad, do not support multiple interitance, to keep behaviours simpler. In order to try to provide the same useful properties (that of being able to share code from multiple component classes), they provide a somewhat different idea, called roles. A role can be considered similar to a partial class which can be merged into a real class. A role can provide methods, BUILD blocks, and slot variables. In many ways a role appears the same as a class, except that instances of it cannot be directly created. To be used as an instance a role must be applied to a class. This has the effect of copying all of the pieces of that role into the target class.

For example, in the Tickit-Widget-Menu distribution there are two different classes of object that can appear in a menu - an individual menu item, or a submenu. In order to avoid code duplication by copying parts of the implementation around both classes, the common behaviours are implemented in a role, by using the role keyword:

use Object::Pad 0.33;

role Tickit::Widget::Menu::itembase;

has $_name;

BUILD (%args)
    $_name = $args{name}


To apply this role to both of the required classes each uses the implements keyword on its class statement to copy the components of that role into the class:

use Object::Pad 0.33;

class Tickit::Widget::Menu:::Item
    implements Tickit::Widget::Menu::itembase;

class Tickit::Widget::Menu::base
    implements Tickit::Widget::Menu::itembase;

Superficially this might feel like it suffers the same problems as multiple inheritance, but keep in mind that applying a role is basically just a fancy form of copy-pasting the code into the class. There is no runtime lookup of methods or other class items whenever they are accessed. The parts of a role are simply copied individually into the class that applies it. This means that any naming conflicts are detected as errors at compile-time, alerting the programmer to the potential problem:

use Object::Pad 0.33;

role R
    method collides() {}

class C implements R
    method collides() {}
$ perl example.pl
Method 'collides' clashes with the one provided by role R at ...

A program will only successfully compile if there are no naming collisions. As a result of this, and because the pieces of the role are simply copied into a class, it means that it does not matter in what order individual roles are applied to a class, nor does it matter if the same role is applied multiple times within the hierarchy (e.g. if both a class and its base class tried to apply the same role). The end result is always the same, presuming no conflicts. This compiletime check, and flexibility on ordering and duplicate application, helps to ensure more robust code.

<< First | < Prev | Next >


2020 Perl Advent Calendar - Day 20

<< First | < Prev | Next >

We have now seen the way that the has keyword creates a new kind of variable, called a slot variable, where object instances can store their state values. All of the code in yesterday's examples creates variables that begin, like a new my variable, as the undefined value. Often though with an object instance we want to store some other value initially. For this there are two options available.

In simple cases where slot variables of any new object should start off with the same default value we can use an expression on the has statement itself to assign a default value. In these two examples, the slot is initialised from a simple constant.

class Device::Chip::AD9833 extends Device::Chip;

has $_config = 0;
class Tickit::Widget::LinearSplit
    extends Tickit::ContainerWidget;
has $_split_fraction = 0.5;

These are compiletime constants, though any form of expression is allowed here. However, note: much like would apply to a my or our variable in the scope of an entire package or class, any expression is evaluated just once at the time the class itself is first created. The resulting value is stored as the default for every new instance. This expression is not evaluated for each new instance individually. Thus it is rare in practice to see anything other than a constant here. For example, using an expression that created some new helper object would mean that all new instances of the containing class will share the same reference to the same helper object - unlikely what was intended.

For more complex situations which require code to be evaluated for every new instance of a class we can use a BUILD block. This provides a block of code which is run as part of the construction process for every individual instance of the class. For example, this BUILD block allows us to create a new mutex helper instance for every instance of the containing class:

class Device::Chip::LEO1306
    extends Device::Chip::Base::RegisteredI2C;

use Future::Mutex;

has $_mutex;

    $_mutex = Future::Mutex->new;

The BUILD block is basic syntax, similar to Perl's own BEGIN block for instance. People familiar with object systems like Moo and Moose especially should take note - a BUILD block is not a method. It does not take the sub or method keyword, and it cannot be called like one.

Whenever a new instance is invoked BUILD block is passed a copy of the argument list given to the constructor. A common task is to set slot variables from those, or perhaps applying defaults if values weren't specified. It is also a common style in Perl for constructor arguments to passed in an even-sized key/value list, so they can be easily unpacked as a hash variable. This makes it simple for BUILD blocks to inspect the named keys they're interested in. Despite not being a true method, a BUILD block still permits a signature to unpack its arguments as if it were one.

class Device::Chip::CC1101 extends Device::Chip;

has $_fosc;
has $_poll_interval;

BUILD (%opts)
    $_fosc          = $opts{fosc} // 26E6;
    $_poll_interval = $opts{poll_interval} // 0.05;

There is still much ongoing design work here. It turns out in practice that a large majority of the code in BUILD blocks is something like this form - a series of lines, each setting a slot variable from one constructor argument.

There may be value in having Object::Pad provide a convenient way to let each slot variable declaration specify how it should be initialised from name constructor arguments. This would help keep the code less cluttered by the low-level machinery, and allow additional features such as error checking by rejecting unrecognised key names. This would, however, involve Object::Pad specifying that constructor arguments must be in named argument pairs, which it currently does not.

<< First | < Prev | Next >


2020 Perl Advent Calendar - Day 19

<< First | < Prev | Next >

We have already discussed that the most fundamental property of an object-oriented programming is the idea that a collection of state can be encapsulated into a single piece, and given behaviours that operate on the state. In yesterday's article we saw how to create new classes of object (with the class keyword), and how to add behaviours (with the method keyword). Today we'll take a closer look at the other half of this - how to add state.

While the word "method" seems to be fairly well entrenched, various object systems across various languages have a variety of different words to describe the state values stored for each given instance. The word "field" has been used in Perl before, and refers specifically to the now-obsolete fields pragma. Sometimes programmers refer to "attributes" of an object, but in Perl this is also an overloaded term referring to the :named annotations that can be applied to functions or variables. In Object::Pad the per-instance state is stored in variables called "slots".

Within a class, slots are created by the has keyword. This looks and feels similar to the my and our keywords. It introduces a new variable, optionally initialised with the value of an expression. Whereas a my or our variable is visible to all subsequent code (including nested functions) within its scope, a has variable is only visible within functions declared as method, because it will be associated with individual instances of the object class.

In this example the slot variables storing the label and click behaviour are available within any method:

class Tickit::Widget::Button extends Tickit::Widget;

has $_label;
has $_on_click;

method label { return $_label; }

method set_label
    ( $_label) = @_;

method on_click { return $_on_click; }

method click

In terms of visibility these slot variables behave much like other kinds of lexical variable - namely, they are not visible from outside the source of this particular class. This means that by default any such state variables are private to the class's implementation, inaccessible by other code that uses the class. We can choose to expose certain parts of it via the class's interface by providing these accessor methods, but we are not required to do so.

It is a common style in Object::Pad-based code to name the slot variables with a leading underscore, as in this example, as it helps them to stand out visually in larger code. It helps remind people that these are slot variables, because they now lack other visual signalling (such as $self->{...}) to otherwise distinguish them.

Another common behaviour is creating simple accessor methods to simply return the value of a slot, thus deciding to expose that particular variable as part of the object's interface, visible to callers. So common in fact that Object::Pad provides a shortcut to create these accessor methods automatically:

class Device::Chip::SSD1306 extends Device::Chip;

has $_rows :reader;
has $_columns :reader;

# now the class has ->rows and ->columns methods visible

The :reader attribute requests that a simple accessor method is created to return the current value of the slot. It is named the same as the slot, with a leading underscore first removed to account for the common naming convention.

One key advantage that these variable-like slots have over classical Perl objects built on hash keys or data provided by accessor methods is that the names are scoped within just the class body that defines them. Names cannot collide with those defined by subclasses. This is even checked by one of Object::Pad's own unit tests, which defines a base class and a subclass from it that both have a slot called $data:

class Base::Class {
    has $data;
    method data { $data }
class Derived::Class extends Base::Class {
    has $data;
    method data { $data }

It then has some tests to check that each of these methods behaves differently. In particular, this provides the guarantee that classes can freely add, delete, or rename their own slot variables without risking breaking other related classes. This leads to more robust class definitions.

<< First | < Prev | Next >