Event-Reflexive Programming

I am relatively certain that nobody reading this article for the first time will know what "event-reflexive" means, because it's a term I just made up. Or rather, I made up about a year ago while thinking over a bunch of related ideas that I felt needed a name. I'm hoping I didn't invent the basic concept because the reason for this post is to ask people a bunch of questions about it (as well as get the idea out of my head and onto at least some virtual paper).

Event-reflexive programming is a technique to manage the collection of a number of loosely-connected and perhaps interchangeable pieces into a larger coherent form, to work together to solve some common task. It is perhaps most illustrative to explain this story in roughly chronological order.

My first ever post-University job (and as it happens, the one where I learned Perl, and picked up such controversial habits as three-space indent) was centred around a large Internet Service Provider back-end provisioning system. In this system we had a large number of plugins to provide hooks into various components that made up a user's account. This was all hung together, at its core, by basically a big foreach loop looking something like

sub run_plugins {
  my ( $action, @args ) = @_;

  foreach my $plugin ( @plugins ) {
    $plugin->$action( @args ) if $plugin->can( $action );
  }
}

At its very basic level, this was all there was to it. Plugins are just classes, each can hook into the processing of some named action merely by having a method of its name. The list of plugins was determined by a config file, one per virtual reseller the ISP hosted users for. This made for a very powerful yet very simple way to customise and vary the capabilities per reseller - simply provide different plugin modules for them.

I should perhaps at this point explain the meaning of the word "event". An event here could mean a reaction to some external stimulus - the toplevel events used in this ISP provisioning system were such requests as "create a user" or "change this user's password". However, most large actions were really broken down into a sequence of smaller steps that themselves were run through the plugin system. A simple action like "change the user's password" is likely to be handled directly by each plugin module that cares, but an action as large as creating a user is probably broken down into several stages - create the initial user config in a database, mkdir their home directory, fill it with skeleton files, and so on.

It is at this point that the first of the awkward architectural questions of this approach comes to light. We started off with a list of the plugins stored in an array, and as we know, arrays are ordered containers. This approach necessarily imparts an ordering onto the plugins. The order each plugin reacts in this ISP system is determined by the order the names of the plugins appear in a configuration file. This allows us a certain degree of control, but also brings up the first large problem - it is the plugins that have order applied to them, but sometimes it is the individual action hooks that you wish to order.

For example; the action to create a new user has the stages of creating user's config in a database, then mkdiring their home directory, filling it with skeleton files, and so on. The ordering of these steps is fixed - we can't fill skeleton files in their home until it exists, and we can't create the directory until we have the basic user config in the database to look up where their home is. So naturally this leads to a good ordering relationship between the plugins that are responsible for these steps.

But now lets consider what happens when we want to delete a user again. By this plugin ordering we'd first remove their config from the database, before we rmdir their home, but if we did it that way we'd forget where their home was. Clearly, the solution here is that some events (such as deleting a user) have to be executed in reverse. This leads fairly quickly to the invention of a second control function:

sub run_plugins_reverse {
  my ( $action, @args ) = @_;

  foreach my $plugin ( reverse @plugins ) {
    $plugin->$action( @args ) if $plugin->can( $action );
  }
}

Some actions have more interesting ordering constraints even than this, however. The most complex action we had was the action used to regrade a user out of one product and into a different one. This basically involved a partial tear-down of the user, followed by some shuffling around of data, then a recreation again to add back the pieces that were removed, now in a different product. There is no simple way to express this simply in terms of plugin ordering, as a lot of plugins are needed both near the beginning and again near the end of the process.

The solution to this was to re-invoke the plugin action system again as part of handling a particular action. By decomposing this large action into three smaller steps, the plugin system's action ordering abilities help us ensure things all run correctly.

sub regrade_user {
  my %args = @_;

  run_plugins_reverse teardown_user_for_regrade => (
    user => $args{user},
    product => $args{old_product},
    ...
  );

  run_plugins regrade_user_inner => (
    user => $args{user},
    ...
  );

  run_plugins setup_user_for_regrade => (
    user => $args{user},
    product => $args{new_product},
    ...
  );
}

This toplevel action has virtually nothing to do - the above code is a slight over-simplification of the real implementation of course, but in principle all it contained was three recursive calls back down to the plugin management system to execute the three main stages of regrading a user. This is where the "reflexive" part of the name of the technique comes from - a lot of its power derives from the way that events are broken down into smaller pieces reflected back through the core of the system a second time, or further, to react to the whole thing as a large nested tree of individual steps.

I didn't remain at this job for long enough to properly see this idea through to its logical conclusion, but had I continued further on this route, I may well have decided to break these actions themselves into smaller chunks with small wrapper functions like the one given above, to split stages of other actions - for example, to split the stages of creating a user in the first place into the configuration stage, the home directory stage, and the provisioning of 3rd-party services stage. Continuing to break code down into smaller chunks like this leads to the idea that actually, the order in which the individual plugins run starts not to matter, because the order the steps are performed in is controlled by the way it is reflexively decomposed.

This now leads me on to ask the first of several questions I still have about this as a general programming technique:

Is it ever required to actually have strict ordering control of plugins in an event-reflexive system, or can any ordering problem always be resolved by breaking down actions into ordered requests to perform sequences of sub-actions? Even if so, is it ever desirable to have such an ordering control anyway; does it make any kinds of system neater or easier to create, at the cost of having to manage that order?

Next >

---

Edit 2014/06/09: I am leaning to the conclusion now, that if any kind of ordering were to be guaranteed, it starts to defeat the entire purpose of using event-reflexivity in the first place. The basic premise is to build a large system by loosely glueing together smaller components which are largely independent. If you have to think about which other components exist and what order they come in relative to the current one, it makes it harder to write that, and you might as well have written it by some other method. I think therefore, the answer to this question ought to be:

No; the base of an event-reflexive system should not attempt to give any kind of ordering guarantees between plugins. Any strict ordering that is required should be constructed out of explicit inter-component reflexive calls, and not by relying on the implicit side-effect of plugin execution order.