arrow communication direction display

When I first start­ed writ­ing Perl in my ear­ly 20’s, I tend­ed to fol­low a lot of the struc­tured pro­gram­ming con­ven­tions I had learned in school through Pascal, espe­cial­ly the notion that every func­tion has a sin­gle point of exit. For example:

sub double_even_number {
    # not using signatures, this is mid-1990's code
    my $number = shift;

    if (not $number % 2) {
        $number *= 2;
    }

    return $number; 
}

This could get pret­ty con­vo­lut­ed, espe­cial­ly if I was doing some­thing like val­i­dat­ing mul­ti­ple argu­ments. And at the time I didn’t yet grok how to han­dle excep­tions with eval and die, so I’d end up with code like:

sub print_postal_address {
    # too many arguments, I know
    my ($name, $street1, $street2, $city, $state, $zip) = @_;
    # also this notion of addresses is naive and US-centric

    my $error;

    if (!$name) {
        $error = 'no name';
    }
    else {
        print "$name\n";

        if (!$street1) {
            $error = 'no street';
        }
        else {
            print "$street1\n";

            if ($street2) {
                print "$street2\n";
            }

            if (!$city) {
                $error = 'no city';
            }
            else {
                print "$city, ";

                if (!$state) {
                    $error = 'no state';
                }
                else {
                    print "$state ";

                    if (!$zip) {
                        $error = 'no ZIP code';
                    }
                    else {
                        print "$zip\n";
                    }
                }
            }
        }
    }

    return $error;
}

What a mess. Want to count all those braces to make sure they’re bal­anced? This is some­times called the arrow anti-​pattern, with the arrowhead(s) being the most nest­ed state­ment. The default ProhibitDeepNests perlcritic pol­i­cy is meant to keep you from doing that.

The way out (lit­er­al­ly) is guard claus­es: check­ing ear­ly if some­thing is valid and bail­ing out quick­ly if not. The above exam­ple could be written:

sub print_postal_address {
    my ($name, $street1, $street2, $city, $state, $zip) = @_;

    if (!$name) {
        return 'no name';
    }
    if (!$street1) {
        return 'no street1';
    }
    if (!$city) {
        return 'no city';
    }
    if (!$state) {
        return 'no state';
    }
    if (!$zip) {
        return 'no zip';
    }

    print join "\n",
      $name,
      $street1,
      $street2 ? $street2 : (),
      "$city, $state $zip\n";

    return;
}

With Perl’s state­ment mod­i­fiers (some­times called post­fix con­trols) we can do even better:

    ...

    return 'no name'    if !$name;
    return 'no street1' if !$street1;
    return 'no city'    if !$city;
    return 'no state'   if !$state;
    return 'no zip'     if !$zip;

    ...

(Why if instead of unless? Because the lat­ter can be con­fus­ing with double-​negatives.)

Guard claus­es aren’t lim­it­ed to the begin­nings of func­tions or even exit­ing func­tions entire­ly. Often you’ll want to skip or even exit ear­ly con­di­tions in a loop, like this exam­ple that process­es files from stan­dard input or the com­mand line:

while (<>) {
    next if /^SKIP THIS LINE: /;
    last if /^END THINGS HERE$/;

    ...
}

Of course, if you are val­i­dat­ing func­tion argu­ments, you should con­sid­er using actu­al sub­rou­tine sig­na­tures if you have a Perl new­er than v5.20 (released in 2014), or one of the oth­er type val­i­da­tion solu­tions if not. Today I would write that postal func­tion like this, using Type::Params for val­i­da­tion and named arguments:

use feature qw(say state); 
use Types::Standard 'Str';
use Type::Params 'compile_named';

sub print_postal_address {
    state $check = compile_named(
        name    => Str,
        street1 => Str,
        street2 => Str, {optional => 1},
        city    => Str,
        state   => Str,
        zip     => Str,
    );
    my $arg = $check->(@_);

    say join "\n",
      $arg->{name},
      $arg->{street1},
      $arg->{street2} ? $arg->{street2} : (),
      "$arg->{city}, $arg->{state} $arg->{zip}";

    return;
}

print_postal_address(
    name    => 'J. Random Hacker',
    street1 => '123 Any Street',
    city    => 'Somewhereville',
    state   => 'TX',
    zip     => 12345,
);

Note that was this part of a larg­er pro­gram, I’d wrap that print_postal_address call in a try block and catch excep­tions such as those thrown by the code ref­er­ence $check gen­er­at­ed by compile_named. This high­lights one con­cern of guard claus­es and oth­er return ear­ly” pat­terns: depend­ing on how much has already occurred in your pro­gram, you may have to per­form some resource cleanup either in a catch block or some­thing like Syntax::Keyword::Try’s finally block if you need to tidy up after both suc­cess and failure.

Friday, December 17, 2021, marked the thirty-​fourth birth­day of the Perl pro­gram­ming lan­guage, and coin­ci­den­tal­ly this year saw the release of ver­sion 5.34. There are plen­ty of Perl devel­op­ers out there who haven’t kept up with recent (and not-​so-​recent) improve­ments to the lan­guage and its ecosys­tem, so I thought I might list a batch. (You may have seen some of these before in May’s post Perl can do that now!”)

The feature pragma

Perl v5.10 was released in December 2007, and with it came feature, a way of enabling new syn­tax with­out break­ing back­ward com­pat­i­bil­i­ty. You can enable indi­vid­ual fea­tures by name (e.g., use feature qw(say fc); for the say and fc key­words), or by using a fea­ture bun­dle based on the Perl ver­sion that intro­duced them. For exam­ple, the following:

use feature ':5.34';

…gives you the equiv­a­lent of:

use feature qw(bareword_filehandles bitwise current_sub evalbytes fc indirect multidimensional postderef_qq say state switch unicode_eval unicode_strings);

Boy, that’s a mouth­ful. Feature bun­dles are good. The cor­re­spond­ing bun­dle also gets implic­it­ly loaded if you spec­i­fy a min­i­mum required Perl ver­sion, e.g., with use v5.32;. If you use v5.12; or high­er, strict mode is enabled for free. So just say:

use v5.34;

And last­ly, one-​liners can use the -E switch instead of -e to enable all fea­tures for that ver­sion of Perl, so you can say the fol­low­ing on the com­mand line:

perl -E 'say "Hello world!"'

Instead of:

perl -e 'print "Hello world!\n"'

Which is great when you’re try­ing to save some typing.

The experimental pragma

Sometimes new Perl fea­tures need to be dri­ven a cou­ple of releas­es around the block before their behav­ior set­tles. Those exper­i­ments are doc­u­ment­ed in the per­l­ex­per­i­ment page, and usu­al­ly, you need both a use feature (see above) and no warnings state­ment to safe­ly enable them. Or you can sim­ply pass a list to use experimental of the fea­tures you want, e.g.:

use experimental qw(isa postderef signatures);

Ever-​expanding warnings categories

March 2000 saw the release of Perl 5.6, and with it, the expan­sion of the -w command-​line switch to a sys­tem of fine-​grained con­trols for warn­ing against dubi­ous con­structs” that can be turned on and off depend­ing on the lex­i­cal scope. What start­ed as 26 main and 20 sub­cat­e­gories has expand­ed into 31 main and 43 sub­cat­e­gories, includ­ing warn­ings for the afore­men­tioned exper­i­men­tal features.

As the rel­e­vant Perl::Critic pol­i­cy says, Using warn­ings, and pay­ing atten­tion to what they say, is prob­a­bly the sin­gle most effec­tive way to improve the qual­i­ty of your code.” If you must vio­late warn­ings (per­haps because you’re reha­bil­i­tat­ing some lega­cy code), you can iso­late such vio­la­tions to a small scope and indi­vid­ual cat­e­gories. Check out the stric­tures mod­ule on CPAN if you’d like to go fur­ther and make a safe sub­set of these cat­e­gories fatal dur­ing development.

Document other recently-​introduced syntax with Syntax::Construct

Not every new bit of Perl syn­tax is enabled with a feature guard. For the rest, there’s E. Choroba’s Syntax::Construct mod­ule on CPAN. Rather than hav­ing to remem­ber which ver­sion of Perl intro­duced what, Syntax::Construct lets you declare only what you use and pro­vides a help­ful error mes­sage if some­one tries to run your code on an old­er unsup­port­ed ver­sion. Between it and the feature prag­ma, you can pre­vent many head-​scratching moments and give your users a chance to either upgrade or workaround.

Make built-​in functions throw exceptions with autodie

Many of Perl’s built-​in func­tions only return false on fail­ure, requir­ing the devel­op­er to check every time whether a file can be opened or a system com­mand exe­cut­ed. The lex­i­cal autodie prag­ma replaces them with ver­sions that raise an excep­tion with an object that can be inter­ro­gat­ed for fur­ther details. No mat­ter how many func­tions or meth­ods deep a prob­lem occurs, you can choose to catch it and respond appro­pri­ate­ly. This leads us to…

try/​catch exception handling and Feature::Compat::Try

This year’s Perl v5.34 release intro­duced exper­i­men­tal try/​catch syn­tax for excep­tion han­dling that should look more famil­iar to users of oth­er lan­guages while han­dling the issues sur­round­ing using block eval and test­ing of the spe­cial [email protected] vari­able. If you need to remain com­pat­i­ble with old­er ver­sions of Perl (back to v5.14), just use the Feature::Compat::Try mod­ule from CPAN to auto­mat­i­cal­ly select either v5.34’s native try/​catch or a sub­set of the func­tion­al­i­ty pro­vid­ed by Syntax::Keyword::Try.

Pluggable keywords

The above­men­tioned Syntax::Keyword::Try was made pos­si­ble by the intro­duc­tion of a plug­gable key­word mech­a­nism in 2010’s Perl v5.12. So was the Future::AsyncAwait asyn­chro­nous pro­gram­ming library and the Object::Pad test­bed for new object-​oriented Perl syn­tax. If you’re handy with C and Perl’s XS glue lan­guage, check out Paul LeoNerd” Evans’ XS::Parse::Keyword mod­ule to get a leg up on devel­op­ing your own syn­tax module.

Define packages with versions and blocks

Perl v5.12 also helped reduce clut­ter by enabling a package name­space dec­la­ra­tion to also include a ver­sion num­ber, instead of requir­ing a sep­a­rate our $VERSION = ...; v5.14 fur­ther refined packages to be spec­i­fied in code blocks, so a name­space dec­la­ra­tion can be the same as a lex­i­cal scope. Putting the two togeth­er gives you:

package Local::NewHotness v1.2.3 {
    ...
}

Instead of:

{
    package Local::OldAndBusted;
    use version 0.77; our $VERSION = version->declare("v1.2.3");
    ...
}

I know which I’d rather do. (Though you may want to also use Syntax::Construct qw(package-version package-block); to help along with old­er instal­la­tions as described above.)

The // defined-​or operator

This is an easy win from Perl v5.10:

defined $foo ? $foo : $bar  # replace this
$foo // $bar                # with this

And:

$foo = $bar unless defined $foo  # replace this
$foo //= $bar                    # with this

Perfect for assign­ing defaults to variables.

state variables only initialize once

Speaking of vari­ables, ever want one to keep its old val­ue the next time a scope is entered, like in a sub? Declare it with state instead of my. Before Perl v5.10, you need­ed to use a clo­sure instead.

Save some typing with say

Perl v5.10’s bumper crop of enhance­ments also includ­ed the say func­tion, which han­dles the com­mon use case of printing a string or list of strings with a new­line. It’s less noise in your code and saves you four char­ac­ters. What’s not to love?

Note unimplemented code with ...

The ... ellip­sis state­ment (col­lo­qui­al­ly yada-​yada”) gives you an easy place­hold­er for yet-​to-​be-​implemented code. It pars­es OK but will throw an excep­tion if exe­cut­ed. Hopefully, your test cov­er­age (or at least sta­t­ic analy­sis) will catch it before your users do.

Loop and enumerate arrays with each, keys, and values

The each, keys, and values func­tions have always been able to oper­ate on hash­es. Perl v5.12 and above make them work on arrays, too. The lat­ter two are main­ly for con­sis­ten­cy, but you can use each to iter­ate over an array’s indices and val­ues at the same time:

while (my ($index, $value) = each @array) {
    ...
}

This can be prob­lem­at­ic in non-​trivial loops, but I’ve found it help­ful in quick scripts and one-liners.

delete local hash (and array) entries

Ever need­ed to delete an entry from a hash (e.g, an envi­ron­ment vari­able from %ENV or a sig­nal han­dler from %SIG) just inside a block? Perl v5.12 lets you do that with delete local.

Paired hash slices

Jumping for­ward to 2014’s Perl v5.20, the new %foo{'bar', 'baz'} syn­tax enables you to slice a sub­set of a hash with its keys and val­ues intact. Very help­ful for cherry-​picking or aggre­gat­ing many hash­es into one. For example:

my %args = (
    verbose => 1,
    name    => 'Mark',
    extra   => 'pizza',
);
# don't frob the pizza
$my_object->frob( %args{ qw(verbose name) };

Paired array slices

Not to be left out, you can also slice arrays in the same way, in this case return­ing indices and values:

my @letters = 'a' .. 'z';
my @subset_kv = %letters[16, 5, 18, 12];
# @subset_kv is now (16, 'p', 5, 'e', 18, 'r', 12, 'l')

More readable dereferencing

Perl v5.20 intro­duced and v5.24 de-​experimentalized a more read­able post­fix deref­er­enc­ing syn­tax for nav­i­gat­ing nest­ed data struc­tures. Instead of using {braces} or smoosh­ing sig­ils to the left of iden­ti­fiers, you can use a post­fixed sigil-and-star:

push @$array_ref,    1, 2, 3;  # noisy
push @{$array_ref},  1, 2, 3;  # a little easier
push $array_ref->@*, 1, 2, 3;  # read from left to right

So much of web devel­op­ment is sling­ing around and pick­ing apart com­pli­cat­ed data struc­tures via JSON, so I wel­come any­thing like this to reduce the cog­ni­tive load.

when as a statement modifier

Starting in Perl v5.12, you can use the exper­i­men­tal switch fea­tures when key­word as a post­fix mod­i­fi­er. For example:

for ($foo) {
    $a =  1 when /^abc/;
    $a = 42 when /^dna/;
    ...
}

But I don’t rec­om­mend when, given, or givens smart­match oper­a­tions as they were ret­conned as exper­i­ments in 2013’s Perl v5.18 and have remained so due to their tricky behav­ior. I wrote about some alter­na­tives using sta­ble syn­tax back in February.

Simple class inheritance with use parent

Sometimes in old­er object-​oriented Perl code, you’ll see use base as a prag­ma to estab­lish inher­i­tance from anoth­er class. Older still is the direct manip­u­la­tion of the package’s spe­cial @ISA array. In most cas­es, both should be avoid­ed in favor of use parent, which was added to core in Perl v5.10.1.

Mind you, if you’re fol­low­ing the Perl object-​oriented tutorial’s advice and have select­ed an OO sys­tem from CPAN, use its sub­class­ing mech­a­nism if it has one. Moose, Moo, and Class::Accessor’s antlers” mode all pro­vide an extends func­tion; Object::Pad pro­vides an :isa attribute on its class key­word.

Test for class membership with the isa operator

As an alter­na­tive to the isa() method pro­vid­ed to all Perl objects, Perl v5.32 intro­duced the exper­i­men­tal isa infix oper­a­tor:

$my_object->isa('Local::MyClass')
# or
$my_object isa Local::MyClass

The lat­ter can take either a bare­word class name or string expres­sion, but more impor­tant­ly, it’s safer as it also returns false if the left argu­ment is unde­fined or isn’t a blessed object ref­er­ence. The old­er isa() method will throw an excep­tion in the for­mer case and might return true if called as a class method when $my_object is actu­al­ly a string of a class name that’s the same as or inher­its from isa()s argu­ment.

Lexical subroutines

Introduced in Perl v5.18 and de-​experimentalized in 2017’s Perl v5.26, you can now pre­cede sub dec­la­ra­tions with my, state, or our. One use of the first two is tru­ly pri­vate func­tions and meth­ods, as described in this 2018 Dave Jacoby blog and as part of Neil Bowers’ 2014 sur­vey of pri­vate func­tion techniques.

Subroutine signatures

I’ve writ­ten and pre­sent­ed exten­sive­ly about sig­na­tures and alter­na­tives over the past year, so I won’t repeat that here. I’ll just add that the Perl 5 Porters devel­op­ment mail­ing list has been mak­ing a con­cert­ed effort over the past month to hash out the remain­ing issues towards ren­der­ing this fea­ture non-​experimental. The pop­u­lar Mojolicious real-​time web frame­work also pro­vides a short­cut for enabling sig­na­tures and uses them exten­sive­ly in examples.

Indented here-​documents with <<~

Perl has had shell-​style here-​document” syn­tax for embed­ding multi-​line strings of quot­ed text for a long time. Starting with Perl v5.26, you can pre­cede the delim­it­ing string with a ~ char­ac­ter and Perl will both allow the end­ing delim­iter to be indent­ed as well as strip inden­ta­tion from the embed­ded text. This allows for much more read­able embed­ded code such as runs of HTML and SQL. For example:

if ($do_query) {
    my $rows_deleted = $dbh->do(<<~'END_SQL', undef, 42);
      DELETE FROM table
      WHERE status = ?
      END_SQL
    say "$rows_deleted rows were deleted."; 
}

More readable chained comparisons

When I learned math in school, my teach­ers and text­books would often describe mul­ti­ple com­par­isons and inequal­i­ties as a sin­gle expres­sion. Unfortunately, when it came time to learn pro­gram­ming every com­put­er lan­guage I saw required them to be bro­ken up with a series of and (or &&) oper­a­tors. With Perl v5.32, this is no more:

if ( $x < $y && $y <= $z ) { ... }  # old way
if ( $x < $y <= $z )       { ... }  # new way

It’s more con­cise, less noisy, and more like what reg­u­lar math looks like.

Self-​documenting named regular expression captures

Perl’s expres­sive reg­u­lar expres­sion match­ing and text-​processing prowess are leg­endary, although overuse and poor use of read­abil­i­ty enhance­ments often turn peo­ple away from them (and Perl in gen­er­al). We often use reg­ex­ps for extract­ing data from a matched pat­tern. For example:

if ( /Time: (..):(..):(..)/ ) {  # parse out values
    say "$1 hours, $2 minutes, $3 seconds";
}

Named cap­ture groups, intro­duced in Perl v5.10, make both the pat­tern more obvi­ous and retrieval of its data less cryptic:

if ( /Time: (?<hours>..):(?<minutes>..):(?<seconds>..)/ ) {
    say "$+{hours} hours, $+{minutes} minutes, $+{seconds} seconds";
}

More readable regexp character classes

The /x reg­u­lar expres­sion mod­i­fi­er already enables bet­ter read­abil­i­ty by telling the pars­er to ignore most white­space, allow­ing you to break up com­pli­cat­ed pat­terns into spaced-​out groups and mul­ti­ple lines with code com­ments. With Perl v5.26 you can spec­i­fy /xx to also ignore spaces and tabs inside [brack­et­ed] char­ac­ter class­es, turn­ing this:

/[d-eg-i3-7]/
/[[email protected]"#$%^&*()=?<>']/

…into this:

/ [d-e g-i 3-7]/xx
/[ ! @ " # $ % ^ & * () = ? <> ' ]/xx

Set default regexp flags with the re pragma

Beginning with Perl v5.14, writ­ing use re '/xms'; (or any com­bi­na­tion of reg­u­lar expres­sion mod­i­fi­er flags) will turn on those flags until the end of that lex­i­cal scope, sav­ing you the trou­ble of remem­ber­ing them every time.

Non-​destructive substitution with s///r and tr///r

The s/// sub­sti­tu­tion and tr/// translit­er­a­tion oper­a­tors typ­i­cal­ly change their input direct­ly, often in con­junc­tion with the =~ bind­ing oper­a­tor:

s/foo/bar/;  # changes the first foo to bar in $_
$baz =~ s/foo/bar/;  # the same but in $baz

But what if you want to leave the orig­i­nal untouched, such as when pro­cess­ing an array of strings with a map? With Perl v5.14 and above, add the /r flag, which makes the sub­sti­tu­tion on a copy and returns the result:

my @changed = map { s/foo/bar/r } @original;

Unicode case-​folding with fc for better string comparisons

Unicode and char­ac­ter encod­ing in gen­er­al are com­pli­cat­ed beasts. Perl has han­dled Unicode since v5.6 and has kept pace with fix­es and sup­port for updat­ed stan­dards in the inter­ven­ing decades. If you need to test if two strings are equal regard­less of case, use the fc func­tion intro­duced in Perl v5.16.

Safer processing of file arguments with <<>>

The <> null file­han­dle or dia­mond oper­a­tor” is often used in while loops to process input per line com­ing either from stan­dard input (e.g., piped from anoth­er pro­gram) or from a list of files on the com­mand line. Unfortunately, it uses a form of Perl’s open func­tion that inter­prets spe­cial char­ac­ters such as pipes (|) that would allow it to inse­cure­ly run exter­nal com­mands. Using the <<>> dou­ble dia­mond” oper­a­tor intro­duced in Perl v5.22 forces open to treat all command-​line argu­ments as file names only. For old­er Perls, the per­lop doc­u­men­ta­tion rec­om­mends the ARGV::readonly CPAN mod­ule.

Safer loading of Perl libraries and modules from @INC

Perl v5.26 removed the abil­i­ty for all pro­grams to load mod­ules by default from the cur­rent direc­to­ry, clos­ing a secu­ri­ty vul­ner­a­bil­i­ty orig­i­nal­ly iden­ti­fied and fixed as CVE-​2016 – 1238 in pre­vi­ous ver­sions’ includ­ed scripts. If your code relied on this unsafe behav­ior, the v5.26 release notes include steps on how to adapt.

HTTP::Tiny simple HTTP/1.1 client included

To boot­strap access to CPAN on the web in the pos­si­ble absence of exter­nal tools like curl or wget, Perl v5.14 began includ­ing the HTTP::Tiny mod­ule. You can also use it in your pro­grams if you need a sim­ple web client with no dependencies.

Test2: The next generation of Perl testing frameworks

Forked and refac­tored from the ven­er­a­ble Test::Builder (the basis for the Test::More library that many are famil­iar with), Test2 was includ­ed in the core mod­ule library begin­ning with Perl v5.26. I’ve exper­i­ment­ed recent­ly with using the Test2::Suite CPAN library instead of Test::More and it looks pret­ty good. I’m also intrigued by Test2::Harness’ sup­port for thread­ing, fork­ing, and pre­load­ing mod­ules to reduce test run times.

Task::Kensho: Where to start for recommended Perl modules

This last item may not be includ­ed when you install Perl, but it’s where I turn for a col­lec­tion of well-​regarded CPAN mod­ules for accom­plish­ing a wide vari­ety of com­mon tasks span­ning from asyn­chro­nous pro­gram­ming to XML. Use it as a start­ing point or inter­ac­tive­ly select the mix of libraries appro­pri­ate to your project.


And there you have it: a selec­tion of 34 fea­tures, enhance­ments, and improve­ments for the first 34 years of Perl. What’s your favorite? Did I miss any­thing? Let me know in the comments.

chocolate bar and sugar cubes on a hand
What about My::Favorite::Module?

I men­tioned at the Ephemeral Miniconf last month that as soon as I write about one Perl mod­ule (or five), some­one inevitably brings up anoth­er (or sev­en) I’ve missed. And of course, it hap­pened again last week: no soon­er had I writ­ten in pass­ing that I was using Exception::Class than the denizens of the Libera Chat IRC #perl chan­nel insist­ed I should use Throwable instead for defin­ing my excep­tions. (I’ve already blogged about var­i­ous ways of catch­ing excep­tions.)

Why Throwable? Aside from Exception::Class’s author rec­om­mend­ing it over his own work due to a nicer, more mod­ern inter­face,” Throwable is a Moo role, so it’s com­pos­able into class­es along with oth­er roles instead of muck­ing about with mul­ti­ple inher­i­tance. This means that if your excep­tions need to do some­thing reusable in your appli­ca­tion like log­ging, you can also con­sume a role that does that and not have so much dupli­cate code. (No, I’m not going to pick a favorite log­ging mod­ule; I’ll prob­a­bly get that wrong too.)

However, since Throwable is a role instead of a class, I would have to define sev­er­al addi­tion­al packages in my tiny mod­uli­no script from last week, one for each excep­tion class I want. The beau­ty of Exception::Class is its sim­ple declar­a­tive nature: just use it and pass a list of desired class names along with options for attrib­ut­es and what­not. What’s need­ed for sim­ple use cas­es like mine is a declar­a­tive syn­tax for defin­ing sev­er­al excep­tion class­es with­out the noise of mul­ti­ple packages.

Enter Throwable::SugarFactory, a mod­ule that enables you to do just that by adding an exception func­tion for declar­ing excep­tion class­es. (There’s also the similarly-​named Throwable::Factory; see the above dis­cus­sion about nev­er being able to cov­er everybody’s favorites.) The exception func­tion takes three argu­ments: the name of the desired excep­tion class as a string, a descrip­tion, and an option­al list of instruc­tions Moo uses to build the class. It might look some­thing like this:

package Local::My::Exceptions;
use Throwable::SugarFactory;

exception GenericError  => 'something bad happened';
exception DetailedError => 'something specific happened' =>
  ( has => [ message => ( is => 'ro' ) ] );

1;

Throwable::SugarFactory takes care of cre­at­ing con­struc­tor func­tions in Perl-​style snake_case as well as func­tions for detect­ing what kind of excep­tion is being caught, so you can use your new excep­tion library like this:

#!/usr/bin/env perl

use experimental qw(isa);
use Feature::Compat::Try;
use JSON::MaybeXS;
use Local::My::Exceptions;

try {
    die generic_error();
}
catch ($e) {
    warn 'whoops!';
}

try {
    die detailed_error( message => 'you got me' );
}
catch ($e) {
    die encode_json( $e->to_hash )
      if $e isa DetailedError and defined $e->message;
    $e->throw if $e->does('Throwable');
    die $e;
}

The above also demon­strates a cou­ple of oth­er Throwable::SugarFactory fea­tures. First, you get a to_hash method that returns a hash ref­er­ence of all excep­tion data, suit­able for seri­al­iz­ing to JSON. Second, you get all of Throwable’s meth­ods, includ­ing throw for re-​throwing exceptions. 

So where does this leave last week’s FOAAS.com mod­uli­no client demon­stra­tion of object mock­ing tests? With a lit­tle bit of rewrit­ing to define and then use our sweet­er excep­tion library, it looks like this. You can review for a descrip­tion of the rest of its workings.

#!/usr/bin/env perl

package Local::CallFOAAS::Exceptions;
use Throwable::SugarFactory;

BEGIN {
    exception NoMethodError =>
      'no matching WebService::FOAAS method' =>
      ( has => [ method => ( is => 'ro' ) ] );
    exception ServiceError =>
      'error from WebService::FOAAS' =>
      ( has => [ message => ( is => 'ro' ) ] );
}

package Local::CallFOAAS;  # this is a modulino
use Test2::V0;             # enables strict, warnings, utf8

# declare all the new stuff we're using
use feature qw(say state);
use experimental qw(isa postderef signatures);
use Feature::Compat::Try;
use Syntax::Construct qw(non-destructive-substitution);

use WebService::FOAAS ();
use Package::Stash;
BEGIN { Local::CallFOAAS::Exceptions->import() }

my $foaas = Package::Stash->new('WebService::FOAAS');

my $run_as =
    !!$ENV{CPANTEST}       ? 'test'
  : !defined scalar caller ? 'run'
  :                          undef;
__PACKAGE__->$run_as(@ARGV) if defined $run_as;

sub run ( $class, @args ) {
    try { say $class->call_method(@args) }
    catch ($e) {
        die 'No method ', $e->method, "\n"
          if $e isa NoMethodError;
        die 'Service error: ', $e->message, "\n"
          if $e isa ServiceError;
        die "$e\n";
    }
    return;
}

# Utilities

sub methods ($) {
    state @methods = sort map s/^foaas_(.+)/$1/r,
      grep /^foaas_/, $foaas->list_all_symbols('CODE');
    return @methods;
}

sub call_method ( $class, $method = '', @args ) {
    state %methods = map { $_ => 1 } $class->methods();
    die no_method_error( method => $method )
      unless $methods{$method};
    return do {
        try { $foaas->get_symbol("&$method")->(@args) }
        catch ($e) { die service_error( message => $e ) }
    };
}

# Testing

sub test ( $class, @ ) {
    state $stash = Package::Stash->new($class);
    state @tests = sort grep /^_test_/,
      $stash->list_all_symbols('CODE');

    for my $test (@tests) {
        subtest $test => sub {
            try { $class->$test() }
            catch ($e) { diag $e }
        };
    }
    done_testing();
    return;
}

sub _test_can ($class) {
    state @subs = qw(run call_method methods test);
    can_ok $class, \@subs, "can do: @subs";
    return;
}

sub _test_methods ($class) {
    my $mock = mock 'WebService::FOAAS' => ( track => 1 );

    for my $method ( $class->methods() ) {
        $mock->override( $method => 1 );

        ok lives { $class->call_method($method) },
          "$method lives";
        ok scalar $mock->sub_tracking->{$method}->@*,
          "$method called";
    }
    return;
}

sub _test_service_failure ($class) {
    my $mock = mock 'WebService::FOAAS';

    for my $method ( $class->methods() ) {
        $mock->override( $method => sub { die 'mocked' } );

        my $exception =
          dies { $class->call_method($method) };
        isa_ok $exception, [ServiceError],
          "$method throws ServiceError on failure";
        like $exception->message, qr/^mocked/,
          "correct error in $method exception";
    }
    return;
}

1;

[Updated, thanks to Dan Book, Karen Etheridge, and Bob Kleemann] The only goofy bit above is the need to put the exception calls in a BEGIN block and then explic­it­ly call BEGIN { Local::CallFOAAS::Exceptions->import() }. Since the two pack­ages are in the same file, I can’t do a use state­ment since the implied require would look for a cor­re­spond­ing file or entry in %INC. (You can get around this by mess­ing with %INC direct­ly or through a mod­ule like me::inlined that does that mess­ing for you, but for a single-​purpose mod­uli­no like this it’s fine.)


happy man funny sticking tongue out

Over the past two years, I’ve got­ten back into play­ing Dungeons & Dragons, the famous table­top fan­ta­sy role-​playing game. As a soft­ware devel­op­er and musi­cian, one of my favorite char­ac­ter class­es to play is the bard, a mag­i­cal and inspir­ing per­former or word­smith. The list of basic bardic spells includes Vicious Mockery, enchant­i­ng ver­bal barbs that have the pow­er to psy­chi­cal­ly dam­age and dis­ad­van­tage an oppo­nent even if they don’t under­stand the words. (Can you see why this is so appeal­ing to a coder?)

Mocking has a role to play in soft­ware test­ing as well, in the form of mock objects that sim­u­late parts of a sys­tem that are too brit­tle, too slow, too com­pli­cat­ed, or oth­er­wise too finicky to use in real­i­ty. They enable dis­crete unit test­ing with­out rely­ing on depen­den­cies exter­nal to the code being test­ed. Mocks are great for data­bas­es, web ser­vices, or oth­er net­work resources where the goal is to test what you wrote, not what’s out in the cloud” somewhere.

Speaking of web ser­vices and mock­ing, one of my favorites is the long-​running FOAAS (link has lan­guage not safe for work), a sur­pris­ing­ly expan­sive RESTful insult ser­vice. There’s a cor­re­spond­ing Perl client API, of course, but what I was miss­ing was a handy Perl script to call that API from the ter­mi­nal com­mand line. So I wrote the fol­low­ing over Thanksgiving break, try­ing to keep it sim­ple while also show­ing the basics of mock­ing such an API. It also demon­strates some new­er Perl syn­tax and test­ing tech­niques as well as bri­an d foys mod­uli­no con­cept from Mastering Perl (sec­ond edi­tion, 2014) that mar­ries script and mod­ule into a self-​contained exe­cutable library.

#!/usr/bin/env perl

package Local::CallFOAAS;  # this is a modulino
use Test2::V0;             # enables strict, warnings, utf8

# declare all the new stuff we're using
use feature qw(say state);
use experimental qw(isa postderef signatures);
use Feature::Compat::Try;
use Syntax::Construct qw(non-destructive-substitution);

use WebService::FOAAS ();
use Package::Stash;
use Exception::Class (
    NoMethodException => {
        alias  => 'throw_no_method',
        fields => 'method',
    },
    ServiceException => { alias => 'throw_service' },
);

my $foaas = Package::Stash->new('WebService::FOAAS');

my $run_as =
    !!$ENV{CPANTEST}       ? 'test'
  : !defined scalar caller ? 'run'
  :                          undef;
__PACKAGE__->$run_as(@ARGV) if defined $run_as;

sub run ( $class, @args ) {
    try { say $class->call_method(@args) }
    catch ($e) {
        die 'No method ', $e->method, "\n"
          if $e isa NoMethodException;
        die 'Service error: ', $e->error, "\n"
          if $e isa ServiceException;
        die "$e\n";
    }
    return;
}

# Utilities

sub methods ($) {
    state @methods = sort map s/^foaas_(.+)/$1/r,
      grep /^foaas_/, $foaas->list_all_symbols('CODE');
    return @methods;
}

sub call_method ( $class, $method = '', @args ) {
    state %methods = map { $_ => 1 } $class->methods();
    throw_no_method( method => $method )
      unless $methods{$method};
    return do {
        try { $foaas->get_symbol("&$method")->(@args) }
        catch ($e) { throw_service( error => $e ) }
    };
}

# Testing

sub test ( $class, @ ) {
    state $stash = Package::Stash->new($class);
    state @tests = sort grep /^_test_/,
      $stash->list_all_symbols('CODE');

    for my $test (@tests) {
        subtest $test => sub {
            try { $class->$test() }
            catch ($e) { diag $e }
        };
    }
    done_testing();
    return;
}

sub _test_can ($class) {
    state @subs = qw(run call_method methods test);
    can_ok( $class, \@subs, "can do: @subs" );
    return;
}

sub _test_methods ($class) {
    my $mock = mock 'WebService::FOAAS' => ( track => 1 );

    for my $method ( $class->methods() ) {
        $mock->override( $method => 1 );

        ok lives { $class->call_method($method) },
          "$method lives";
        ok scalar $mock->sub_tracking->{$method}->@*,
          "$method called";
    }
    return;
}

sub _test_service_failure ($class) {
    my $mock = mock 'WebService::FOAAS';

    for my $method ( $class->methods() ) {
        $mock->override( $method => sub { die 'mocked' } );

        my $exception =
          dies { $class->call_method($method) };
        isa_ok $exception, ['ServiceException'],
          "$method throws ServiceException on failure";
        like $exception->error, qr/^mocked/,
          "correct error in $method exception";
    }
    return;
}

1;

Let’s walk through the code above.

Preliminaries

First, there’s a gener­ic she­bang line to indi­cate that Unix and Linux sys­tems should use the perl exe­cutable found in the user’s PATH via the env com­mand. I declare a pack­age name (in the Local:: name­space) so as not to pol­lute the default main pack­age of oth­er scripts that might want to require this as a mod­ule. Then I use the Test2::V0 bun­dle from Test2::Suite since the embed­ded test­ing code uses many of its func­tions. This also has the side effect of enabling the strict, warn­ings, and utf8 prag­mas, so there’s no need to explic­it­ly use them here.

(Why Test2 instead of Test::More and its deriv­a­tives and add-​ons? Both are main­tained by the same author, who rec­om­mends the for­mer. I’m see­ing more and more mod­ules using it, so I thought this would be a great oppor­tu­ni­ty to learn.)

I then declare all the new-​ish Perl fea­tures I’d like to use that need to be explic­it­ly enabled so as not to sac­ri­fice back­ward com­pat­i­bil­i­ty with old­er ver­sions of Perl 5. As of this writ­ing, some of these fea­tures (the isa class instance oper­a­tor, named argu­ment sub­rou­tine sig­na­tures, and try/​catch excep­tion han­dling syn­tax) are con­sid­ered experimental, with the lat­ter enabled in old­er ver­sions of Perl via the Feature::Compat::Try mod­ule. The friend­lier post­fix deref­er­enc­ing syn­tax was main­lined in Perl ver­sion 5.24, but ver­sions 5.20 and 5.22 still need it exper­i­men­tal. Finally, I use Syntax::Construct to announce the /r flag for non-​destructive reg­u­lar expres­sion text sub­sti­tu­tions intro­duced in ver­sion 5.14.

Next, I bring in the afore­men­tioned FOAAS Perl API with­out import­ing any of its func­tions, Package::Stash to make metapro­gram­ming eas­i­er, and a cou­ple of excep­tion class­es so that the com­mand line func­tion and oth­er con­sumers might bet­ter tell what caused a fail­ure. In prepa­ra­tion for the meth­ods below dynam­i­cal­ly dis­cov­er­ing what func­tions are pro­vid­ed by WebService::FOAAS, I gath­er up its sym­bol table (or stash) into the $foaas variable.

The next block deter­mines how, if at all, I’m going to run the code as a script. If the CPANTEST envi­ron­ment vari­able is set, I’ll call the test class method sub, but if there’s no sub­rou­tine call­ing me I’ll exe­cute the run class method. Either will receive the com­mand line argu­ments from @ARGV. If nei­ther of these con­di­tions is true, do noth­ing; the rest of the code is method declarations.

Modulino methods, metaprogramming, and exceptions

The first of these is the run method. It’s a thin wrap­per around the call_method class method detailed below, either out­putting its result or dieing with an appro­pri­ate error depend­ing on the class of excep­tion thrown. Although I chose not to write tests for this out­put, future tests might call this method and catch these rethrown excep­tions to match against them. The mes­sages end with a \n new­line char­ac­ter so die knows not to append the cur­rent script line number.

Next is a util­i­ty method called methods that uses Package::Stash’s list_all_symbols to retrieve the names of all named CODE blocks (i.e., subs) from WebService::FOAAS’s sym­bol table. Reading from right to left, these are then fil­tered with grep to only find those begin­ning in foaas_ and then trans­formed with map to remove that pre­fix. The list is then sorted and stored in a state vari­able and returned so it need not be ini­tial­ized again.

(As an aside, although perlcritic stern­ly warns against it I’ve cho­sen the expres­sion forms of grep and map here over their block forms for sim­plic­i­ty’s sake. It’s OK to bend the rules if you have a good reason.)

sub call_method is where the real action takes place. Its para­me­ters are the class that called it, the name of a FOAAS $method (default­ed to the emp­ty string), and an array of option­al argu­ments in @args. I build a hash or asso­cia­tive array from the ear­li­er methods method which I then use to see if the passed method name is one I know about. If not, I throw a NoMethodException using the throw_no_method alias func­tion cre­at­ed when I used Exception::Class at the begin­ning. Using a func­tion instead of NoMethodException->throw() means that it’s checked at com­pile time rather than run­time, catch­ing typos.

I get the sub­rou­tine (denot­ed by a & sig­il) named by $method from the $foaas stash and pass it any fur­ther received argu­ments from @args. If that WebService::FOAAS sub­rou­tine throws an excep­tion it’ll be caught and re-​thrown as a ServiceException; oth­er­wise call_method returns the result. It’s up to the caller to deter­mine what, if any­thing, to do with that result or any thrown exceptions.

Testing the modulino with mocks

This is where I start using those Test2::Suite tools I men­tioned at the begin­ning. The test class method starts by build­ing a fil­tered list of all subs begin­ning with _test_ in the cur­rent class, much like methods did above with WebService::FOAAS. I then loop through that list of subs, run­ning each as a subtest con­tain­ing a class method with any excep­tions report­ed as diag­nos­tics.

The rest of the mod­uli­no is sub­test meth­ods, start­ing with a sim­ple _test_can san­i­ty check for the pub­lic meth­ods in the class. Following that is _test_methods, which starts by mocking the WebService::FOAAS pack­age and telling Test2::Mock I want to track any added, over­rid­den, or set subs. I then loop through all the method names returned by the methods class method, overrideing each one to return a sim­ple true val­ue. I then test pass­ing those names to call_method and use the hash ref­er­ence returned by sub_tracking to check that the over­rid­den sub was called. This seems a lot sim­pler than the Test::Builder-based mock­ing libraries I’ve tried like Test::MockModule and Test::MockObject.

_test_service_failure acts in much the same way, check­ing that call_method cor­rect­ly throws ServiceExceptions if the wrapped WebService::FOAAS func­tion dies. The main dif­fer­ence is that the mocked WebService::FOAAS subs are now over­rid­den with a code ref­er­ence (sub { die 'mocked' }), which call_method uses to pop­u­late the rethrown ServiceExceptions error field.

Wrapping up

With luck, this arti­cle has giv­en you some ideas, whether it’s in mak­ing scripts (per­haps lega­cy code) testable to improve them, or writ­ing bet­ter unit tests that mock depen­den­cies, or delv­ing a lit­tle into metapro­gram­ming so you can dynam­i­cal­ly sup­port and test new fea­tures of said depen­den­cies. I hope you haven’t come away too offend­ed, at least. Let me know in the com­ments what you think.

I’m busy this week host­ing my par­ents’ first vis­it to Houston, but I didn’t want to let this Tuesday go by with­out link­ing to my talk from last week’s Ephemeral Miniconf. Thanks so much to Thibault Duponchelle for orga­niz­ing such a ter­rif­ic event, to all the oth­er speak­ers for com­ing togeth­er to present, and to every­one who attend­ed for wel­com­ing me.

circus theme party

Last week’s arti­cle got a great response on Hacker News, and this par­tic­u­lar com­ment caught my eye:

I think this is the real point about Perl code read­abil­i­ty: it gives you enough flex­i­bil­i­ty to do things how­ev­er you like, and as a result many pro­gram­mers are faced with a mir­ror that reflects their own bad prac­tices back at them.

orev, Hacker News

This is why Damian Conway’s Perl Best Practices (2005) is one of my favorite books and perlcritic, the code ana­lyz­er is one of my favorite tools. (Though the for­mer could do with an update and the lat­ter includes poli­cies that con­tra­dict Conway.) Point perlcritic at your code, maybe add some oth­er poli­cies that agree with your house style, and grad­u­al­ly ratch­et up the sever­i­ty lev­el from gen­tle” to bru­tal.” All kinds of bad juju will come to light, from waste­ful­ly using grep to hav­ing too many sub­rou­tine argu­ments to catch­ing pri­vate vari­able use from oth­er pack­ages. perlcritic offers a use­ful base­line of con­duct and you can always cus­tomize its con­fig­u­ra­tion to your own tastes.

The oth­er con­for­mance tool in a Perl devel­op­er’s belt is perltidy, and it too has a Conway-​compatible con­fig­u­ra­tion as well as its default Perl Style Guide set­tings. I’ve found that more than any­thing else, perltidy helps set­tle argu­ments both between devel­op­ers and between their code in help­ing to avoid exces­sive merge conflicts.

But apart from extra tools, Perl the lan­guage itself can be bent and even bro­ken to suit just about any­one’s agen­da. Those used to more bondage-​and-​discipline lan­guages (hi, Java!) might feel revul­sion at the lengths to which this has some­times been tak­en, but per the quote above this is less an indict­ment of the lan­guage and more of its less method­i­cal pro­gram­mers.

Some of this behav­ior can be reha­bil­i­tat­ed with perlcritic and perltidy, but what about oth­er sins attrib­uted to Perl? Here are a few peren­ni­al favorites”:

Objects and Object-​Oriented Programming

Perl has a min­i­mal­ist object sys­tem based on earlier-​available lan­guage con­cepts like data struc­tures (often hash­es, which it has in com­mon with JavaScript), pack­ages, and sub­rou­tines. Since Perl 5’s release in 1994 much ver­bose OO code has been writ­ten using these tools.

The good news is that since 2007 we’ve had a sophis­ti­cat­ed metaobject-​protocol-​based lay­er on top of them called Moose, since 2010 a light­weight but forward-​compatible sys­tem called Moo, and a cou­ple of even tinier options as described in the Perl OO Tutorial. Waiting in the wings is Corinna, an effort to bring next-​generation object capa­bil­i­ties into the Perl core itself, and Object::Pad, a test­bed for some of the ideas in Corinna that you can use today in cur­rent code. (Really, please try it — the author needs feed­back!)

All this is to say that 99% of the time you nev­er need trou­ble your­self with bless, con­struc­tors, or writ­ing acces­sors for class or object attrib­ut­es. Smarter peo­ple than me have done the work for you, and you might even find a con­cept or three that you wish oth­er lan­guages had.

Contexts

There are two major ones: list and scalar. Another way to think of it is plur­al” vs. sin­gu­lar” in English, which is hope­ful­ly a thing you’re famil­iar with as you’re read­ing this blog.

Some func­tions in Perl act dif­fer­ent­ly depend­ing on whether the expect­ed return val­ue is a list or a scalar, and a func­tion will pro­vide a list or scalar con­text to its argu­ments. Mostly these act just as you would expect or would like them to, and you can find out how a func­tion behaves by read­ing its doc­u­men­ta­tion. Your own func­tions can behave like this too, but there’s usu­al­ly no need as both scalars and lists are auto­mat­i­cal­ly inter­pret­ed into lists.” Again, Perl’s DWIMmery at work.

Subroutine and Method Arguments

I’ve already writ­ten about this. Twice. And pre­sent­ed about it. Twice. The short ver­sion: Perl has sig­na­tures, but they’ve been con­sid­ered exper­i­men­tal for a while. In the mean­time, there are alter­na­tives on CPAN. You can even have type con­straints if you want.


I’ll leave you with this: Over the past month, Neil Bowers of the Perl Steering Council has been col­lect­ing quirks like these from Perl devel­op­ers. The PSC is review­ing this col­lec­tion for poten­tial doc­u­men­ta­tion fix­es, bug fix­es, fur­ther dis­cus­sion, etc. I would­n’t expect to see any fun­da­men­tal changes to the lan­guage out of this effort, but it’s a good sign that poten­tial­ly con­fus­ing fea­tures are being addressed. 

brown wooden arrow signed

A mentee asked me over the week­end if there was a way with­in a Mojolicious web appli­ca­tion to store the routes sep­a­rate­ly from the main appli­ca­tion class. Here’s one way. These instruc­tions assume you’re using Perl 5.34 and Mojolicious 9.19 (the lat­est as of this writ­ing) via the ter­mi­nal com­mand line on a Linux, Unix, or macOS sys­tem; make the appro­pri­ate changes if this does­n’t apply to you.

First, if you haven’t already, cre­ate your Mojolicious app at your shell prompt:

$ mojo generate app Local::RouteDemo
  [mkdir] /Users/mgardner/Projects/blog/local_route_demo/script
  [write] /Users/mgardner/Projects/blog/local_route_demo/script/local_route_demo
  [chmod] /Users/mgardner/Projects/blog/local_route_demo/script/local_route_demo 744
  [mkdir] /Users/mgardner/Projects/blog/local_route_demo/lib/Local
  [write] /Users/mgardner/Projects/blog/local_route_demo/lib/Local/RouteDemo.pm
  [exist] /Users/mgardner/Projects/blog/local_route_demo
  [write] /Users/mgardner/Projects/blog/local_route_demo/local-route_demo.yml
  [mkdir] /Users/mgardner/Projects/blog/local_route_demo/lib/Local/RouteDemo/Controller
  [write] /Users/mgardner/Projects/blog/local_route_demo/lib/Local/RouteDemo/Controller/Example.pm
  [mkdir] /Users/mgardner/Projects/blog/local_route_demo/t
  [write] /Users/mgardner/Projects/blog/local_route_demo/t/basic.t
  [mkdir] /Users/mgardner/Projects/blog/local_route_demo/public
  [write] /Users/mgardner/Projects/blog/local_route_demo/public/index.html
  [mkdir] /Users/mgardner/Projects/blog/local_route_demo/templates/layouts
  [write] /Users/mgardner/Projects/blog/local_route_demo/templates/layouts/default.html.ep
  [mkdir] /Users/mgardner/Projects/blog/local_route_demo/templates/example
  [write] /Users/mgardner/Projects/blog/local_route_demo/templates/example/welcome.html.ep
$ cd local_route_demo

Create a new Perl mod­ule in your edi­tor for stor­ing your routes. Here we’re using Local::RouteDemo::Routes:

$ touch lib/Local/RouteDemo/Routes.pm
$ $EDITOR lib/Local/RouteDemo/Routes.pm

Make the mod­ule with a func­tion that will cre­ate the routes you want, giv­en a Mojolicious::Routes object. Here we’re just bring­ing over the default route cre­at­ed when we cre­at­ed our app:

package Local::RouteDemo::Routes;
use strict;
use warnings qw(all -experimental::signatures);
use feature 'signatures';
use Exporter 'import';
our @EXPORT_OK = qw(make_routes);

sub make_routes ($router) {
    $router->get('/')->to('Example#welcome');
    # add more routes here

    return;
}

1;

Adjust the appli­ca­tion class to load your new Routes mod­ule and call its export­ed function:

package Local::RouteDemo;
use Mojo::Base 'Mojolicious', -signatures;
use Local::RouteDemo::Routes 'make_routes';

# This method will run once at server start
sub startup ($self) {

    # Load configuration from config file
    my $config = $self->plugin('NotYAMLConfig');

    # Configure the application
    $self->secrets($config->{secrets});

    # Make routes
    make_routes($self->routes);

    return;
}

1;

Finally, run your tests and/​or man­u­al­ly test your routes to be sure every­thing works OK:

$ prove -vlr t
t/basic.t .. [2021-06-07 12:21:55.36917] [58779] [debug] [elVGykGVWlOt] GET "/"
[2021-06-07 12:21:55.36972] [58779] [debug] [elVGykGVWlOt] Routing to controller "Local::RouteDemo::Controller::Example" and action "welcome"
[2021-06-07 12:21:55.37137] [58779] [debug] [elVGykGVWlOt] Rendering template "example/welcome.html.ep"
[2021-06-07 12:21:55.37343] [58779] [debug] [elVGykGVWlOt] Rendering template "layouts/default.html.ep"
[2021-06-07 12:21:55.37495] [58779] [debug] [elVGykGVWlOt] 200 OK (0.005772s, 173.250/s)

ok 1 - GET /
ok 2 - 200 OK
ok 3 - content is similar
1..3
ok
All tests successful.
Files=1, Tests=3,  1 wallclock secs ( 0.02 usr  0.01 sys +  0.38 cusr  0.11 csys =  0.52 CPU)
Result: PASS
$ script/local_route_demo get /
[2021-06-07 12:22:29.55930] [58889] [debug] [f3YoaFhkwJ42] GET "/"
[2021-06-07 12:22:29.55990] [58889] [debug] [f3YoaFhkwJ42] Routing to controller "Local::RouteDemo::Controller::Example" and action "welcome"
[2021-06-07 12:22:29.56059] [58889] [debug] [f3YoaFhkwJ42] Rendering template "example/welcome.html.ep"
[2021-06-07 12:22:29.56269] [58889] [debug] [f3YoaFhkwJ42] Rendering template "layouts/default.html.ep"
[2021-06-07 12:22:29.56432] [58889] [debug] [f3YoaFhkwJ42] 200 OK (0.005004s, 199.840/s)
<!DOCTYPE html>
<html>
  <head><title>Welcome</title></head>
  <body><h2>Welcome to the Mojolicious real-time web framework!</h2>
<p>
  This page was generated from the template "templates/example/welcome.html.ep"
  and the layout "templates/layouts/default.html.ep",
  <a href="/">click here</a> to reload the page or
  <a href="/index.html">here</a> to move forward to a static page.
</p>
</body>
</html>

You can find a git repos­i­to­ry of this work on GitHub, and here’s a com­mit of all the changes made to the default Mojolicious appli­ca­tion so you can see the differences.

Update

Joel Berger from the Mojolicious project told me at The Perl and Raku Conference that it would be more idiomat­ic to use a Mojolicious plu­g­in rather than a plain mod­ule with an export, so here you go:

lib/Local/RouteDemo.pm:

package Local::RouteDemo;
use Mojo::Base 'Mojolicious', -signatures;

# This method will run once at server start
sub startup ($self) {

    # Load configuration from config file
    my $config = $self->plugin('NotYAMLConfig');

    # Configure the application
    $self->secrets($config->{secrets});

    # Add routes from plugin
    $self->plugin('Local::RouteDemo::Plugin::Routes');

    return;
}

1;

lib/Local/RouteDemo/Plugin/Routes.pm:

package Local::RouteDemo::Plugin::Routes;
use Mojo::Base 'Mojolicious::Plugin', -signatures;

sub register ($self, $app, $conf) {
    my $r = $app->routes;

    $r->get('/')->to('Example#welcome');
    # add more routes here

    return;
}

1;

Back To The Future DeLorean

Last week saw the release of Perl 5.34.0 (you can get it here), and with it comes a year’s worth of new fea­tures, per­for­mance enhance­ments, bug fix­es, and oth­er improve­ments. It seems like a good time to high­light some of my favorite changes over the past decade and a half, espe­cial­ly for those with more dat­ed knowl­edge of Perl. You can always click on the head­ers below for the full releas­es’ perldelta pages.

Perl 5.10 (2007)

This was a big release, com­ing as it did over five years after the pre­vi­ous major 5.8 release. Not that Perl devel­op­ers were idle — but it would­n’t be until ver­sion 5.14 that the lan­guage would adopt a steady year­ly release cadence.

Due to the build-​up time, many core enhance­ments were made but the most impor­tant was arguably the feature prag­ma, enabling the addi­tion of new syn­tax that would oth­er­wise break Perl’s back­ward com­pat­i­bil­i­ty. 5.10 also intro­duced the defined-​or oper­a­tor (//), state vari­ables that per­sist their pre­vi­ous val­ue, the say func­tion for auto­mat­i­cal­ly append­ing a new­line on out­put (so much saved typ­ing), and a large col­lec­tion of improve­ments to reg­u­lar expres­sions. In addi­tion, this release intro­duced smart match­ing (~~), though ver­sion 5.18 would even­tu­al­ly rel­e­gate it to exper­i­men­tal sta­tus.

Perl 5.12 (2010)

This release also saw many new fea­tures added, but if I had to pick one mar­quee item it would be exper­i­men­tal sup­port for plug­gable key­words, which enabled authors to extend the lan­guage itself with­out mod­i­fy­ing the core. Previously one would either use plain func­tions, hacky source fil­ters, or the dep­re­cat­ed Devel::Declare mod­ule to sim­u­late this func­tion­al­i­ty. CPAN authors would go on to cre­ate all kinds of new syn­tax, some­times pro­to­typ­ing fea­tures that would even­tu­al­ly make their way into core.

Perl 5.14 (2011)

5.14 had a big list of enhance­ments, includ­ing Unicode 6.0 sup­port and a gag­gle of reg­u­lar expres­sion fea­tures. My favorite of these was the /r switch for non-​destructive sub­sti­tu­tions.

But as the first year­ly cadence release, the changes in pol­i­cy took cen­ter stage. The Perl 5 Porters (p5p) explic­it­ly com­mit­ted to sup­port­ing the two most recent sta­ble release series, pro­vid­ing secu­ri­ty patch­es only for release series occur­ring in the past three years. They also defined an explic­it com­pat­i­bil­i­ty and dep­re­ca­tion pol­i­cy, with def­i­n­i­tions for fea­tures that may be exper­i­men­tal, dep­re­cat­ed, dis­cour­aged, and removed.

Perl 5.16 (2012)

Another year, anoth­er ver­sion bump. This time the core enhance­ments were all over the map (although no enhance­ments to the map function 😀 ).

May I high­light anoth­er doc­u­men­ta­tion change, though? The perlootut Object-​Oriented Programming in Perl Tutorial replaced the old perltoot, perltooc, perlboot, and perlbot pages, pro­vid­ing an intro­duc­tion to object-​oriented design con­cepts before strong­ly rec­om­mend­ing the use of one of the OO sys­tems from CPAN. Mentioned are Moose, its alter­na­tive Mouse, Class::Accessor, Object::Tiny, and Role::Tinys usage with the lat­ter two. Later ver­sions of perlootut would rec­om­mend Moo rather than Mouse.

Perl 5.18 (2013)

As men­tioned ear­li­er, Perl 5.18 ren­dered smart­match exper­i­men­tal, as well as lex­i­cal use of the $_ vari­able. With these came a new cat­e­go­ry of warn­ings for exper­i­men­tal fea­tures and a method for over­rid­ing such warn­ings feature-​by-​feature. Fitting in with the secu­ri­ty and safe­ty theme, hash­es were over­hauled to ran­dom­ize key/​value order, increas­ing their resis­tance to algo­rith­mic com­plex­i­ty attacks.

But it was­n’t all fenc­ing in bad behav­ior. Lexical sub­rou­tines made their first (exper­i­men­tal) appear­ance, and although I con­fess I haven’t had much call for them in my work, oth­ers have come up with some inter­est­ing uses. Four years lat­er they became non-​experimental.

Perl 5.20 (2014)

Three new syn­tax fea­tures arrived in 2014: exper­i­men­tal sub­rou­tine sig­na­tures (of which I’ve writ­ten more about here), key/​value hash slices and index/​value array slices, and exper­i­men­tal post­fix deref­er­enc­ing. This last enables clean­er left-​to-​right syn­tax when deref­er­enc­ing variables:

  • @{ $array_ref } becomes $array_ref->@*
  • %{ $hash_ref } becomes $hash_ref->%*
  • Etc.

Postfix deref­er­enc­ing became non-​experimental in Perl 5.24, and vig­or­ous dis­cus­sion con­tin­ues on sub­rou­tine sig­na­tures’ future.

Perl 5.22 (2015)

Speaking of sub­rou­tine sig­na­tures, their loca­tion moved to between the sub­rou­tine name (if any) and the attribute list (if any). Previously they appeared after attrib­ut­es. The les­son? Remain con­scious of exper­i­men­tal fea­tures in your code, and be pre­pared to make changes when upgrading.

In addi­tion to the enhance­ments, secu­ri­ty updates, per­for­mance fix­es, and dep­re­ca­tions, devel­op­ers removed the his­tor­i­cal­ly notable CGI mod­ule. First added to core in 1997 in recog­ni­tion of its crit­i­cal role in enabling web devel­op­ment, it’s been sup­plant­ed by bet­ter alter­na­tives on CPAN.

Perl 5.24 (2016)

Perl 5.20s post­fix deref­er­enc­ing was no longer exper­i­men­tal, and devel­op­ers removed both lex­i­cal $_ and autoderef­er­enc­ing on calls to push, pop, shift, unshift, splice, keys, values, and each.

Perl 5.26 (2017)

The incor­po­ra­tion of exper­i­men­tal fea­tures con­tin­ued, with lex­i­cal sub­rou­tines mov­ing into full sup­port. I like the added read­abil­i­ty enhance­ments, though: indent­ed here-​documents; the /xx reg­u­lar expres­sion mod­i­fi­er for tabs and spaces in char­ac­ter class­es; and @{^CAPTURE}, %{^CAPTURE}, and %{^CAPTURE_ALL} for reg­exp match­es with a lit­tle more self-documentation.

Perl 5.28 (2018)

Experimental sub­rou­tine sig­na­ture and attribute order­ing flipped back to its Perl 5.20 sequence of attributes-​then-​signature. Bit of a roller­coast­er ride on this one. You could do worse than using some­thing like Type::Params until this set­tles and get a wide vari­ety of type con­straints in the bargain.

Perl 5.30 (2019)

Pour one out for AWK and Fortran pro­gram­mers migrat­ing to Perl: the $[vari­able for set­ting the low­er bound of arrays could no longer be set to any­thing oth­er than zero. This had a long dep­re­ca­tion cycle start­ing in Perl 5.12.

Perl 5.32 (2020)

In 2020 Perl’s devel­op­ment moved to GitHub. And once again, I’m going to high­light read­abil­i­ty enhance­ments: the exper­i­men­tal isa oper­a­tor could be used to say:

if ( $obj isa Some::Class ) { ... }

instead of

use Scalar::Util 'blessed';
if ( blessed($obj) and $obj->isa('Some::Class') { ... }

You could also chain com­par­i­son oper­a­tors, lead­ing to the more math­e­mat­i­cal­ly con­cise if ( $x < $y <= $z ) {...} rather than if ( $x < $y and $y <= $z ) {...}.

Perl 5.34 (2021)

Finally, we come to last week’s release and its intro­duc­tion of exper­i­men­tal try/​catch excep­tion han­dling syn­tax. If you need to sup­port ear­li­er ver­sions of Perl back to 5.14, you can use Feature::Compat::Try. Earlier this year I inter­viewed the fea­ture and mod­ule’s author, Paul LeoNerd” Evans, for Perl.com. This year also marked the debut of Perl’s new gov­er­nance mod­el with the appoint­ment of a Core Team and a three-​member Steering Council.

What are some of your favorite Perl improve­ments over the years? Check out the perlhist doc­u­ment for a detailed chronol­o­gy and refresh­er with the var­i­ous perldelta pages and leave me a com­ment below.

text

Last week’s arti­cle received a com­ment on a pri­vate Facebook group that amount­ed to just use JavaScript’s built-​in for­mat­ting.” So what would that look like?

#!/usr/bin/env perl

use Mojolicious::Lite -signatures;
use DateTime;

get '/' =>
    sub ($c) { $c->render( template => 'index', date => DateTime->today ) };

helper localize_date => sub ( $c, $date = DateTime->today, $style = 'full' ) {
    my $date_params = join ',' => $date->year, $date->month_0, $date->day;
    return
        qq<new Date($date_params).toLocaleString( [], {dateStyle: "$style"})>;
};

app->start;
__DATA__
@@ index.html.ep
% layout 'default';
% title 'Today';
<ul>
    <li><script>
        document.write(<%== localize_date $date %>)
    </script></li>
    % for my $style ( qw(long medium short) ) {
    <li><script>
        document.write(<%== localize_date $date, $style %>)
    </script></li>
    % }
</ul>
@@ layouts/default.html.ep
<!DOCTYPE html>
<html>
    <head><title><%= title %></title></head>
    <body><%= content %></body>
</html>

It’s struc­tured much like the Perl-​only solu­tion, with a default "/" route and a localize_date Mojolicious helper to do the for­mat­ting. I opt­ed to out­put a piece of JavaScript from the helper on lines 11 through 14 since it could be repeat­ed sev­er­al times in a doc­u­ment. You could instead declare a func­tion in the default lay­out’s HTML <head> on line 38 that would receive a date and a for­mat­ting style, out­putting the result­ing for­mat­ted date.

In the tem­plate’s list from lines 22 through 31 I decid­ed to use JavaScript document.write method calls to add our gen­er­at­ed code. This has a slew of caveats but works for our exam­ple here.

Worth not­ing is the dou­ble equals sign (<%== %>) when embed­ding a Perl expres­sion. This pre­vents Mojolicious from XML-​escaping spe­cial char­ac­ters, e.g., replac­ing "quotes" with &quot;, <angle brack­ets> with &lt; and &gt;, etc.. This is impor­tant when return­ing HTML and JavaScript code.

I also chose to use the JavaScript Date objec­t’s toLocaleString() method for my for­mat­ting on line 12. There are oth­er ways to do this:

Note that line 10 builds the para­me­ters for JavaScript’s Date con­struc­tor using the year, month_0, and day meth­ods of our Perl DateTime object; month_0 because the Date con­struc­tor takes its month as an inte­ger from 0 to 11 rather than 1 to 12. JavaScript Dates can be con­struct­ed in many ways; this seemed the sim­plest with­out hav­ing to explain things like epochs and incon­sis­tent parsing.

Why are we using Perl DateTimes and a helper any­way? I’m assum­ing that our dates are com­ing from the back­end of our appli­ca­tion, pos­si­bly inflat­ed from a data­base col­umn. If your dates are strict­ly on the fron­tend, you might decide to put your for­mat­ting code there in a JavaScript func­tion, per­haps using a JavaScript-​based tem­plat­ing library.

The bot­tom line is to do what­ev­er makes sense for your sit­u­a­tion. I pre­fer the Perl solu­tion because I like the lan­guage and its ecosys­tem and per­haps have accli­mat­ed to its quirks. The com­pli­ca­tions of JavaScript brows­er sup­port, com­pet­ing frame­works, and lay­ers of tool­ing make my head hurt. Despite this, I’m still learn­ing; if you have any com­ments or sug­ges­tions, please leave them below.

Western and eastern hemispheres of the Earth

When we’re writ­ing soft­ware for a glob­al audi­ence, it’s nice if we can pro­vide it accord­ing to their native lan­guages and con­ven­tions. Translating all of the text can be a huge under­tak­ing, but we can start small by mak­ing sure that when we show the day and date it appears as the user expects. For exam­ple, to me it’s Tuesday, April 20, 2021; to my friend Paul in the UK it’s Tuesday, 20 April 2021 (note the dif­fer­ence in order), and to my oth­er friend Gabór in Israel it’s יום שלישי, 20 באפריל 2021 (note the dif­fer­ent direc­tion of the text, dif­fer­ent lan­guage, and char­ac­ter set).

Thankfully, we have a num­ber of tools to assist us:

  • The DateTime::Locale library, which enables our Perl soft­ware to rep­re­sent dates and times glob­al­ly and con­tains a cat­a­log of locales. It works with the DateTime library for stor­ing our dates as objects that can be eas­i­ly manip­u­lat­ed and formatted.
  • The HTTP Accept-​Language head­er, which lets a web brows­er com­mu­ni­cate to the serv­er what nat­ur­al lan­guages and locale vari­ants the user understands.
  • The HTTP::AcceptLanguage mod­ule, which helps us parse the Accept-​Language head­er and select a com­pat­i­ble locale.

Our sam­ple code uses the Mojolicious frame­work and is very sim­ple; almost half of it is just HTML web page tem­plates. You could eas­i­ly adapt it to oth­er frame­works or tem­plat­ing systems.

#!/usr/bin/env perl

use Mojolicious::Lite -signatures;
use DateTime;
use DateTime::Locale;
use HTTP::AcceptLanguage;

my %locales
    = map { $_ => DateTime::Locale->load($_) } DateTime::Locale->codes;

get '/' =>
    sub ($c) { $c->render( template => 'index', date => DateTime->today ) };

helper localize_date => sub ( $c, $date = DateTime->today, $format = 'full' )
{
    my $locale = $locales{ HTTP::AcceptLanguage->new(
            $c->req->headers->accept_language )->match( keys %locales ) };

    my $method_name = "date_format_$format";
    return $date->clone->set_locale($locale)
        ->format_cldr( $locale->$method_name );
};

app->start;
__DATA__
@@ index.html.ep
% layout 'default';
% title 'Today';
<ul>
    <li><%= localize_date $date %></li>
    % for my $format ( qw(long medium short) ) {
    <li><%= localize_date $date, $format %></li>
    % }
</ul>
@@ layouts/default.html.ep
<!DOCTYPE html>
<html>
    <head><title><%= title %></title></head>
    <body><%= content %></body>
</html>

Lines 1 through 5 tell our code to use the Perl inter­preter in our exe­cu­tion PATH and load our pre­req­ui­site mod­ules. Note we’re using the micro ver­sion of Mojolicious, Mojolicious::Lite; lat­er you can grow your appli­ca­tion into a well-​structured Mojolicious app. We’re also using Perl sub­rou­tine sig­na­tures, which requires Perl 5.20 or lat­er (released in 2014).

Lines 7 and 8 pre­load all of the avail­able DateTime::Locale objects so that we can serve requests faster with­out hav­ing to load a new locale every time. We cre­ate a hash where the keys are the locale iden­ti­fiers (for exam­ple, en-US for United States English), and the val­ues are the locale objects.

Line 10 begins our route han­dler for HTTP GET requests on the default / route in our web appli­ca­tion. When a brows­er hits the home page of our app, it will exe­cute the code in the anony­mous sub in line 11, which is passed the con­troller object as $c. It’s a very sim­ple han­dler that ren­ders a tem­plate called index (described below), pass­ing it a date object with today’s date.

Lines 13 through 23 are where the smarts of our appli­ca­tion lie. It’s a helper that we’ll call from our tem­plate to local­ize a date object, and it’s anoth­er anony­mous sub. This time it’s passed a Mojolicious con­troller as $c, a $date para­me­ter that defaults to today, and a $format para­me­ter that defaults to full’.

Lines 14 through 18 in the helper get our locale. Working from the inside out, we get the HTTP Accept-​Language head­er from the request on line 16, cre­ate a new HTTP::AcceptLanguage object in line 15 for pars­ing that head­er, and then match it against the keys in our glob­al %locales hash in line 17. That matched key then looks up the appro­pri­ate DateTime::Locale object from the hash.

DateTime only allows you to set a locale at object con­struc­tion time, so in line 19 we cre­ate a new object from the old one, set­ting its locale to our newly-​discovered $locale object. Finally, in lines 21 and 22 we deter­mine what method to call on that object to retrieve the CLDR (Common Locale Data Repository) for­mat­ting pat­tern for the request­ed for­mat and then return the for­mat­ted date.

Finally, line 25 starts the appli­ca­tion. To run it using the devel­op­ment serv­er includ­ed with Mojolicious, do this at the com­mand line:

$ morbo perl_date_locale.pl

There are oth­er options for deploy­ing your appli­ca­tion, includ­ing Mojolicious’ built-​in web serv­er, inside a con­tain­er, using oth­er web servers, etc.

The rest of the above script is in the __DATA__ por­tion and con­tains two pseudo-​files that Mojolicious knows how to read in the absence of actu­al tem­plates and lay­outs. First on line 28 is the actu­al index.html.ep HTML page, which uses Mojolicious’ Embedded Perl (ep) tem­plat­ing sys­tem to select a lay­out of shared HTML to use (the layouts/default.html.ep file start­ing on line 39).

Lines 32 through 37 ren­der an HTML unordered list that runs through the var­i­ous for­mat­ting options avail­able to our localize_date helper, first with the default full’ for­mat­ting, and then a loop through long’, medi­um’, and short’. Note that we call our helper as an expres­sion, with an equals (=) sign after the per­cent (%) sign.

If you want to test dif­fer­ent locales with­out chang­ing your brows­er or oper­at­ing sys­tem set­tings, you can invoke the script from the com­mand line along with the HTTP request and head­ers to pass along. Here’s an exam­ple using German:

$ perl perl_date_locale.pl get -H 'Accept-Language: de' /
[2021-04-17 16:39:57.81379] [5425] [debug] [LcCSBKMVd90t] GET "/"
[2021-04-17 16:39:57.81408] [5425] [debug] [LcCSBKMVd90t] Routing to a callback
[2021-04-17 16:39:57.81610] [5425] [debug] [LcCSBKMVd90t] Rendering template "index.html.ep" from DATA section
[2021-04-17 16:39:57.81714] [5425] [debug] [LcCSBKMVd90t] Rendering template "layouts/default.html.ep" from DATA section
[2021-04-17 16:39:57.81792] [5425] [debug] [LcCSBKMVd90t] 200 OK (0.004118s, 242.836/s)
<!DOCTYPE html>
<html>
    <head><title>Today</title></head>
    <body>
<ul>
    <li>Sonntag, 18. April 2021</li>
    <li>18. April 2021</li>
    <li>18.04.2021</li>
    <li>18.04.21</li>
</ul>
</body>
</html>

And here’s Japanese:

$ perl perl_date_locale.pl get -H 'Accept-Language: ja' /
[2021-04-17 16:40:56.10840] [5478] [debug] [Wmr6cN5KUJlP] GET "/"
[2021-04-17 16:40:56.10874] [5478] [debug] [Wmr6cN5KUJlP] Routing to a callback
[2021-04-17 16:40:56.11101] [5478] [debug] [Wmr6cN5KUJlP] Rendering template "index.html.ep" from DATA section
[2021-04-17 16:40:56.11255] [5478] [debug] [Wmr6cN5KUJlP] Rendering template "layouts/default.html.ep" from DATA section
[2021-04-17 16:40:56.11360] [5478] [debug] [Wmr6cN5KUJlP] 200 OK (0.005164s, 193.648/s)
<!DOCTYPE html>
<html>
    <head><title>Today</title></head>
    <body>
<ul>
    <li>2021年4月18日日曜日</li>
    <li>2021年4月18日</li>
    <li>2021/04/18</li>
    <li>2021/04/18</li>
</ul>
</body>
</html>

A full list of sup­port­ed locales is pro­vid­ed in the DateTime::Locale::Catalog documentation.

I hope this arti­cle has helped demon­strate that it’s not too hard to make your Perl web appli­ca­tions respect glob­al audi­ences, if only with dates. For more on local­iza­tion and Perl, start with the Locale::Maketext framework.