arrow communication direction display

When I first started writing Perl in my early 20’s, I tended to follow a lot of the structured programming conventions I had learned in school through Pascal, especially the notion that every function has a single point of exit. For example:

sub double_even_number {
    # not using signatures, this is mid-1990's code
    my $number = shift;

    if (not $number % 2) {
        $number *= 2;
    }

    return $number; 
}

This could get pretty convoluted, especially if I was doing something like validating multiple arguments. And at the time I didn’t yet grok how to handle exceptions with eval and die, so I’d end up with code like:

sub print_postal_address {
    # too many arguments, I know
    my ($name, $street1, $street2, $city, $state, $zip) = @_;
    # also this notion of addresses is naive and US-centric

    my $error;

    if (!$name) {
        $error = 'no name';
    }
    else {
        print "$name\n";

        if (!$street1) {
            $error = 'no street';
        }
        else {
            print "$street1\n";

            if ($street2) {
                print "$street2\n";
            }

            if (!$city) {
                $error = 'no city';
            }
            else {
                print "$city, ";

                if (!$state) {
                    $error = 'no state';
                }
                else {
                    print "$state ";

                    if (!$zip) {
                        $error = 'no ZIP code';
                    }
                    else {
                        print "$zip\n";
                    }
                }
            }
        }
    }

    return $error;
}

What a mess. Want to count all those braces to make sure they’re balanced? This is sometimes called the arrow anti-​pattern, with the arrowhead(s) being the most nested statement. The default ProhibitDeepNests perlcritic policy is meant to keep you from doing that.

The way out (literally) is guard clauses: checking early if something is valid and bailing out quickly if not. The above example could be written:

sub print_postal_address {
    my ($name, $street1, $street2, $city, $state, $zip) = @_;

    if (!$name) {
        return 'no name';
    }
    if (!$street1) {
        return 'no street1';
    }
    if (!$city) {
        return 'no city';
    }
    if (!$state) {
        return 'no state';
    }
    if (!$zip) {
        return 'no zip';
    }

    print join "\n",
      $name,
      $street1,
      $street2 ? $street2 : (),
      "$city, $state $zip\n";

    return;
}

With Perl’s statement modifiers (sometimes called postfix controls) we can do even better:

    ...

    return 'no name'    if !$name;
    return 'no street1' if !$street1;
    return 'no city'    if !$city;
    return 'no state'   if !$state;
    return 'no zip'     if !$zip;

    ...

(Why if instead of unless? Because the latter can be confusing with double-​negatives.)

Guard clauses aren’t limited to the beginnings of functions or even exiting functions entirely. Often you’ll want to skip or even exit early conditions in a loop, like this example that processes files from standard input or the command line:

while (<>) {
    next if /^SKIP THIS LINE: /;
    last if /^END THINGS HERE$/;

    ...
}

Of course, if you are validating function arguments, you should consider using actual subroutine signatures if you have a Perl newer than v5.20 (released in 2014), or one of the other type validation solutions if not. Today I would write that postal function like this, using Type::Params for validation and named arguments:

use feature qw(say state); 
use Types::Standard 'Str';
use Type::Params 'compile_named';

sub print_postal_address {
    state $check = compile_named(
        name    => Str,
        street1 => Str,
        street2 => Str, {optional => 1},
        city    => Str,
        state   => Str,
        zip     => Str,
    );
    my $arg = $check->(@_);

    say join "\n",
      $arg->{name},
      $arg->{street1},
      $arg->{street2} ? $arg->{street2} : (),
      "$arg->{city}, $arg->{state} $arg->{zip}";

    return;
}

print_postal_address(
    name    => 'J. Random Hacker',
    street1 => '123 Any Street',
    city    => 'Somewhereville',
    state   => 'TX',
    zip     => 12345,
);

Note that was this part of a larger program, I’d wrap that print_postal_address call in a try block and catch exceptions such as those thrown by the code reference $check generated by compile_named. This highlights one concern of guard clauses and other return early” patterns: depending on how much has already occurred in your program, you may have to perform some resource cleanup either in a catch block or something like Syntax::Keyword::Try’s finally block if you need to tidy up after both success and failure.

Friday, December 17, 2021, marked the thirty-​fourth birthday of the Perl programming language, and coincidentally this year saw the release of version 5.34. There are plenty of Perl developers out there who haven’t kept up with recent (and not-​so-​recent) improvements to the language and its ecosystem, so I thought I might list a batch. (You may have seen some of these before in May’s post Perl can do that now!”)

The feature pragma

Perl v5.10 was released in December 2007, and with it came feature, a way of enabling new syntax without breaking backward compatibility. You can enable individual features by name (e.g., use feature qw(say fc); for the say and fc keywords), or by using a feature bundle based on the Perl version that introduced them. For example, the following:

use feature ':5.34';

…gives you the equivalent of:

use feature qw(bareword_filehandles bitwise current_sub evalbytes fc indirect multidimensional postderef_qq say state switch unicode_eval unicode_strings);

Boy, that’s a mouthful. Feature bundles are good. The corresponding bundle also gets implicitly loaded if you specify a minimum required Perl version, e.g., with use v5.32;. If you use v5.12; or higher, strict mode is enabled for free. So just say:

use v5.34;

And lastly, one-​liners can use the -E switch instead of -e to enable all features for that version of Perl, so you can say the following on the command line:

perl -E 'say "Hello world!"'

Instead of:

perl -e 'print "Hello world!\n"'

Which is great when you’re trying to save some typing.

The experimental pragma

Sometimes new Perl features need to be driven a couple of releases around the block before their behavior settles. Those experiments are documented in the perlexperiment page, and usually, you need both a use feature (see above) and no warnings statement to safely enable them. Or you can simply pass a list to use experimental of the features you want, e.g.:

use experimental qw(isa postderef signatures);

Ever-​expanding warnings categories

March 2000 saw the release of Perl 5.6, and with it, the expansion of the -w command-​line switch to a system of fine-​grained controls for warning against dubious constructs” that can be turned on and off depending on the lexical scope. What started as 26 main and 20 subcategories has expanded into 31 main and 43 subcategories, including warnings for the aforementioned experimental features.

As the relevant Perl::Critic policy says, Using warnings, and paying attention to what they say, is probably the single most effective way to improve the quality of your code.” If you must violate warnings (perhaps because you’re rehabilitating some legacy code), you can isolate such violations to a small scope and individual categories. Check out the strictures module on CPAN if you’d like to go further and make a safe subset of these categories fatal during development.

Document other recently-​introduced syntax with Syntax::Construct

Not every new bit of Perl syntax is enabled with a feature guard. For the rest, there’s E. Choroba’s Syntax::Construct module on CPAN. Rather than having to remember which version of Perl introduced what, Syntax::Construct lets you declare only what you use and provides a helpful error message if someone tries to run your code on an older unsupported version. Between it and the feature pragma, you can prevent many head-​scratching moments and give your users a chance to either upgrade or workaround.

Make built-​in functions throw exceptions with autodie

Many of Perl’s built-​in functions only return false on failure, requiring the developer to check every time whether a file can be opened or a system command executed. The lexical autodie pragma replaces them with versions that raise an exception with an object that can be interrogated for further details. No matter how many functions or methods deep a problem occurs, you can choose to catch it and respond appropriately. This leads us to…

try/​catch exception handling and Feature::Compat::Try

This year’s Perl v5.34 release introduced experimental try/​catch syntax for exception handling that should look more familiar to users of other languages while handling the issues surrounding using block eval and testing of the special $@ variable. If you need to remain compatible with older versions of Perl (back to v5.14), just use the Feature::Compat::Try module from CPAN to automatically select either v5.34’s native try/​catch or a subset of the functionality provided by Syntax::Keyword::Try.

Pluggable keywords

The abovementioned Syntax::Keyword::Try was made possible by the introduction of a pluggable keyword mechanism in 2010’s Perl v5.12. So was the Future::AsyncAwait asynchronous programming library and the Object::Pad testbed for new object-​oriented Perl syntax. If you’re handy with C and Perl’s XS glue language, check out Paul LeoNerd” Evans’ XS::Parse::Keyword module to get a leg up on developing your own syntax module.

Define packages with versions and blocks

Perl v5.12 also helped reduce clutter by enabling a package namespace declaration to also include a version number, instead of requiring a separate our $VERSION = ...; v5.14 further refined packages to be specified in code blocks, so a namespace declaration can be the same as a lexical scope. Putting the two together gives you:

package Local::NewHotness v1.2.3 {
    ...
}

Instead of:

{
    package Local::OldAndBusted;
    use version 0.77; our $VERSION = version->declare("v1.2.3");
    ...
}

I know which I’d rather do. (Though you may want to also use Syntax::Construct qw(package-version package-block); to help along with older installations as described above.)

The // defined-​or operator

This is an easy win from Perl v5.10:

defined $foo ? $foo : $bar  # replace this
$foo // $bar                # with this

And:

$foo = $bar unless defined $foo  # replace this
$foo //= $bar                    # with this

Perfect for assigning defaults to variables.

state variables only initialize once

Speaking of variables, ever want one to keep its old value the next time a scope is entered, like in a sub? Declare it with state instead of my. Before Perl v5.10, you needed to use a closure instead.

Save some typing with say

Perl v5.10’s bumper crop of enhancements also included the say function, which handles the common use case of printing a string or list of strings with a newline. It’s less noise in your code and saves you four characters. What’s not to love?

Note unimplemented code with ...

The ... ellipsis statement (colloquially yada-​yada”) gives you an easy placeholder for yet-​to-​be-​implemented code. It parses OK but will throw an exception if executed. Hopefully, your test coverage (or at least static analysis) will catch it before your users do.

Loop and enumerate arrays with each, keys, and values

The each, keys, and values functions have always been able to operate on hashes. Perl v5.12 and above make them work on arrays, too. The latter two are mainly for consistency, but you can use each to iterate over an array’s indices and values at the same time:

while (my ($index, $value) = each @array) {
    ...
}

This can be problematic in non-​trivial loops, but I’ve found it helpful in quick scripts and one-liners.

delete local hash (and array) entries

Ever needed to delete an entry from a hash (e.g, an environment variable from %ENV or a signal handler from %SIG) just inside a block? Perl v5.12 lets you do that with delete local.

Paired hash slices

Jumping forward to 2014’s Perl v5.20, the new %foo{'bar', 'baz'} syntax enables you to slice a subset of a hash with its keys and values intact. Very helpful for cherry-​picking or aggregating many hashes into one. For example:

my %args = (
    verbose => 1,
    name    => 'Mark',
    extra   => 'pizza',
);
# don't frob the pizza
$my_object->frob( %args{ qw(verbose name) };

Paired array slices

Not to be left out, you can also slice arrays in the same way, in this case returning indices and values:

my @letters = 'a' .. 'z';
my @subset_kv = %letters[16, 5, 18, 12];
# @subset_kv is now (16, 'p', 5, 'e', 18, 'r', 12, 'l')

More readable dereferencing

Perl v5.20 introduced and v5.24 de-​experimentalized a more readable postfix dereferencing syntax for navigating nested data structures. Instead of using {braces} or smooshing sigils to the left of identifiers, you can use a postfixed sigil-and-star:

push @$array_ref,    1, 2, 3;  # noisy
push @{$array_ref},  1, 2, 3;  # a little easier
push $array_ref->@*, 1, 2, 3;  # read from left to right

So much of web development is slinging around and picking apart complicated data structures via JSON, so I welcome anything like this to reduce the cognitive load.

when as a statement modifier

Starting in Perl v5.12, you can use the experimental switch features when keyword as a postfix modifier. For example:

for ($foo) {
    $a =  1 when /^abc/;
    $a = 42 when /^dna/;
    ...
}

But I don’t recommend when, given, or givens smartmatch operations as they were retconned as experiments in 2013’s Perl v5.18 and have remained so due to their tricky behavior. I wrote about some alternatives using stable syntax back in February.

Simple class inheritance with use parent

Sometimes in older object-​oriented Perl code, you’ll see use base as a pragma to establish inheritance from another class. Older still is the direct manipulation of the package’s special @ISA array. In most cases, both should be avoided in favor of use parent, which was added to core in Perl v5.10.1.

Mind you, if you’re following the Perl object-​oriented tutorial’s advice and have selected an OO system from CPAN, use its subclassing mechanism if it has one. Moose, Moo, and Class::Accessor’s antlers” mode all provide an extends function; Object::Pad provides an :isa attribute on its class keyword.

Test for class membership with the isa operator

As an alternative to the isa() method provided to all Perl objects, Perl v5.32 introduced the experimental isa infix operator:

$my_object->isa('Local::MyClass')
# or
$my_object isa Local::MyClass

The latter can take either a bareword class name or string expression, but more importantly, it’s safer as it also returns false if the left argument is undefined or isn’t a blessed object reference. The older isa() method will throw an exception in the former case and might return true if called as a class method when $my_object is actually a string of a class name that’s the same as or inherits from isa()s argument.

Lexical subroutines

Introduced in Perl v5.18 and de-​experimentalized in 2017’s Perl v5.26, you can now precede sub declarations with my, state, or our. One use of the first two is truly private functions and methods, as described in this 2018 Dave Jacoby blog and as part of Neil Bowers’ 2014 survey of private function techniques.

Subroutine signatures

I’ve written and presented extensively about signatures and alternatives over the past year, so I won’t repeat that here. I’ll just add that the Perl 5 Porters development mailing list has been making a concerted effort over the past month to hash out the remaining issues towards rendering this feature non-​experimental. The popular Mojolicious real-​time web framework also provides a shortcut for enabling signatures and uses them extensively in examples.

Indented here-​documents with <<~

Perl has had shell-​style here-​document” syntax for embedding multi-​line strings of quoted text for a long time. Starting with Perl v5.26, you can precede the delimiting string with a ~ character and Perl will both allow the ending delimiter to be indented as well as strip indentation from the embedded text. This allows for much more readable embedded code such as runs of HTML and SQL. For example:

if ($do_query) {
    my $rows_deleted = $dbh->do(<<~'END_SQL', undef, 42);
      DELETE FROM table
      WHERE status = ?
      END_SQL
    say "$rows_deleted rows were deleted."; 
}

More readable chained comparisons

When I learned math in school, my teachers and textbooks would often describe multiple comparisons and inequalities as a single expression. Unfortunately, when it came time to learn programming every computer language I saw required them to be broken up with a series of and (or &&) operators. With Perl v5.32, this is no more:

if ( $x < $y && $y <= $z ) { ... }  # old way
if ( $x < $y <= $z )       { ... }  # new way

It’s more concise, less noisy, and more like what regular math looks like.

Self-​documenting named regular expression captures

Perl’s expressive regular expression matching and text-​processing prowess are legendary, although overuse and poor use of readability enhancements often turn people away from them (and Perl in general). We often use regexps for extracting data from a matched pattern. For example:

if ( /Time: (..):(..):(..)/ ) {  # parse out values
    say "$1 hours, $2 minutes, $3 seconds";
}

Named capture groups, introduced in Perl v5.10, make both the pattern more obvious and retrieval of its data less cryptic:

if ( /Time: (?<hours>..):(?<minutes>..):(?<seconds>..)/ ) {
    say "$+{hours} hours, $+{minutes} minutes, $+{seconds} seconds";
}

More readable regexp character classes

The /x regular expression modifier already enables better readability by telling the parser to ignore most whitespace, allowing you to break up complicated patterns into spaced-​out groups and multiple lines with code comments. With Perl v5.26 you can specify /xx to also ignore spaces and tabs inside [bracketed] character classes, turning this:

/[d-eg-i3-7]/
/[!@"#$%^&*()=?<>']/

…into this:

/ [d-e g-i 3-7]/xx
/[ ! @ " # $ % ^ & * () = ? <> ' ]/xx

Set default regexp flags with the re pragma

Beginning with Perl v5.14, writing use re '/xms'; (or any combination of regular expression modifier flags) will turn on those flags until the end of that lexical scope, saving you the trouble of remembering them every time.

Non-​destructive substitution with s///r and tr///r

The s/// substitution and tr/// transliteration operators typically change their input directly, often in conjunction with the =~ binding operator:

s/foo/bar/;  # changes the first foo to bar in $_
$baz =~ s/foo/bar/;  # the same but in $baz

But what if you want to leave the original untouched, such as when processing an array of strings with a map? With Perl v5.14 and above, add the /r flag, which makes the substitution on a copy and returns the result:

my @changed = map { s/foo/bar/r } @original;

Unicode case-​folding with fc for better string comparisons

Unicode and character encoding in general are complicated beasts. Perl has handled Unicode since v5.6 and has kept pace with fixes and support for updated standards in the intervening decades. If you need to test if two strings are equal regardless of case, use the fc function introduced in Perl v5.16.

Safer processing of file arguments with <<>>

The <> null filehandle or diamond operator” is often used in while loops to process input per line coming either from standard input (e.g., piped from another program) or from a list of files on the command line. Unfortunately, it uses a form of Perl’s open function that interprets special characters such as pipes (|) that would allow it to insecurely run external commands. Using the <<>> double diamond” operator introduced in Perl v5.22 forces open to treat all command-​line arguments as file names only. For older Perls, the perlop documentation recommends the ARGV::readonly CPAN module.

Safer loading of Perl libraries and modules from @INC

Perl v5.26 removed the ability for all programs to load modules by default from the current directory, closing a security vulnerability originally identified and fixed as CVE-20161238 in previous versions’ included scripts. If your code relied on this unsafe behavior, the v5.26 release notes include steps on how to adapt.

HTTP::Tiny simple HTTP/1.1 client included

To bootstrap access to CPAN on the web in the possible absence of external tools like curl or wget, Perl v5.14 began including the HTTP::Tiny module. You can also use it in your programs if you need a simple web client with no dependencies.

Test2: The next generation of Perl testing frameworks

Forked and refactored from the venerable Test::Builder (the basis for the Test::More library that many are familiar with), Test2 was included in the core module library beginning with Perl v5.26. I’ve experimented recently with using the Test2::Suite CPAN library instead of Test::More and it looks pretty good. I’m also intrigued by Test2::Harness’ support for threading, forking, and preloading modules to reduce test run times.

Task::Kensho: Where to start for recommended Perl modules

This last item may not be included when you install Perl, but it’s where I turn for a collection of well-​regarded CPAN modules for accomplishing a wide variety of common tasks spanning from asynchronous programming to XML. Use it as a starting point or interactively select the mix of libraries appropriate to your project.


And there you have it: a selection of 34 features, enhancements, and improvements for the first 34 years of Perl. What’s your favorite? Did I miss anything? Let me know in the comments.

happy man funny sticking tongue out

Over the past two years, I’ve gotten back into playing Dungeons & Dragons, the famous tabletop fantasy role-​playing game. As a software developer and musician, one of my favorite character classes to play is the bard, a magical and inspiring performer or wordsmith. The list of basic bardic spells includes Vicious Mockery, enchanting verbal barbs that have the power to psychically damage and disadvantage an opponent even if they don’t understand the words. (Can you see why this is so appealing to a coder?)

Mocking has a role to play in software testing as well, in the form of mock objects that simulate parts of a system that are too brittle, too slow, too complicated, or otherwise too finicky to use in reality. They enable discrete unit testing without relying on dependencies external to the code being tested. Mocks are great for databases, web services, or other network resources where the goal is to test what you wrote, not what’s out in the cloud” somewhere.

Speaking of web services and mocking, one of my favorites is the long-​running FOAAS (link has language not safe for work), a surprisingly expansive RESTful insult service. There’s a corresponding Perl client API, of course, but what I was missing was a handy Perl script to call that API from the terminal command line. So I wrote the following over Thanksgiving break, trying to keep it simple while also showing the basics of mocking such an API. It also demonstrates some newer Perl syntax and testing techniques as well as brian d foys modulino concept from Mastering Perl (second edition, 2014) that marries script and module into a self-​contained executable library.

#!/usr/bin/env perl

package Local::CallFOAAS;  # this is a modulino
use Test2::V0;             # enables strict, warnings, utf8

# declare all the new stuff we're using
use feature qw(say state);
use experimental qw(isa postderef signatures);
use Feature::Compat::Try;
use Syntax::Construct qw(non-destructive-substitution);

use WebService::FOAAS ();
use Package::Stash;
use Exception::Class (
    NoMethodException => {
        alias  => 'throw_no_method',
        fields => 'method',
    },
    ServiceException => { alias => 'throw_service' },
);

my $foaas = Package::Stash->new('WebService::FOAAS');

my $run_as =
    !!$ENV{CPANTEST}       ? 'test'
  : !defined scalar caller ? 'run'
  :                          undef;
__PACKAGE__->$run_as(@ARGV) if defined $run_as;

sub run ( $class, @args ) {
    try { say $class->call_method(@args) }
    catch ($e) {
        die 'No method ', $e->method, "\n"
          if $e isa NoMethodException;
        die 'Service error: ', $e->error, "\n"
          if $e isa ServiceException;
        die "$e\n";
    }
    return;
}

# Utilities

sub methods ($) {
    state @methods = sort map s/^foaas_(.+)/$1/r,
      grep /^foaas_/, $foaas->list_all_symbols('CODE');
    return @methods;
}

sub call_method ( $class, $method = '', @args ) {
    state %methods = map { $_ => 1 } $class->methods();
    throw_no_method( method => $method )
      unless $methods{$method};
    return do {
        try { $foaas->get_symbol("&$method")->(@args) }
        catch ($e) { throw_service( error => $e ) }
    };
}

# Testing

sub test ( $class, @ ) {
    state $stash = Package::Stash->new($class);
    state @tests = sort grep /^_test_/,
      $stash->list_all_symbols('CODE');

    for my $test (@tests) {
        subtest $test => sub {
            try { $class->$test() }
            catch ($e) { diag $e }
        };
    }
    done_testing();
    return;
}

sub _test_can ($class) {
    state @subs = qw(run call_method methods test);
    can_ok( $class, \@subs, "can do: @subs" );
    return;
}

sub _test_methods ($class) {
    my $mock = mock 'WebService::FOAAS' => ( track => 1 );

    for my $method ( $class->methods() ) {
        $mock->override( $method => 1 );

        ok lives { $class->call_method($method) },
          "$method lives";
        ok scalar $mock->sub_tracking->{$method}->@*,
          "$method called";
    }
    return;
}

sub _test_service_failure ($class) {
    my $mock = mock 'WebService::FOAAS';

    for my $method ( $class->methods() ) {
        $mock->override( $method => sub { die 'mocked' } );

        my $exception =
          dies { $class->call_method($method) };
        isa_ok $exception, ['ServiceException'],
          "$method throws ServiceException on failure";
        like $exception->error, qr/^mocked/,
          "correct error in $method exception";
    }
    return;
}

1;

Let’s walk through the code above.

Preliminaries

First, there’s a generic shebang line to indicate that Unix and Linux systems should use the perl executable found in the user’s PATH via the env command. I declare a package name (in the Local:: namespace) so as not to pollute the default main package of other scripts that might want to require this as a module. Then I use the Test2::V0 bundle from Test2::Suite since the embedded testing code uses many of its functions. This also has the side effect of enabling the strict, warnings, and utf8 pragmas, so there’s no need to explicitly use them here.

(Why Test2 instead of Test::More and its derivatives and add-​ons? Both are maintained by the same author, who recommends the former. I’m seeing more and more modules using it, so I thought this would be a great opportunity to learn.)

I then declare all the new-​ish Perl features I’d like to use that need to be explicitly enabled so as not to sacrifice backward compatibility with older versions of Perl 5. As of this writing, some of these features (the isa class instance operator, named argument subroutine signatures, and try/​catch exception handling syntax) are considered experimental, with the latter enabled in older versions of Perl via the Feature::Compat::Try module. The friendlier postfix dereferencing syntax was mainlined in Perl version 5.24, but versions 5.20 and 5.22 still need it experimental. Finally, I use Syntax::Construct to announce the /r flag for non-​destructive regular expression text substitutions introduced in version 5.14.

Next, I bring in the aforementioned FOAAS Perl API without importing any of its functions, Package::Stash to make metaprogramming easier, and a couple of exception classes so that the command line function and other consumers might better tell what caused a failure. In preparation for the methods below dynamically discovering what functions are provided by WebService::FOAAS, I gather up its symbol table (or stash) into the $foaas variable.

The next block determines how, if at all, I’m going to run the code as a script. If the CPANTEST environment variable is set, I’ll call the test class method sub, but if there’s no subroutine calling me I’ll execute the run class method. Either will receive the command line arguments from @ARGV. If neither of these conditions is true, do nothing; the rest of the code is method declarations.

Modulino methods, metaprogramming, and exceptions

The first of these is the run method. It’s a thin wrapper around the call_method class method detailed below, either outputting its result or dieing with an appropriate error depending on the class of exception thrown. Although I chose not to write tests for this output, future tests might call this method and catch these rethrown exceptions to match against them. The messages end with a \n newline character so die knows not to append the current script line number.

Next is a utility method called methods that uses Package::Stash’s list_all_symbols to retrieve the names of all named CODE blocks (i.e., subs) from WebService::FOAAS’s symbol table. Reading from right to left, these are then filtered with grep to only find those beginning in foaas_ and then transformed with map to remove that prefix. The list is then sorted and stored in a state variable and returned so it need not be initialized again.

(As an aside, although perlcritic sternly warns against it I’ve chosen the expression forms of grep and map here over their block forms for simplicity’s sake. It’s OK to bend the rules if you have a good reason.)

sub call_method is where the real action takes place. Its parameters are the class that called it, the name of a FOAAS $method (defaulted to the empty string), and an array of optional arguments in @args. I build a hash or associative array from the earlier methods method which I then use to see if the passed method name is one I know about. If not, I throw a NoMethodException using the throw_no_method alias function created when I used Exception::Class at the beginning. Using a function instead of NoMethodException->throw() means that it’s checked at compile time rather than runtime, catching typos.

I get the subroutine (denoted by a & sigil) named by $method from the $foaas stash and pass it any further received arguments from @args. If that WebService::FOAAS subroutine throws an exception it’ll be caught and re-​thrown as a ServiceException; otherwise call_method returns the result. It’s up to the caller to determine what, if anything, to do with that result or any thrown exceptions.

Testing the modulino with mocks

This is where I start using those Test2::Suite tools I mentioned at the beginning. The test class method starts by building a filtered list of all subs beginning with _test_ in the current class, much like methods did above with WebService::FOAAS. I then loop through that list of subs, running each as a subtest containing a class method with any exceptions reported as diagnostics.

The rest of the modulino is subtest methods, starting with a simple _test_can sanity check for the public methods in the class. Following that is _test_methods, which starts by mocking the WebService::FOAAS package and telling Test2::Mock I want to track any added, overridden, or set subs. I then loop through all the method names returned by the methods class method, overrideing each one to return a simple true value. I then test passing those names to call_method and use the hash reference returned by sub_tracking to check that the overridden sub was called. This seems a lot simpler than the Test::Builder-based mocking libraries I’ve tried like Test::MockModule and Test::MockObject.

_test_service_failure acts in much the same way, checking that call_method correctly throws ServiceExceptions if the wrapped WebService::FOAAS function dies. The main difference is that the mocked WebService::FOAAS subs are now overridden with a code reference (sub { die 'mocked' }), which call_method uses to populate the rethrown ServiceExceptions error field.

Wrapping up

With luck, this article has given you some ideas, whether it’s in making scripts (perhaps legacy code) testable to improve them, or writing better unit tests that mock dependencies, or delving a little into metaprogramming so you can dynamically support and test new features of said dependencies. I hope you haven’t come away too offended, at least. Let me know in the comments what you think.

woman looking at the map

Six months ago I gave an overview of Perl’s list processing fundamentals, briefly describing what lists are and then introducing the built-​in map and grep functions for transforming and filtering them. Later on, I compiled a list (how appropriate) of list processing modules available via CPAN, noting there’s some confusing duplication of effort. But you’re a busy developer, and you just want to know the Right Thing To Do™ when faced with a list processing challenge.

First, some credit is due: these are all restatements of several Perl::Critic policies which in turn codify standards described in Damian Conway’s Perl Best Practices (2005). I’ve repeatedly recommended the latter as a starting point for higher-​quality Perl development. Over the years these practices continue to be re-​evaluated (including by the author himself) and various authors release new policy modules, but perlcritic remains a great tool for ensuring you (and your team or other contributors) maintain a consistent high standard in your code.

With that said, on to the recommendations!

Don’t use grep to check if any list elements match

It might sound weird to lead off by recommending not to use grep, but sometimes it’s not the right tool for the job. If you’ve got a list and want to determine if a condition matches any item in it, you might try:

if (grep { some_condition($_) } @my_list) {
    ... # don't do this!
}

Yes, this works because (in scalar context) grep returns the number of matches found, but it’s wasteful, checking every element of @my_list (which could be lengthy) before finally providing a result. Use the standard List::Util module’s any function, which immediately returns (“short-​circuits”) on the first match:

use List::Util 1.33 qw(any);

if (any { some_condition($_) } @my_list) {
... # do something
}

Perl has included the requisite version of this module since version 5.20 in 2014; for earlier releases, you’ll need to update from CPAN. List::Util has many other great list-​reduction, key/​value pair, and other related functions you can import into your code, so check it out before you attempt to re-​invent any wheels.

As a side note for web developers, the Perl Dancer framework also includes an any keyword for declaring multiple HTTP routes, so if you’re mixing List::Util in there don’t import it. Instead, call it explicitly like this or you’ll get an error about a redefined function:

use List::Util 1.33;

if (List::Util::any { some_condition($_) } @my_list) {
... # do something
}

This recommendation is codified in the BuiltinFunctions::ProhibitBooleanGrep Perl::Critic policy, comes directly from Perl Best Practices, and is recommended by the Software Engineering Institute Computer Emergency Response Team (SEI CERT)’s Perl Coding Standard.

Don’t change $_ in map or grep

I mentioned this back in March, but it bears repeating: map and grep are intended as pure functions, not mutators with side effects. This means that the original list should remain unchanged. Yes, each element aliases in turn to the $_ special variable, but that’s for speed and can have surprising results if changed even if it’s technically allowed. If you need to modify an array in-​place use something like:

for (@my_array) {
$_ = ...; # make your changes here
}

If you want something that looks like map but won’t change the original list (and don’t mind a few CPAN dependencies), consider List::SomeUtilsapply function:

use List::SomeUtils qw(apply);

my @doubled_array = apply {$_ *= 2} @old_array;

Lastly, side effects also include things like manipulating other variables or doing input and output. Don’t use map or grep in a void context (i.e., without a resulting array or list); do something with the results or use a for or foreach loop:

map { print foo($_) } @my_array; # don't do this
print map { foo($_) } @my_array; # do this instead

map { push @new_array, foo($_) } @my_array; # don't do this
@new_array = map { foo($_) } @my_array; # do this instead

This recommendation is codified by the BuiltinFunctions::ProhibitVoidGrep, BuiltinFunctions::ProhibitVoidMap, and ControlStructures::ProhibitMutatingListFunctions Perl::Critic policies. The latter comes from Perl Best Practices and is an SEI CERT Perl Coding Standard rule.

Use blocks with map and grep, not expressions

You can call map or grep like this (parentheses are optional around built-​in functions):

my @new_array  = map foo($_), @old_array; # don't do this
my @new_array2 = grep !/^#/, @old_array; # don't do this

Or like this:

my @new_array  = map { foo($_) } @old_array;
my @new_array2 = grep {!/^#/} @old_array;

Do it the second way. It’s easier to read, especially if you’re passing in a literal list or multiple arrays, and the expression forms can conceal bugs. This recommendation is codified by the BuiltinFunctions::RequireBlockGrep and BuiltinFunctions::RequireBlockMap Perl::Critic policies and comes from Perl Best Practices.

Refactor multi-​statement maps, greps, and other list functions

map, grep, and friends should follow the Unix philosophy of Do One Thing and Do It Well.” Your readability and maintainability drop with every statement you place inside one of their blocks. Consider junior developers and future maintainers (this includes you!) and refactor anything with more than one statement into a separate subroutine or at least a for loop. This goes for list processing functions (like the aforementioned any) imported from other modules, too.

This recommendation is codified by the Perl Best Practices-inspired BuiltinFunctions::ProhibitComplexMappings and BuiltinFunctions::RequireSimpleSortBlock Perl::Critic policies, although those only cover map and sort functions, respectively.


Do you have any other suggestions for list processing best practices? Feel free to leave them in the comments or better yet, consider creating new Perl::Critic policies for them or contacting the Perl::Critic team to develop them for your organization.

Look, I get it. You don’t like the Perl programming language or have otherwise disregarded it as dead.” (Or perhaps you haven’t, in which case please check out my other blog posts!) It has weird noisy syntax, mixing regular expressions, sigils on variable names, various braces and brackets for data structures, and a menagerie of cryptic special variables. It’s old: 34 years in December, with a history of (sometimes amateur) developers that have used and abused that syntax to ship code of questionable quality. Maybe you grudgingly accept its utility but think it should die gracefully, maintained only to run legacy applications.

But you know what? Perl’s still going. It’s had a steady cadence of yearly releases for the past decade, introducing new features and fencing in bad behavior while maintaining an admirable level of backward compatibility. Yes, there was a too-​long adventure developing what started as Perl 6, but that language now has its own identity as Raku and even has facilities for mixing Perl with its native code or vice versa.

And then there’s CPAN, the Comprehensive Perl Archive Network: a continually-​updated collection of over 200,000 open-​source modules written by over 14,000 authors, the best of which are well-​tested and ‑documented (applying peer pressure to those that fall short), presented through a search engine and front-​end built by scores of contributors. Through CPAN you can find distributions for things like:

All of this is available through a mature installation toolchain that doesn’t break from month to month.

Finally and most importantly, there’s the global Perl community. The COVID-​19 pandemic has put a damper on the hundreds of global Perl Mongers groups’ meetups, but that hasn’t stopped the yearly Perl and Raku Conference from meeting virtually. (In the past there have also been yearly European and Asian conferences, occasional forays into South America and Russia, as well as hackathons and workshops worldwide.) There are IRC servers and channels for chat, mailing lists galore, blogs (yes, apart from this one), and a quirky social network that predates Facebook and Twitter.

So no, Perl isn’t dead or even dying, but if you don’t like it and favor something newer, that’s OK! Technologies can coexist on their own merits and advocates of one don’t have to beat down their contemporaries to be successful. Perl happens to be battle-​tested (to borrow a term from my friend Curtis Ovid” Poe), it runs large parts of the Web (speaking from direct and ongoing experience in the hosting business here), and it’s still evolving to meet the needs of its users.