The Phoenix Trap

Code, music, philosophy, etc.

Tag: smartmatch

34 at 34 for v5.34: Modern Perl features for Perl’s birthday
Friday, December 17, 2021, marked the thirty-fourth birthday of the Perl programming language, and coincidentally this year saw the release of version 5.34. There are plenty of Perl developers out there who haven’t kept up with recent (and not-so-recent) improvements to the language and its ecosystem, so I thought I might list a batch. (You may have seen some of these before in May’s post “Perl can do that now!”)

The feature pragma

Perl v5.10 was released in December 2007, and with it came feature, a way of enabling new syntax without breaking backward compatibility. You can enable individual features by name (e.g., use feature qw(say fc); for the say and fc keywords), or by using a feature bundle based on the Perl version that introduced them. For example, the following:
```
use feature ':5.34';
```
…gives you the equivalent of:
```
use feature qw(bareword_filehandles bitwise current_sub evalbytes fc indirect multidimensional postderef_qq say state switch unicode_eval unicode_strings);
```
Boy, that’s a mouthful. Feature bundles are good. The corresponding bundle also gets implicitly loaded if you specify a minimum required Perl version, e.g., with use v5.32;. If you use v5.12; or higher, strict mode is enabled for free. So just say:
```
use v5.34;
```
And lastly, one-liners can use the -E switch instead of -e to enable all features for that version of Perl, so you can say the following on the command line:
```
perl -E 'say "Hello world!"'
```
Instead of:
```
perl -e 'print "Hello world!\n"'
```
Which is great when you’re trying to save some typing.

The experimental pragma

Sometimes new Perl features need to be driven a couple of releases around the block before their behavior settles. Those experiments are documented in the perlexperiment page, and usually, you need both a use feature (see above) and no warnings statement to safely enable them. Or you can simply pass a list to use experimental of the features you want, e.g.:
```
use experimental qw(isa postderef signatures);
```
Ever-expanding warnings categories

March 2000 saw the release of Perl 5.6, and with it, the expansion of the -w command-line switch to a system of fine-grained controls for warning against “dubious constructs” that can be turned on and off depending on the lexical scope. What started as 26 main and 20 subcategories has expanded into 31 main and 43 subcategories, including warnings for the aforementioned experimental features.

As the relevant Perl::Critic policy says, “Using warnings, and paying attention to what they say, is probably the single most effective way to improve the quality of your code.” If you must violate warnings (perhaps because you’re rehabilitating some legacy code), you can isolate such violations to a small scope and individual categories. Check out the strictures module on CPAN if you’d like to go further and make a safe subset of these categories fatal during development.

Document other recently-introduced syntax with Syntax::Construct

Not every new bit of Perl syntax is enabled with a feature guard. For the rest, there’s E. Choroba’s Syntax::Construct module on CPAN. Rather than having to remember which version of Perl introduced what, Syntax::Construct lets you declare only what you use and provides a helpful error message if someone tries to run your code on an older unsupported version. Between it and the feature pragma, you can prevent many head-scratching moments and give your users a chance to either upgrade or workaround.

Make built-in functions throw exceptions with autodie

Many of Perl’s built-in functions only return false on failure, requiring the developer to check every time whether a file can be opened or a system command executed. The lexical autodie pragma replaces them with versions that raise an exception with an object that can be interrogated for further details. No matter how many functions or methods deep a problem occurs, you can choose to catch it and respond appropriately. This leads us to…

try/catch exception handling and Feature::Compat::Try

This year’s Perl v5.34 release introduced experimental try/catch syntax for exception handling that should look more familiar to users of other languages while handling the issues surrounding using block eval and testing of the special $@ variable. If you need to remain compatible with older versions of Perl (back to v5.14), just use the Feature::Compat::Try module from CPAN to automatically select either v5.34’s native try/catch or a subset of the functionality provided by Syntax::Keyword::Try.

Pluggable keywords

The abovementioned Syntax::Keyword::Try was made possible by the introduction of a pluggable keyword mechanism in 2010’s Perl v5.12. So was the Future::AsyncAwait asynchronous programming library and the Object::Pad testbed for new object-oriented Perl syntax. If you’re handy with C and Perl’s XS glue language, check out Paul “LeoNerd” Evans’ XS::Parse::Keyword module to get a leg up on developing your own syntax module.

Define packages with versions and blocks

Perl v5.12 also helped reduce clutter by enabling a package namespace declaration to also include a version number, instead of requiring a separate our $VERSION = ...; v5.14 further refined packages to be specified in code blocks, so a namespace declaration can be the same as a lexical scope. Putting the two together gives you:
```
package Local::NewHotness v1.2.3 {
    ...
}
```
Instead of:
```
{
    package Local::OldAndBusted;
    use version 0.77; our $VERSION = version->declare("v1.2.3");
    ...
}
```
I know which I’d rather do. (Though you may want to also use Syntax::Construct qw(package-version package-block); to help along with older installations as described above.)

The // defined-or operator

This is an easy win from Perl v5.10:
```
defined $foo ? $foo : $bar  # replace this
$foo // $bar                # with this
```
And:
```
$foo = $bar unless defined $foo  # replace this
$foo //= $bar                    # with this
```
Perfect for assigning defaults to variables.

state variables only initialize once

Speaking of variables, ever want one to keep its old value the next time a scope is entered, like in a sub? Declare it with state instead of my. Before Perl v5.10, you needed to use a closure instead.

Save some typing with say

Perl v5.10’s bumper crop of enhancements also included the say function, which handles the common use case of printing a string or list of strings with a newline. It’s less noise in your code and saves you four characters. What’s not to love?

Note unimplemented code with ...

The ... ellipsis statement (colloquially “yada-yada”) gives you an easy placeholder for yet-to-be-implemented code. It parses OK but will throw an exception if executed. Hopefully, your test coverage (or at least static analysis) will catch it before your users do.

Loop and enumerate arrays with each, keys, and values

The each, keys, and values functions have always been able to operate on hashes. Perl v5.12 and above make them work on arrays, too. The latter two are mainly for consistency, but you can use each to iterate over an array’s indices and values at the same time:
```
while (my ($index, $value) = each @array) {
    ...
}
```
This can be problematic in non-trivial loops, but I’ve found it helpful in quick scripts and one-liners.

delete local hash (and array) entries

Ever needed to delete an entry from a hash (e.g, an environment variable from %ENV or a signal handler from %SIG) just inside a block? Perl v5.12 lets you do that with delete local.

Paired hash slices

Jumping forward to 2014’s Perl v5.20, the new %foo{'bar', 'baz'} syntax enables you to slice a subset of a hash with its keys and values intact. Very helpful for cherry-picking or aggregating many hashes into one. For example:
```
my %args = (
    verbose => 1,
    name    => 'Mark',
    extra   => 'pizza',
);
# don't frob the pizza
$my_object->frob( %args{ qw(verbose name) };
```
Paired array slices

Not to be left out, you can also slice arrays in the same way, in this case returning indices and values:
```
my @letters = 'a' .. 'z';
my @subset_kv = %letters[16, 5, 18, 12];
# @subset_kv is now (16, 'p', 5, 'e', 18, 'r', 12, 'l')
```
More readable dereferencing

Perl v5.20 introduced and v5.24 de-experimentalized a more readable postfix dereferencing syntax for navigating nested data structures. Instead of using {braces} or smooshing sigils to the left of identifiers, you can use a postfixed sigil-and-star:
```
push @$array_ref,    1, 2, 3;  # noisy
push @{$array_ref},  1, 2, 3;  # a little easier
push $array_ref->@*, 1, 2, 3;  # read from left to right
```
So much of web development is slinging around and picking apart complicated data structures via JSON, so I welcome anything like this to reduce the cognitive load.

when as a statement modifier

Starting in Perl v5.12, you can use the experimental switch feature’s when keyword as a postfix modifier. For example:
```
for ($foo) {
    $a =  1 when /^abc/;
    $a = 42 when /^dna/;
    ...
}
```
But I don’t recommend when, given, or given’s smartmatch operations as they were retconned as experiments in 2013’s Perl v5.18 and have remained so due to their tricky behavior. I wrote about some alternatives using stable syntax back in February.

Simple class inheritance with use parent

Sometimes in older object-oriented Perl code, you’ll see use base as a pragma to establish inheritance from another class. Older still is the direct manipulation of the package’s special @ISA array. In most cases, both should be avoided in favor of use parent, which was added to core in Perl v5.10.1.

Mind you, if you’re following the Perl object-oriented tutorial’s advice and have selected an OO system from CPAN, use its subclassing mechanism if it has one. Moose, Moo, and Class::Accessor’s “antlers” mode all provide an extends function; Object::Pad provides an :isa attribute on its class keyword.

Test for class membership with the isa operator

As an alternative to the isa() method provided to all Perl objects, Perl v5.32 introduced the experimental isa infix operator:
```
$my_object->isa('Local::MyClass')
# or
$my_object isa Local::MyClass
```
The latter can take either a bareword class name or string expression, but more importantly, it’s safer as it also returns false if the left argument is undefined or isn’t a blessed object reference. The older isa() method will throw an exception in the former case and might return true if called as a class method when $my_object is actually a string of a class name that’s the same as or inherits from isa()’s argument.

Lexical subroutines

Introduced in Perl v5.18 and de-experimentalized in 2017’s Perl v5.26, you can now precede sub declarations with my, state, or our. One use of the first two is truly private functions and methods, as described in this 2018 Dave Jacoby blog and as part of Neil Bowers’ 2014 survey of private function techniques.

Subroutine signatures

I’ve written and presented extensively about signatures and alternatives over the past year, so I won’t repeat that here. I’ll just add that the Perl 5 Porters development mailing list has been making a concerted effort over the past month to hash out the remaining issues towards rendering this feature non-experimental. The popular Mojolicious real-time web framework also provides a shortcut for enabling signatures and uses them extensively in examples.

Indented here-documents with <<~

Perl has had shell-style “here-document” syntax for embedding multi-line strings of quoted text for a long time. Starting with Perl v5.26, you can precede the delimiting string with a ~ character and Perl will both allow the ending delimiter to be indented as well as strip indentation from the embedded text. This allows for much more readable embedded code such as runs of HTML and SQL. For example:
```
if ($do_query) {
    my $rows_deleted = $dbh->do(<<~'END_SQL', undef, 42);
      DELETE FROM table
      WHERE status = ?
      END_SQL
    say "$rows_deleted rows were deleted."; 
}
```
More readable chained comparisons

When I learned math in school, my teachers and textbooks would often describe multiple comparisons and inequalities as a single expression. Unfortunately, when it came time to learn programming every computer language I saw required them to be broken up with a series of and (or &&) operators. With Perl v5.32, this is no more:
```
if ( $x < $y && $y <= $z ) { ... }  # old way
if ( $x < $y <= $z )       { ... }  # new way
```
It’s more concise, less noisy, and more like what regular math looks like.

Self-documenting named regular expression captures

Perl’s expressive regular expression matching and text-processing prowess are legendary, although overuse and poor use of readability enhancements often turn people away from them (and Perl in general). We often use regexps for extracting data from a matched pattern. For example:
```
if ( /Time: (..):(..):(..)/ ) {  # parse out values
    say "$1 hours, $2 minutes, $3 seconds";
}
```
Named capture groups, introduced in Perl v5.10, make both the pattern more obvious and retrieval of its data less cryptic:
```
if ( /Time: (?<hours>..):(?<minutes>..):(?<seconds>..)/ ) {
    say "$+{hours} hours, $+{minutes} minutes, $+{seconds} seconds";
}
```
More readable regexp character classes

The /x regular expression modifier already enables better readability by telling the parser to ignore most whitespace, allowing you to break up complicated patterns into spaced-out groups and multiple lines with code comments. With Perl v5.26 you can specify /xx to also ignore spaces and tabs inside [bracketed] character classes, turning this:
```
/[d-eg-i3-7]/
/[!@"#$%^&*()=?<>']/
```
…into this:
```
/ [d-e g-i 3-7]/xx
/[ ! @ " # $ % ^ & * () = ? <> ' ]/xx
```
Set default regexp flags with the re pragma

Beginning with Perl v5.14, writing use re '/xms'; (or any combination of regular expression modifier flags) will turn on those flags until the end of that lexical scope, saving you the trouble of remembering them every time.

Non-destructive substitution with s///r and tr///r

The s/// substitution and tr/// transliteration operators typically change their input directly, often in conjunction with the =~ binding operator:
```
s/foo/bar/;  # changes the first foo to bar in $_
$baz =~ s/foo/bar/;  # the same but in $baz
```
But what if you want to leave the original untouched, such as when processing an array of strings with a map? With Perl v5.14 and above, add the /r flag, which makes the substitution on a copy and returns the result:
```
my @changed = map { s/foo/bar/r } @original;
```
Unicode case-folding with fc for better string comparisons

Unicode and character encoding in general are complicated beasts. Perl has handled Unicode since v5.6 and has kept pace with fixes and support for updated standards in the intervening decades. If you need to test if two strings are equal regardless of case, use the fc function introduced in Perl v5.16.

Safer processing of file arguments with <<>>

The <> null filehandle or “diamond operator” is often used in while loops to process input per line coming either from standard input (e.g., piped from another program) or from a list of files on the command line. Unfortunately, it uses a form of Perl’s open function that interprets special characters such as pipes (|) that would allow it to insecurely run external commands. Using the <<>> “double diamond” operator introduced in Perl v5.22 forces open to treat all command-line arguments as file names only. For older Perls, the perlop documentation recommends the ARGV::readonly CPAN module.

Safer loading of Perl libraries and modules from @INC

Perl v5.26 removed the ability for all programs to load modules by default from the current directory, closing a security vulnerability originally identified and fixed as CVE-2016–1238 in previous versions’ included scripts. If your code relied on this unsafe behavior, the v5.26 release notes include steps on how to adapt.

HTTP::Tiny simple HTTP/1.1 client included

To bootstrap access to CPAN on the web in the possible absence of external tools like curl or wget, Perl v5.14 began including the HTTP::Tiny module. You can also use it in your programs if you need a simple web client with no dependencies.

Test2: The next generation of Perl testing frameworks

Forked and refactored from the venerable Test::Builder (the basis for the Test::More library that many are familiar with), Test2 was included in the core module library beginning with Perl v5.26. I’ve experimented recently with using the Test2::Suite CPAN library instead of Test::More and it looks pretty good. I’m also intrigued by Test2::Harness’ support for threading, forking, and preloading modules to reduce test run times.

Task::Kensho: Where to start for recommended Perl modules

This last item may not be included when you install Perl, but it’s where I turn for a collection of well-regarded CPAN modules for accomplishing a wide variety of common tasks spanning from asynchronous programming to XML. Use it as a starting point or interactively select the mix of libraries appropriate to your project.

And there you have it: a selection of 34 features, enhancements, and improvements for the first 34 years of Perl. What’s your favorite? Did I miss anything? Let me know in the comments.
December 21, 2021
Perl can do that now!
Last week saw the release of Perl 5.34.0 (you can get it here), and with it comes a year’s worth of new features, performance enhancements, bug fixes, and other improvements. It seems like a good time to highlight some of my favorite changes over the past decade and a half, especially for those with more dated knowledge of Perl. You can always click on the headers below for the full releases’ perldelta pages.

Perl 5.10 (2007)

This was a big release, coming as it did over five years after the previous major 5.8 release. Not that Perl developers were idle—but it wouldn’t be until version 5.14 that the language would adopt a steady yearly release cadence.

Due to the build-up time, many core enhancements were made but the most important was arguably the feature pragma, enabling the addition of new syntax that would otherwise break Perl’s backward compatibility. 5.10 also introduced the defined-or operator (//), state variables that persist their previous value, the say function for automatically appending a newline on output (so much saved typing), and a large collection of improvements to regular expressions. In addition, this release introduced smart matching (~~), though version 5.18 would eventually relegate it to experimental status.

Perl 5.12 (2010)

This release also saw many new features added, but if I had to pick one marquee item it would be experimental support for pluggable keywords, which enabled authors to extend the language itself without modifying the core. Previously one would either use plain functions, hacky source filters, or the deprecated Devel::Declare module to simulate this functionality. CPAN authors would go on to create all kinds of new syntax, sometimes prototyping features that would eventually make their way into core.

Perl 5.14 (2011)

5.14 had a big list of enhancements, including Unicode 6.0 support and a gaggle of regular expression features. My favorite of these was the /r switch for non-destructive substitutions.

But as the first yearly cadence release, the changes in policy took center stage. The Perl 5 Porters (p5p) explicitly committed to supporting the two most recent stable release series, providing security patches only for release series occurring in the past three years. They also defined an explicit compatibility and deprecation policy, with definitions for features that may be experimental, deprecated, discouraged, and removed.

Perl 5.16 (2012)

Another year, another version bump. This time the core enhancements were all over the map (although no enhancements to the map function 😀 ).

May I highlight another documentation change, though? The perlootut Object-Oriented Programming in Perl Tutorial replaced the old perltoot, perltooc, perlboot, and perlbot pages, providing an introduction to object-oriented design concepts before strongly recommending the use of one of the OO systems from CPAN. Mentioned are Moose, its alternative Mouse, Class::Accessor, Object::Tiny, and Role::Tiny’s usage with the latter two. Later versions of perlootut would recommend Moo rather than Mouse.

Perl 5.18 (2013)

As mentioned earlier, Perl 5.18 rendered smartmatch experimental, as well as lexical use of the $_ variable. With these came a new category of warnings for experimental features and a method for overriding such warnings feature-by-feature. Fitting in with the security and safety theme, hashes were overhauled to randomize key/value order, increasing their resistance to algorithmic complexity attacks.

But it wasn’t all fencing in bad behavior. Lexical subroutines made their first (experimental) appearance, and although I confess I haven’t had much call for them in my work, others have come up with some interesting uses. Four years later they became non-experimental.

Perl 5.20 (2014)

Three new syntax features arrived in 2014: experimental subroutine signatures (of which I’ve written more about here), key/value hash slices and index/value array slices, and experimental postfix dereferencing. This last enables cleaner left-to-right syntax when dereferencing variables:
- @{ $array_ref } becomes $array_ref->@*
- %{ $hash_ref } becomes $hash_ref->%*
- Etc.
Postfix dereferencing became non-experimental in Perl 5.24, and vigorous discussion continues on subroutine signatures’ future.

Perl 5.22 (2015)

Speaking of subroutine signatures, their location moved to between the subroutine name (if any) and the attribute list (if any). Previously they appeared after attributes. The lesson? Remain conscious of experimental features in your code, and be prepared to make changes when upgrading.

In addition to the enhancements, security updates, performance fixes, and deprecations, developers removed the historically notable CGI module. First added to core in 1997 in recognition of its critical role in enabling web development, it’s been supplanted by better alternatives on CPAN.

Perl 5.24 (2016)

Perl 5.20’s postfix dereferencing was no longer experimental, and developers removed both lexical $_ and autodereferencing on calls to push, pop, shift, unshift, splice, keys, values, and each.

Perl 5.26 (2017)

The incorporation of experimental features continued, with lexical subroutines moving into full support. I like the added readability enhancements, though: indented here-documents; the /xx regular expression modifier for tabs and spaces in character classes; and @{^CAPTURE}, %{^CAPTURE}, and %{^CAPTURE_ALL} for regexp matches with a little more self-documentation.

Perl 5.28 (2018)

Experimental subroutine signature and attribute ordering flipped back to its Perl 5.20 sequence of attributes-then-signature. Bit of a rollercoaster ride on this one. You could do worse than using something like Type::Params until this settles and get a wide variety of type constraints in the bargain.

Perl 5.30 (2019)

Pour one out for AWK and Fortran programmers migrating to Perl: the $[variable for setting the lower bound of arrays could no longer be set to anything other than zero. This had a long deprecation cycle starting in Perl 5.12.

Perl 5.32 (2020)

In 2020 Perl’s development moved to GitHub. And once again, I’m going to highlight readability enhancements: the experimental isa operator could be used to say:
```
if ( $obj isa Some::Class ) { ... }
```
instead of
```
use Scalar::Util 'blessed';
if ( blessed($obj) and $obj->isa('Some::Class') { ... }
```
You could also chain comparison operators, leading to the more mathematically concise if ( $x < $y <= $z ) {...} rather than if ( $x < $y and $y <= $z ) {...}.

Perl 5.34 (2021)

Finally, we come to last week’s release and its introduction of experimental try/catch exception handling syntax. If you need to support earlier versions of Perl back to 5.14, you can use Feature::Compat::Try. Earlier this year I interviewed the feature and module’s author, Paul “LeoNerd” Evans, for Perl.com. This year also marked the debut of Perl’s new governance model with the appointment of a Core Team and a three-member Steering Council.

What are some of your favorite Perl improvements over the years? Check out the perlhist document for a detailed chronology and refresher with the various perldelta pages and leave me a comment below.
May 25, 2021
Switching up my switches in Perl
Pretty soon after I started writing Perl in 1994, I noticed that it lacked a construct often found in other languages: the switch statement. Not to worry, though—you can achieve the same effect with a cascading series of if-elsif statements, right?

Well, no, you shouldn’t do that, especially if the chain is super-long. There’s even a perlcritic policy about it, which suggests that you use given and when instead.

But given and when (and the smartmatch operator they imply, ~~) are still experimental, with behavior subject to change. So, what’s a responsible developer to do?

The answer is to use the for statement as a topicalizer, which is a fancy way of saying it assigns its expression to $_. You can then use things that act on $_ to your heart’s content, like regular expressions. Here’s an example:
```
for ($foo) {
    /^abc/ and do {
        ...
        last;
    };
    /^def/ and do {
        ...
        last;
    };
    # FALL THRU
    ...
}
```
This will cover a lot of cases (ha-ha, see what I did there? A lot of languages use a case statement… oh, never mind.) And if all you’re doing is exact string matching, there’s no need to bring in regexps. You can use a hash as a lookup table:
```
my %lookup = (
    foo => sub { ... },
    bar => sub { ... },
);
$lookup{$match}->();
```
EDIT: If every alternative is an assignment to the same variable, a ternary table is another possibility. This is a chained set of ternary conditional (? :) operators arranged for readability. I first heard about this technique from Damian Conway’s Perl Best Practices (2005).
```
           # Name format                 # Salutation
my $salute = $name eq ''                 ? 'Dear Customer'
           : $name =~ /(Mrs?[.]?\s+\S+)/ ? "Dear $1"
           : $name =~ /(.*),\s+Ph[.]?D/  ? "Dear Dr. $1"
           :                               "Dear $name"
           ;
```
Note that although this is just as inefficient as a cascaded-if/elsif, it’s clearer that it’s a single assignment. It’s also more compact and reads like a table with columns of matches and alternatives.

Any of these patterns are preferable to cascading if/elsifs. And if you want to check the development of given, when, and ~~, see this issue on GitHub. The last comment was eight years ago, though, so I wouldn’t hold my breath.
February 14, 2021