happy man funny sticking tongue out

Over the past two years, I’ve got­ten back into play­ing Dungeons & Dragons, the famous table­top fan­ta­sy role-​playing game. As a soft­ware devel­op­er and musi­cian, one of my favorite char­ac­ter class­es to play is the bard, a mag­i­cal and inspir­ing per­former or word­smith. The list of basic bardic spells includes Vicious Mockery, enchant­i­ng ver­bal barbs that have the pow­er to psy­chi­cal­ly dam­age and dis­ad­van­tage an oppo­nent even if they don’t under­stand the words. (Can you see why this is so appeal­ing to a coder?)

Mocking has a role to play in soft­ware test­ing as well, in the form of mock objects that sim­u­late parts of a sys­tem that are too brit­tle, too slow, too com­pli­cat­ed, or oth­er­wise too finicky to use in real­i­ty. They enable dis­crete unit test­ing with­out rely­ing on depen­den­cies exter­nal to the code being test­ed. Mocks are great for data­bas­es, web ser­vices, or oth­er net­work resources where the goal is to test what you wrote, not what’s out in the cloud” somewhere.

Speaking of web ser­vices and mock­ing, one of my favorites is the long-​running FOAAS (link has lan­guage not safe for work), a sur­pris­ing­ly expan­sive RESTful insult ser­vice. There’s a cor­re­spond­ing Perl client API, of course, but what I was miss­ing was a handy Perl script to call that API from the ter­mi­nal com­mand line. So I wrote the fol­low­ing over Thanksgiving break, try­ing to keep it sim­ple while also show­ing the basics of mock­ing such an API. It also demon­strates some new­er Perl syn­tax and test­ing tech­niques as well as bri­an d foys mod­uli­no con­cept from Mastering Perl (sec­ond edi­tion, 2014) that mar­ries script and mod­ule into a self-​contained exe­cutable library.

#!/usr/bin/env perl

package Local::CallFOAAS;  # this is a modulino
use Test2::V0;             # enables strict, warnings, utf8

# declare all the new stuff we're using
use feature qw(say state);
use experimental qw(isa postderef signatures);
use Feature::Compat::Try;
use Syntax::Construct qw(non-destructive-substitution);

use WebService::FOAAS ();
use Package::Stash;
use Exception::Class (
    NoMethodException => {
        alias  => 'throw_no_method',
        fields => 'method',
    },
    ServiceException => { alias => 'throw_service' },
);

my $foaas = Package::Stash->new('WebService::FOAAS');

my $run_as =
    !!$ENV{CPANTEST}       ? 'test'
  : !defined scalar caller ? 'run'
  :                          undef;
__PACKAGE__->$run_as(@ARGV) if defined $run_as;

sub run ( $class, @args ) {
    try { say $class->call_method(@args) }
    catch ($e) {
        die 'No method ', $e->method, "\n"
          if $e isa NoMethodException;
        die 'Service error: ', $e->error, "\n"
          if $e isa ServiceException;
        die "$e\n";
    }
    return;
}

# Utilities

sub methods ($) {
    state @methods = sort map s/^foaas_(.+)/$1/r,
      grep /^foaas_/, $foaas->list_all_symbols('CODE');
    return @methods;
}

sub call_method ( $class, $method = '', @args ) {
    state %methods = map { $_ => 1 } $class->methods();
    throw_no_method( method => $method )
      unless $methods{$method};
    return do {
        try { $foaas->get_symbol("&$method")->(@args) }
        catch ($e) { throw_service( error => $e ) }
    };
}

# Testing

sub test ( $class, @ ) {
    state $stash = Package::Stash->new($class);
    state @tests = sort grep /^_test_/,
      $stash->list_all_symbols('CODE');

    for my $test (@tests) {
        subtest $test => sub {
            try { $class->$test() }
            catch ($e) { diag $e }
        };
    }
    done_testing();
    return;
}

sub _test_can ($class) {
    state @subs = qw(run call_method methods test);
    can_ok( $class, \@subs, "can do: @subs" );
    return;
}

sub _test_methods ($class) {
    my $mock = mock 'WebService::FOAAS' => ( track => 1 );

    for my $method ( $class->methods() ) {
        $mock->override( $method => 1 );

        ok lives { $class->call_method($method) },
          "$method lives";
        ok scalar $mock->sub_tracking->{$method}->@*,
          "$method called";
    }
    return;
}

sub _test_service_failure ($class) {
    my $mock = mock 'WebService::FOAAS';

    for my $method ( $class->methods() ) {
        $mock->override( $method => sub { die 'mocked' } );

        my $exception =
          dies { $class->call_method($method) };
        isa_ok $exception, ['ServiceException'],
          "$method throws ServiceException on failure";
        like $exception->error, qr/^mocked/,
          "correct error in $method exception";
    }
    return;
}

1;

Let’s walk through the code above.

Preliminaries

First, there’s a gener­ic she­bang line to indi­cate that Unix and Linux sys­tems should use the perl exe­cutable found in the user’s PATH via the env com­mand. I declare a pack­age name (in the Local:: name­space) so as not to pol­lute the default main pack­age of oth­er scripts that might want to require this as a mod­ule. Then I use the Test2::V0 bun­dle from Test2::Suite since the embed­ded test­ing code uses many of its func­tions. This also has the side effect of enabling the strict, warn­ings, and utf8 prag­mas, so there’s no need to explic­it­ly use them here.

(Why Test2 instead of Test::More and its deriv­a­tives and add-​ons? Both are main­tained by the same author, who rec­om­mends the for­mer. I’m see­ing more and more mod­ules using it, so I thought this would be a great oppor­tu­ni­ty to learn.)

I then declare all the new-​ish Perl fea­tures I’d like to use that need to be explic­it­ly enabled so as not to sac­ri­fice back­ward com­pat­i­bil­i­ty with old­er ver­sions of Perl 5. As of this writ­ing, some of these fea­tures (the isa class instance oper­a­tor, named argu­ment sub­rou­tine sig­na­tures, and try/​catch excep­tion han­dling syn­tax) are con­sid­ered experimental, with the lat­ter enabled in old­er ver­sions of Perl via the Feature::Compat::Try mod­ule. The friend­lier post­fix deref­er­enc­ing syn­tax was main­lined in Perl ver­sion 5.24, but ver­sions 5.20 and 5.22 still need it exper­i­men­tal. Finally, I use Syntax::Construct to announce the /r flag for non-​destructive reg­u­lar expres­sion text sub­sti­tu­tions intro­duced in ver­sion 5.14.

Next, I bring in the afore­men­tioned FOAAS Perl API with­out import­ing any of its func­tions, Package::Stash to make metapro­gram­ming eas­i­er, and a cou­ple of excep­tion class­es so that the com­mand line func­tion and oth­er con­sumers might bet­ter tell what caused a fail­ure. In prepa­ra­tion for the meth­ods below dynam­i­cal­ly dis­cov­er­ing what func­tions are pro­vid­ed by WebService::FOAAS, I gath­er up its sym­bol table (or stash) into the $foaas variable.

The next block deter­mines how, if at all, I’m going to run the code as a script. If the CPANTEST envi­ron­ment vari­able is set, I’ll call the test class method sub, but if there’s no sub­rou­tine call­ing me I’ll exe­cute the run class method. Either will receive the com­mand line argu­ments from @ARGV. If nei­ther of these con­di­tions is true, do noth­ing; the rest of the code is method declarations.

Modulino methods, metaprogramming, and exceptions

The first of these is the run method. It’s a thin wrap­per around the call_method class method detailed below, either out­putting its result or dieing with an appro­pri­ate error depend­ing on the class of excep­tion thrown. Although I chose not to write tests for this out­put, future tests might call this method and catch these rethrown excep­tions to match against them. The mes­sages end with a \n new­line char­ac­ter so die knows not to append the cur­rent script line number.

Next is a util­i­ty method called methods that uses Package::Stash’s list_all_symbols to retrieve the names of all named CODE blocks (i.e., subs) from WebService::FOAAS’s sym­bol table. Reading from right to left, these are then fil­tered with grep to only find those begin­ning in foaas_ and then trans­formed with map to remove that pre­fix. The list is then sorted and stored in a state vari­able and returned so it need not be ini­tial­ized again.

(As an aside, although perlcritic stern­ly warns against it I’ve cho­sen the expres­sion forms of grep and map here over their block forms for sim­plic­i­ty’s sake. It’s OK to bend the rules if you have a good reason.)

sub call_method is where the real action takes place. Its para­me­ters are the class that called it, the name of a FOAAS $method (default­ed to the emp­ty string), and an array of option­al argu­ments in @args. I build a hash or asso­cia­tive array from the ear­li­er methods method which I then use to see if the passed method name is one I know about. If not, I throw a NoMethodException using the throw_no_method alias func­tion cre­at­ed when I used Exception::Class at the begin­ning. Using a func­tion instead of NoMethodException->throw() means that it’s checked at com­pile time rather than run­time, catch­ing typos.

I get the sub­rou­tine (denot­ed by a & sig­il) named by $method from the $foaas stash and pass it any fur­ther received argu­ments from @args. If that WebService::FOAAS sub­rou­tine throws an excep­tion it’ll be caught and re-​thrown as a ServiceException; oth­er­wise call_method returns the result. It’s up to the caller to deter­mine what, if any­thing, to do with that result or any thrown exceptions.

Testing the modulino with mocks

This is where I start using those Test2::Suite tools I men­tioned at the begin­ning. The test class method starts by build­ing a fil­tered list of all subs begin­ning with _test_ in the cur­rent class, much like methods did above with WebService::FOAAS. I then loop through that list of subs, run­ning each as a subtest con­tain­ing a class method with any excep­tions report­ed as diag­nos­tics.

The rest of the mod­uli­no is sub­test meth­ods, start­ing with a sim­ple _test_can san­i­ty check for the pub­lic meth­ods in the class. Following that is _test_methods, which starts by mocking the WebService::FOAAS pack­age and telling Test2::Mock I want to track any added, over­rid­den, or set subs. I then loop through all the method names returned by the methods class method, overrideing each one to return a sim­ple true val­ue. I then test pass­ing those names to call_method and use the hash ref­er­ence returned by sub_tracking to check that the over­rid­den sub was called. This seems a lot sim­pler than the Test::Builder-based mock­ing libraries I’ve tried like Test::MockModule and Test::MockObject.

_test_service_failure acts in much the same way, check­ing that call_method cor­rect­ly throws ServiceExceptions if the wrapped WebService::FOAAS func­tion dies. The main dif­fer­ence is that the mocked WebService::FOAAS subs are now over­rid­den with a code ref­er­ence (sub { die 'mocked' }), which call_method uses to pop­u­late the rethrown ServiceExceptions error field.

Wrapping up

With luck, this arti­cle has giv­en you some ideas, whether it’s in mak­ing scripts (per­haps lega­cy code) testable to improve them, or writ­ing bet­ter unit tests that mock depen­den­cies, or delv­ing a lit­tle into metapro­gram­ming so you can dynam­i­cal­ly sup­port and test new fea­tures of said depen­den­cies. I hope you haven’t come away too offend­ed, at least. Let me know in the com­ments what you think.

Look, I get it. You don’t like the Perl pro­gram­ming lan­guage or have oth­er­wise dis­re­gard­ed it as dead.” (Or per­haps you haven’t, in which case please check out my oth­er blog posts!) It has weird noisy syn­tax, mix­ing reg­u­lar expres­sions, sig­ils on vari­able names, var­i­ous braces and brack­ets for data struc­tures, and a menagerie of cryp­tic spe­cial vari­ables. It’s old: 34 years in December, with a his­to­ry of (some­times ama­teur) devel­op­ers that have used and abused that syn­tax to ship code of ques­tion­able qual­i­ty. Maybe you grudg­ing­ly accept its util­i­ty but think it should die grace­ful­ly, main­tained only to run lega­cy applications.

But you know what? Perl’s still going. It’s had a steady cadence of year­ly releas­es for the past decade, intro­duc­ing new fea­tures and fenc­ing in bad behav­ior while main­tain­ing an admirable lev­el of back­ward com­pat­i­bil­i­ty. Yes, there was a too-​long adven­ture devel­op­ing what start­ed as Perl 6, but that lan­guage now has its own iden­ti­ty as Raku and even has facil­i­ties for mix­ing Perl with its native code or vice versa.

And then there’s CPAN, the Comprehensive Perl Archive Network: a continually-​updated col­lec­tion of over 200,000 open-​source mod­ules writ­ten by over 14,000 authors, the best of which are well-​tested and ‑doc­u­ment­ed (apply­ing peer pres­sure to those that fall short), pre­sent­ed through a search engine and front-​end built by scores of con­trib­u­tors. Through CPAN you can find dis­tri­b­u­tions for things like:

All of this is avail­able through a mature instal­la­tion tool­chain that doesn’t break from month to month.

Finally and most impor­tant­ly, there’s the glob­al Perl com­mu­ni­ty. The COVID-​19 pan­dem­ic has put a damper on the hun­dreds of glob­al Perl Mongers groups’ mee­tups, but that hasn’t stopped the year­ly Perl and Raku Conference from meet­ing vir­tu­al­ly. (In the past there have also been year­ly European and Asian con­fer­ences, occa­sion­al for­ays into South America and Russia, as well as hackathons and work­shops world­wide.) There are IRC servers and chan­nels for chat, mail­ing lists galore, blogs (yes, apart from this one), and a quirky social net­work that pre­dates Facebook and Twitter.

So no, Perl isn’t dead or even dying, but if you don’t like it and favor some­thing new­er, that’s OK! Technologies can coex­ist on their own mer­its and advo­cates of one don’t have to beat down their con­tem­po­raries to be suc­cess­ful. Perl hap­pens to be battle-​tested (to bor­row a term from my friend Curtis Ovid” Poe), it runs large parts of the Web (speak­ing from direct and ongo­ing expe­ri­ence in the host­ing busi­ness here), and it’s still evolv­ing to meet the needs of its users.

In March I wrote The Perl debug­ger can be your super­pow­er, intro­duc­ing the step debug­ger as a bet­ter way to debug your Perl code rather than lit­ter­ing your source with tem­po­rary print state­ments or log­ging. I use the debug­ger all the time, and I’ve real­ized that some more tech­niques are worth covering.

Although I men­tioned a caveat when debug­ging web appli­ca­tions, our apps at work all adhere to the Perl Web Server Gateway Interface (PSGI) spec­i­fi­ca­tion and thus we can use tools like Test::WWW::Mechanize::PSGI or Plack::Test to run tests and debug­ging ses­sions in the same Perl process. (Mojolicious users can use some­thing like Test::Mojo for the same effect.)

To demon­strate, let’s get start­ed with some­thing like this which tests that a giv­en route (/say-hello) returns a cer­tain JSON struc­ture ({"message": "Hello world!"}):

#!/usr/bin/env perl

use Test::Most;
use Test::WWW::Mechanize::PSGI;
use JSON::MaybeXS;
use Local::MyApp; # name of app's main module

my $mech = Test::WWW::Mechanize::PSGI->new(
    # a Dancer2 app, so to_app returns a PSGI coderef
    app => Local::MyApp->to_app(),
);
$mech->get_ok('/say-hello');
lives_and {
    my $json = decode_json($mech->content);
    cmp_deeply( $json, {message => 'Hello world!'} );
} 'message is Hello world!';

done_testing;

All very fine and well, but what hap­pens if that route starts return­ing a dif­fer­ent mes­sage or worse, invalid out­put that caus­es decode_json to fail? Eventually, you’ll rewrite the test in the script to out­put the offend­ing con­tent when some­thing goes wrong, but right now you want to suss out the root cause.

Debuggers have the con­cept of break­points, which are flags that tell the debug­ger to stop at a cer­tain line of code and wait for instruc­tions. We can set them while run­ning the debug­ger with the b com­mand or con­tin­ue to a one-​time break­point with the c com­mand, or we can insert them into the code our­selves before run­ning it through the debug­ger in the first place.

Add this line right after the lives_and { line:

$DB::single = 1;

This sim­u­lates hav­ing typed the s com­mand in the debug­ger at that line, stop­ping exe­cu­tion at that point. Run our test with per­l’s -d option, and then type c to con­tin­ue to that breakpoint:

$ perl -d -Ilib t/test_psgi.t

Loading DB routines from perl5db.pl version 1.60
Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

[Local::MyApp:7170] core @2021-07-06 07:33:22> Built config from files: /Users/mgardner/Projects/blog/myapp/config.yml /Users/mgardner/Projects/blog/myapp/environments/development.yml in (eval 310)[/Users/mgardner/.plenv/versions/5.34.0/lib/perl5/site_perl/5.34.0/Sub/Quote.pm:3] l. 910
Test2::API::CODE(0x7ffabea39ee8)(/Users/mgardner/.plenv/versions/5.34.0/lib/perl5/site_perl/5.34.0/Test2/API.pm:71):
71:	    INIT { eval 'END { test2_set_is_end() }; 1' or die [email protected] }

  DB<1> c

[...]
ok 1 - GET /say-hello
main::CODE(0x7f8069caf2c8)(t/test_psgi.t:14):
15:	    my $json = decode_json($mech->content);

  DB<1> 

From here we can exam­ine vari­ables, set oth­er break­points, or even exe­cute arbi­trary lines of code. Let’s see what became of that HTTP GET request:

  DB<1> x $mech->content

0  '{"error":"Undefined subroutine &Local::MyApp::build_frog called at lib/Local/MyApp.pm line 11.\\n"}'

  DB<2> 

Aha, some­thing has returned some dif­fer­ent JSON indi­cat­ing an error. Let’s look at the lines around (10−20) the offend­ing line (11):

  DB<2> f lib/Local/MyApp.pm

  DB<3> l 10-20

10:	        my $method = 'build_frog';
11:	        $method->();
12 	    }
13:	    catch ($e) {
14:	        send_as JSON => {error => $e};
15 	    }
16:	    send_as JSON => {message => 'Hello world!'};
17:	};
18
19 	sub build_frob {
20:	    return;

  DB<4>

Yep, a typo on line 11, and one that was­n’t caught at com­pile time since it’s gen­er­at­ed at runtime.

Just to be sure (and to demon­strate some oth­er cool debug­ger fea­tures), let’s set anoth­er break­point while in the debug­ger and then exer­cise that route again. Then we’ll check that $method vari­able against the list of avail­able meth­ods in the Local::MyApp package.

  DB<4> b 11

  DB<5> $mech->get('/say-hello')

[...]
Local::MyApp::CODE(0x7f8066f2db60)(lib/Local/MyApp.pm:11):
11:	        $method->();

  DB<<6>> x $method

0  'build_frog'

  DB<<7>> m Local::MyApp
any
app
body_parameters
build_frob
captures
config
content
[...]
  DB<<8>>

No doubt about it, that vari­able is being set incorrectly.

Quit out of the debug­ger with the q com­mand, make the fix (we prob­a­bly want errors to give some­thing oth­er than an HTTP 200 OK while we’re at it), and re-​run the test:

$ perl -Ilib t/test_psgi.t

[Local::MyApp:8277] core @2021-07-06 07:48:36> Built config from files: /Users/mgardner/Projects/blog/myapp/config.yml /Users/mgardner/Projects/blog/myapp/environments/development.yml in (eval 309) l. 910
Name "DB::single" used only once: possible typo at t/test_psgi.t line 13.
[...]
ok 1 - GET /say-hello
ok 2 - message is Hello world!
1..2

Note that warn­ing about leav­ing $DB::single in there. While harm­less, it’s a good reminder to remove such lines from your code so that they don’t sur­prise you or your team­mates dur­ing future debug­ging sessions.

And that’s it. Note that because we’re using PSGI, we were able to set break­points in our web app code itself and the debug­ger stopped there and enabled us to have a look around. And as you’ve seen, once you’re at a break­point you can switch to dif­fer­ent files, add/​remove more break­points, run arbi­trary code, and more. The perlde­bug doc­u­men­ta­tion page has all the details.

Happy debug­ging! For your ref­er­ence, here’s the full app mod­ule and test script used in this article:

MyApp.pm

package Local::MyApp;
use Dancer2;
use Feature::Compat::Try;

our $VERSION = '0.1';

get '/say-hello' => sub {
    try {
        no strict 'refs';
        my $method = 'build_frob';
        $method->();
    }
    catch ($e) {
        status 'error';
        send_as JSON => {error => $e};
    }
    send_as JSON => {message => 'Hello world!'};
};

sub build_frob {
    return;
}

true;

test_psgi.t

#!/usr/bin/env perl

use Test::Most;
use Test::WWW::Mechanize::PSGI;
use JSON::MaybeXS;
use Local::MyApp; # name of your app's main module goes here

my $mech = Test::WWW::Mechanize::PSGI->new(
    # a Dancer2 app, so to_app returns a PSGI coderef
    app => Local::MyApp->to_app(),
);
$mech->get_ok('/say-hello');
lives_and {
    my $json = decode_json($mech->content);
    cmp_deeply( $json, {message => 'Hello world!'} );
} 'message is Hello world!';

done_testing;

Let’s assume for the moment that you’re writ­ing a Perl mod­ule or appli­ca­tion. You’d like to main­tain some lev­el of soft­ware qual­i­ty (or kwali­tee), so you’re writ­ing a suite of test scripts. Whether you’re writ­ing them first (good for you for prac­tic­ing test-​driven devel­op­ment!) or the appli­ca­tion code is already there, you’ll prob­a­bly be reach­ing for Test::Simple, Test::More, or one of the Test2::Suite bun­dles. With the lat­ter two you’re imme­di­ate­ly con­front­ed with a choice: do you count up the num­ber of tests into a plan, or do you for­sake that in favor of leav­ing a done_testing() call at the end of your test script(s)?

There are good argu­ments for both approach­es. When you first start, you prob­a­bly have no idea how many tests your scripts will con­tain. After all, a test script can be a use­ful tool for design­ing a mod­ule’s inter­face by writ­ing exam­ple code that will use it. Your explorato­ry code would be writ­ten as if the mod­ule or appli­ca­tion was already done, test­ing it in the way you’d like it to work. Not declar­ing a plan makes per­fect sense in this case; just put done_testing() at the end and get back to defin­ing your tests.

You don’t have that option when using Test::Simple, of course — it’s so basic it only has one func­tion (ok()), and you have to pre-​declare how many tests you plan to run when useing the mod­ule, like so:

use Test::Simple tests => 23;

Test::More also sup­ports this form of plan, or you can opt to use its plan func­tion to state the num­ber of tests in your script or subtest. With Test2 you have to use plan. Either way, the plan acts as a sort of meta-​test, mak­ing sure that you exe­cut­ed exact­ly what you intend­ed: no more, no less. While there are sit­u­a­tions where it’s not pos­si­ble to pre­dict how many times a giv­en set of tests should run, I would high­ly sug­gest that in all oth­er cas­es you should clean up” your tests and declare a plan. Later on, if you add or remove tests you’ll imme­di­ate­ly be aware that some­thing has changed and it’s time to tal­ly up a new plan.

What about oth­er Perl test­ing frame­works? They can use plans, too. Here are two examples:

Thoughts? Does declar­ing a test plan make writ­ing tests too inflex­i­ble? Does not hav­ing a plan encour­age bad behav­ior? Tell me what you think in the com­ments below.

Yesterday’s pair pro­gram­ming ses­sion had Gábor Szabó and I thrash­ing around for a bit try­ing to fig­ure out how to get test cov­er­age sta­tis­tics for the appli­ca­tion. The Devel::Cover doc­u­men­ta­tion lists how to run the mod­ule sev­er­al ways, but it does­n’t exact­ly describe how to run prove by itself rather than run­ning a Makefiles tests. I worked out how to do it today, and with the Baughs’ help on Twitter I worked out a few more methods.

All exam­ples below use the bash or zsh com­mand shells and were test­ed on macOS Catalina 10.15.7 run­ning zsh 5.7.1 and Perl 5.32.1. If you’re using some­thing very dif­fer­ent (e.g., Microsoft Windows’ CMD or PowerShell), you may have to set envi­ron­ment vari­ables differently.

Ad-​hoc test coverage

If all you want to do is run one shell com­mand, here it is:

$ prove -vlre 'perl -MDevel::Cover -Ilib' t

This takes advan­tage of proves --exec option (abbre­vi­at­ed as -e) to run a dif­fer­ent exe­cutable for tests. It recur­sive­ly (-r) runs all your tests ver­bose­ly (-v) from the t direc­to­ry while load­ing your appli­ca­tion’s libraries (-l), while the perl exe­cutable uses (-M) Devel::Cover and the lib sub­di­rec­to­ry. I use a sim­i­lar tech­nique when debug­ging tests.

$ HARNESS_PERL_SWITCHES=-MDevel::Cover prove -vlr t

This does almost the same thing as above with­out run­ning a dif­fer­ent exe­cutable. It sets Test::HarnessHARNESS_PERL_SWITCHES envi­ron­ment vari­able for the dura­tion of the prove com­mand. You won’t get the text out­put of your test cov­er­age at the end, though, and will still have to run Devel::Covers cover com­mand to both see the cov­er­age and gen­er­ate web pages.

In a dedicated test session, window, or tab

If you have a ter­mi­nal ses­sion, win­dow, or tab ded­i­cat­ed sole­ly to run­ning your tests, set one of the envi­ron­ment vari­ables above for that session:

$ export HARNESS_PERL_SWITCHES=-MDevel::Cover

Now all of your test scripts will pick up that option. You can add more options by enclos­ing the envi­ron­ment vari­able’s val­ue in 'quotes'. For exam­ple, you might also want to load Devel::NYTProf for code profiling:

$ export HARNESS_PERL_SWITCHES='-MDevel::Cover -MDevel::NYTProf'

Why not PERL5OPT?

Setting the PERL5OPT envi­ron­ment vari­able also sets options for the perl run­ning prove, which means that your test cov­er­age, pro­fil­ing, etc. will pick up proves exe­cu­tion as well as your test scripts.

What about yath?

I don’t know for sure; I don’t use the Test2 suite. But it looks like it has a --cover option for load­ing and pass­ing option to Devel::Cover.

In February I wrote an arti­cle sur­vey­ing excep­tion han­dling in Perl, rec­om­mend­ing that devel­op­ers use Test::Exception to make sure their code behaves as expect­ed. A com­menter on Reddit sug­gest­ed I check out Test::Fatal as an alter­na­tive. What advan­tages does it hold over Test::Exception?

  • It only exports one func­tion com­pared to Test::Exception’s four: exception, which you can then use with the full suite of reg­u­lar Test::More func­tions as well as oth­er test­ing libraries such as Test::Deep.
  • It does­n’t over­ride the caller func­tion or use Sub::Uplevel to hide your test blocks from the call stack, so if your excep­tion returns a stack trace you’ll get out­put from the test as well as the thing throw­ing the excep­tion. The author con­sid­ers this a fea­ture since Sub::Uplevel is twitchy.”

To ease port­ing, Test::Fatal also includes two func­tions, dies_ok and lives_ok, replac­ing Test::Exception’s func­tions of the same names. dies_ok does not pro­vide the excep­tion thrown, though, so if you’re test­ing that you’ll need to use exception along with a TAP-emit­ting func­tion like is() or like().

And that’s it! Either is a valid choice; it comes down to whether you pre­fer one approach over anoth­er. Test::Exception is also includ­ed as part of Test::Mosts require­ments, so if you’re using the lat­ter to reduce boil­er­plate you’ll be get­ting the former.

Postscript:

I’d be remiss if I did­n’t also men­tion Test2::Tools::Exception, which is the pre­ferred way to test excep­tions using the Test2 frame­work. If you’re using Test2, ignore all the above and go straight to Test2::Tools::Exception.

Failure is a uni­ver­sal truth of com­put­ers. Files fail to open, web pages fail to load, pro­grams fail to install, mes­sages fail to arrive. As a devel­op­er you have no choice but to work in a seem­ing­ly hos­tile envi­ron­ment in which bugs and errors lurk around every corner.

Hopefully you find and fix the bugs dur­ing devel­op­ment and test­ing, but even with all bugs squashed excep­tion­al con­di­tions can occur. It’s your job as a Perl devel­op­er to use the tools avail­able to you to han­dle these excep­tions. Here are a few of them.

eval, die and $EVAL_ERROR ([email protected]) (updated)

Perl has a prim­i­tive but effec­tive mech­a­nism for run­ning code that may fail called eval. It runs either a string or block of Perl code, trap­ping any errors so that the enclos­ing pro­gram does­n’t crash. It’s your job then to ignore or han­dle the error; eval will return undef (or an emp­ty list in list con­text) and set the mag­ic vari­able [email protected] to the error string. (You can spell that $EVAL_ERROR if you use the English mod­ule, which you prob­a­bly should to allow for more read­able code.) Here’s a con­trived example:

use English;

eval { $foo / 0; 1 }
  or warn "tried to divide by zero: $EVAL_ERROR";

(Why the 1 at the end of the block? It forces the eval to return true if it suc­ceeds; the or con­di­tion is exe­cut­ed if it returns false.)

What if you want to pur­pose­ful­ly cause an excep­tion, so that an enclos­ing eval (pos­si­bly sev­er­al lay­ers up) can han­dle it? You use die:

use English;

eval { process_file('foo.txt'); 1 }
  or warn "couldn't process file: $EVAL_ERROR";

sub process_file {
    my $file = shift;
    open my $fh, '<', $file
      or die "couldn't read $file: $OS_ERROR";

    ... # do something with $fh
}

It’s worth repeat­ing that as a state­ment: You use excep­tions so that enclos­ing code can decide how to han­dle the error. Contrast this with sim­ply han­dling a func­tion’s return val­ue at the time it’s exe­cut­ed: except in the sim­plest of scripts, that part of the code like­ly has no idea what the error means to the rest of the appli­ca­tion or how to best han­dle the problem.

autodie

Since many of Perl’s built-​in func­tions (like open) return false or oth­er val­ues on fail­ure, it can be tedious and error-​prone to make sure that all of them report prob­lems as excep­tions. Enter autodie, which will help­ful­ly replace the func­tions you choose with equiv­a­lents that throw excep­tions. Introduced in Perl 5.10.1, it only affects the enclos­ing code block, and even goes so far as to set $EVAL_ERROR to an object that can be queried for more detail. Here’s an example:

use English;
use autodie; # defaults to everything but system and exec

eval { open my $fh, '<', 'foo.txt'; 1 } or do {
    if ($EVAL_ERROR
      and $EVAL_ERROR->isa('autodie::exception') {
        warn 'Error from open'
          if $EVAL_ERROR->matches('open');
        warn 'I/O error'
          if $EVAL_ERROR->matches(':io');
    }
    elsif ($EVAL_ERROR) {
        warn "Something else went wrong: $EVAL_ERROR";
    }
};

try and catch

If you’re famil­iar with oth­er pro­gram­ming lan­guages, you’re prob­a­bly look­ing for syn­tax like try and catch for your excep­tion needs. The good news is that it’s com­ing in Perl 5.34 thanks to the ever-​productive Paul LeoNerd” Evans; the bet­ter news is that you can use it today with his Feature::Compat::Try mod­ule, itself a dis­til­la­tion of his pop­u­lar Syntax::Keyword::Try. Here’s an example:

use English;
use autodie;
use Feature::Compat::Try;

sub foo {
    try {
        attempt_a_thing();
        return 'success!';
    }
    catch ($exception) {
        return "failure: $exception"
          if not $exception->isa('autodie::exception');

        return 'failed in ' . $exception->function
          . ' line '        . $exception->line
          . ' called with '
          . join ', ', @{$exception->args};
    }
}

Note that autodie and Feature::Compat::Try are com­ple­men­tary and can be used togeth­er; also note that unlike an eval block, you can return from the enclos­ing func­tion in a try block.

The under­ly­ing Syntax::Keyword::Try mod­ule has even more options like a finally block and a cou­ple exper­i­men­tal fea­tures. I now pre­fer it to oth­er mod­ules that imple­ment try/​catch syn­tax like Try::Tiny and TryCatch (even though we use Try::Tiny at work). If all you need is the basic syn­tax above, using Feature::Compat::Try will get you used to the seman­tics that are com­ing in the next ver­sion of Perl.

Other exception modules (updated)

autodie is nice, and some oth­er mod­ules and frame­works imple­ment their own excep­tion class­es, but what if you want some help defin­ing your own? After all, an error string can only con­vey so much infor­ma­tion, may be dif­fi­cult to parse, and may need to change as busi­ness require­ments change.

Although CPAN has the pop­u­lar Exception::Class mod­ule, its author Dave Rolsky rec­om­mends that you use Throwable if you’re using Moose or Moo. If you’re rolling your own objects, use Throwable::Error.

Using Throwable could­n’t be simpler:

package Foo;

use Moo;
with 'Throwable';

has message => (is => 'ro');

... # later...

package main; 
Foo->throw( {message => 'something went wrong'} );

And it comes with Throwable::Error, which you can sub­class to get sev­er­al use­ful methods:

package Local::My::Error;
use parent 'Throwable::Error';

... # later...

package main;
use Feature::Compat::Try;

try {
    Local::My::Error->throw('something bad');
}
catch ($exception) {
    warn $exception->stack_trace->as_string;
}

(That stack_trace attribute comes cour­tesy of the StackTrace::Auto role com­posed into Throwable::Error. Moo and Moose users should sim­ply com­pose it into their class­es to get it.)

Testing exceptions with Test::Exception

Inevitably bugs will creep in to your code, and auto­mat­ed tests are one of the main weapons in a devel­op­er’s arse­nal against them. Use Test::Exception when writ­ing tests against code that emits excep­tions to see whether it behaves as expected:

use English;
use Test::More;
use Test::Exception;

...

throws_ok(sub { $foo->method(42) }, qr/error 42/,
  'method throws an error when it gets 42');
throws_ok(sub { $foo->method(57) }, 'My::Exception::Class',
  'method throws the right exception class');

dies_ok(sub { $bar->method() }, 'method died, no params');

lives_and(sub { is($baz->method(17), 17) },
  'method ran without exception, returned right value'); 

throws_ok(sub { $qux->process('nonexistent_file.txt') },
  'autodie::exception', # hey look, it's autodie again
  'got an autodie exception',
);
my $exception = $EVAL_ERROR;
SKIP: {
    skip 'no autodie exception thrown', 1
      unless $exception
      and $exception->isa('autodie::exception');
    ok($exception->match(':socket'),
      'was a socket error:' . $exception->errno);
}

done_testing();

Note that Test::Exceptions func­tions don’t mess with $EVAL_ERROR, so you’re free to check its val­ue right after you call it.

Documenting errors and exceptions

If I can leave you with one mes­sage, it’s this: Please doc­u­ment every error and excep­tion your code pro­duces, prefer­ably in a place and lan­guage that the end-​user can under­stand. The DIAGNOSTICS sec­tion of your doc­u­men­ta­tion (you are writ­ing doc­u­men­ta­tion, right, not just code com­ments?) is a great can­di­date. You can mod­el this sec­tion after the perldiag man­u­al page, which goes into great detail about many of the error mes­sages gen­er­at­ed by Perl itself.

(A pre­vi­ous ver­sion of this arti­cle did not note that one should make sure a suc­cess­ful eval returns true, and incor­rect­ly stat­ed that Class::Exception and Throwable were dep­re­cat­ed due to a bug in the MetaCPAN web site. Thanks to Dan Book for the corrections.)