Let’s assume for the moment that you’re writing a Perl module or application. You’d like to maintain some level of software quality (or kwalitee), so you’re writing a suite of test scripts. Whether you’re writing them first (good for you for practicing test-driven development!) or the application code is already there, you’ll probably be reaching for Test::Simple, Test::More, or one of the Test2::Suite bundles. With the latter two you’re immediately confronted with a choice: do you count up the number of tests into a plan, or do you forsake that in favor of leaving a done_testing() call at the end of your test script(s)?
There are good arguments for both approaches. When you first start, you probably have no idea how many tests your scripts will contain. After all, a test script can be a useful tool for designing a module’s interface by writing example code that will use it. Your exploratory code would be written as if the module or application was already done, testing it in the way you’d like it to work. Not declaring a plan makes perfect sense in this case; just put done_testing() at the end and get back to defining your tests.
You don’t have that option when using Test::Simple, of course—it’s so basic it only has one function (ok()), and you have to pre-declare how many tests you plan to run when useing the module, like so:
use Test::Simple tests => 23;
Test::More also supports this form of plan, or you can opt to use its plan function to state the number of tests in your script or subtest. With Test2 you have to use plan. Either way, the plan acts as a sort of meta-test, making sure that you executed exactly what you intended: no more, no less. While there are situations where it’s not possible to predict how many times a given set of tests should run, I would highly suggest that in all other cases you should “clean up” your tests and declare a plan. Later on, if you add or remove tests you’ll immediately be aware that something has changed and it’s time to tally up a new plan.
What about other Perl testing frameworks? They can use plans, too. Here are two examples:
Thoughts? Does declaring a test plan make writing tests too inflexible? Does not having a plan encourage bad behavior? Tell me what you think in the comments below.
All examples below use the bash or zsh command shells and were tested on macOS Catalina 10.15.7 running zsh5.7.1 and Perl 5.32.1. If you’re using something very different (e.g., Microsoft Windows’ CMD or PowerShell), you may have to set environment variables differently.
Ad-hoc test coverage
If all you want to do is run one shell command, here it is:
$ prove -vlre 'perl -MDevel::Cover -Ilib' t
This takes advantage of prove’s --exec option (abbreviated as -e) to run a different executable for tests. It recursively (-r) runs all your tests verbosely (-v) from the t directory while loading your application’s libraries (-l), while the perl executable uses (-M) Devel::Cover and the lib subdirectory. I use a similar technique when debugging tests.
$ HARNESS_PERL_SWITCHES=-MDevel::Cover prove -vlr t
This does almost the same thing as above without running a different executable. It sets Test::Harness’ HARNESS_PERL_SWITCHES environment variable for the duration of the prove command. You won’t get the text output of your test coverage at the end, though, and will still have to run Devel::Cover’s cover command to both see the coverage and generate web pages.
In a dedicated test session, window, or tab
If you have a terminal session, window, or tab dedicated solely to running your tests, set one of the environment variables above for that session:
$ export HARNESS_PERL_SWITCHES=-MDevel::Cover
Now all of your test scripts will pick up that option. You can add more options by enclosing the environment variable’s value in 'quotes'. For example, you might also want to load Devel::NYTProf for code profiling:
Setting the PERL5OPT environment variable also sets options for the perl running prove, which means that your test coverage, profiling, etc. will pick up prove’s execution as well as your test scripts.
It only exports one function compared to Test::Exception’s four: exception, which you can then use with the full suite of regular Test::More functions as well as other testing libraries such as Test::Deep.
It doesn’t override the caller function or use Sub::Uplevel to hide your test blocks from the call stack, so if your exception returns a stack trace you’ll get output from the test as well as the thing throwing the exception. The author considers this a feature since Sub::Uplevel is “twitchy.”
To ease porting, Test::Fatal also includes two functions, dies_ok and lives_ok, replacing Test::Exception’s functions of the same names. dies_okdoes not provide the exception thrown, though, so if you’re testing that you’ll need to use exception along with a TAP-emitting function like is() or like().
And that’s it! Either is a valid choice; it comes down to whether you prefer one approach over another. Test::Exception is also included as part of Test::Most’s requirements, so if you’re using the latter to reduce boilerplate you’ll be getting the former.
Postscript:
I’d be remiss if I didn’t also mention Test2::Tools::Exception, which is the preferred way to test exceptions using the Test2 framework. If you’re using Test2, ignore all the above and go straight to Test2::Tools::Exception.
Failure is a universal truth of computers. Files fail to open, web pages fail to load, programs fail to install, messages fail to arrive. As a developer you have no choice but to work in a seemingly hostile environment in which bugs and errors lurk around every corner.
Hopefully you find and fix the bugs during development and testing, but even with all bugs squashed exceptional conditions can occur. It’s your job as a Perl developer to use the tools available to you to handle these exceptions. Here are a few of them.
Perl has a primitive but effective mechanism for running code that may fail called eval. It runs either a string or block of Perl code, trapping any errors so that the enclosing program doesn’t crash. It’s your job then to ignore or handle the error; eval will return undef (or an empty list in list context) and set the magic variable $@ to the error string. (You can spell that $EVAL_ERROR if you use the English module, which you probably should to allow for more readable code.) Here’s a contrived example:
use English;
eval { $foo / 0; 1 }
or warn "tried to divide by zero: $EVAL_ERROR";
(Why the 1 at the end of the block? It forces the eval to return true if it succeeds; the or condition is executed if it returns false.)
What if you want to purposefully cause an exception, so that an enclosing eval (possibly several layers up) can handle it? You use die:
use English;
eval { process_file('foo.txt'); 1 }
or warn "couldn't process file: $EVAL_ERROR";
sub process_file {
my $file = shift;
open my $fh, '<', $file
or die "couldn't read $file: $OS_ERROR";
... # do something with $fh
}
It’s worth repeating that as a statement: You use exceptions so that enclosing code can decide how to handle the error. Contrast this with simply handling a function’s return value at the time it’s executed: except in the simplest of scripts, that part of the code likely has no idea what the error means to the rest of the application or how to best handle the problem.
Since many of Perl’s built-in functions (like open) return false or other values on failure, it can be tedious and error-prone to make sure that all of them report problems as exceptions. Enter autodie, which will helpfully replace the functions you choose with equivalents that throw exceptions. Introduced in Perl 5.10.1, it only affects the enclosing code block, and even goes so far as to set $EVAL_ERROR to an object that can be queried for more detail. Here’s an example:
use English;
use autodie; # defaults to everything but system and exec
eval { open my $fh, '<', 'foo.txt'; 1 } or do {
if ($EVAL_ERROR
and $EVAL_ERROR->isa('autodie::exception') {
warn 'Error from open'
if $EVAL_ERROR->matches('open');
warn 'I/O error'
if $EVAL_ERROR->matches(':io');
}
elsif ($EVAL_ERROR) {
warn "Something else went wrong: $EVAL_ERROR";
}
};
try and catch
If you’re familiar with other programming languages, you’re probably looking for syntax like try and catch for your exception needs. The good news is that it’s coming in Perl 5.34 thanks to the ever-productive Paul “LeoNerd” Evans; the better news is that you can use it today with his Feature::Compat::Try module, itself a distillation of his popular Syntax::Keyword::Try. Here’s an example:
use English;
use autodie;
use Feature::Compat::Try;
sub foo {
try {
attempt_a_thing();
return 'success!';
}
catch ($exception) {
return "failure: $exception"
if not $exception->isa('autodie::exception');
return 'failed in ' . $exception->function
. ' line ' . $exception->line
. ' called with '
. join ', ', @{$exception->args};
}
}
Note that autodie and Feature::Compat::Try are complementary and can be used together; also note that unlike an eval block, you can return from the enclosing function in a try block.
The underlying Syntax::Keyword::Try module has even more options like a finally block and a couple experimental features. I now prefer it to other modules that implement try/catch syntax like Try::Tiny and TryCatch (even though we use Try::Tiny at work). If all you need is the basic syntax above, using Feature::Compat::Try will get you used to the semantics that are coming in the next version of Perl.
Other exception modules (updated)
autodie is nice, and some othermodulesandframeworks implement their own exception classes, but what if you want some help defining your own? After all, an error string can only convey so much information, may be difficult to parse, and may need to change as business requirements change.
(That stack_trace attribute comes courtesy of the StackTrace::Auto role composed into Throwable::Error. Moo and Moose users should simply compose it into their classes to get it.)
Inevitably bugs will creep in to your code, and automated tests are one of the main weapons in a developer’s arsenal against them. Use Test::Exception when writing tests against code that emits exceptions to see whether it behaves as expected:
use English;
use Test::More;
use Test::Exception;
...
throws_ok(sub { $foo->method(42) }, qr/error 42/,
'method throws an error when it gets 42');
throws_ok(sub { $foo->method(57) }, 'My::Exception::Class',
'method throws the right exception class');
dies_ok(sub { $bar->method() }, 'method died, no params');
lives_and(sub { is($baz->method(17), 17) },
'method ran without exception, returned right value');
throws_ok(sub { $qux->process('nonexistent_file.txt') },
'autodie::exception', # hey look, it's autodie again
'got an autodie exception',
);
my $exception = $EVAL_ERROR;
SKIP: {
skip 'no autodie exception thrown', 1
unless $exception
and $exception->isa('autodie::exception');
ok($exception->match(':socket'),
'was a socket error:' . $exception->errno);
}
done_testing();
Note that Test::Exception’s functions don’t mess with $EVAL_ERROR, so you’re free to check its value right after you call it.
Documenting errors and exceptions
If I can leave you with one message, it’s this: Please document every error and exception your code produces, preferably in a place and language that the end-user can understand. The DIAGNOSTICS section of your documentation (you are writing documentation, right, not just code comments?) is a great candidate. You can model this section after the perldiag manual page, which goes into great detail about many of the error messages generated by Perl itself.
(A previous version of this article did not note that one should make sure a successful eval returns true, and incorrectly stated that Class::Exception and Throwable were deprecated due to a bug in the MetaCPAN web site. Thanks to Dan Book for the corrections.)
{"id":"11","mode":"button","open_style":"in_modal","currency_code":"USD","currency_symbol":"$","currency_type":"decimal","blank_flag_url":"https:\/\/phoenixtrap.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/blank.gif","flag_sprite_url":"https:\/\/phoenixtrap.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/flags.png","default_amount":500,"top_media_type":"featured_image","featured_image_url":"https:\/\/phoenixtrap.com\/wp-content\/uploads\/2021\/02\/image-200x200.jpg","featured_embed":"","header_media":null,"file_download_attachment_data":null,"recurring_options_enabled":true,"recurring_options":{"never":{"selected":true,"after_output":"One time only"},"weekly":{"selected":false,"after_output":"Every week"},"monthly":{"selected":false,"after_output":"Every month"},"yearly":{"selected":false,"after_output":"Every year"}},"strings":{"current_user_email":"","current_user_name":"","link_text":"Leave a tip!","complete_payment_button_error_text":"Check info and try again","payment_verb":"Pay","payment_request_label":"The Phoenix Trap","form_has_an_error":"Please check and fix the errors above","general_server_error":"Something isn't working right at the moment. Please try again.","form_title":"The Phoenix Trap","form_subtitle":"Do you like what you see? Leave a one-time or recurring tip!","currency_search_text":"Country or Currency here","other_payment_option":"Other payment option","manage_payments_button_text":"Manage your payments","thank_you_message":"Thank you for being a supporter!","payment_confirmation_title":"The Phoenix Trap","receipt_title":"Your Receipt","print_receipt":"Print Receipt","email_receipt":"Email Receipt","email_receipt_sending":"Sending receipt...","email_receipt_success":"Email receipt successfully sent","email_receipt_failed":"Email receipt failed to send. Please try again.","receipt_payee":"Paid to","receipt_statement_descriptor":"This will show up on your statement as","receipt_date":"Date","receipt_transaction_id":"Transaction ID","receipt_transaction_amount":"Amount","refund_payer":"Refund from","login":"Log in to manage your payments","manage_payments":"Manage Payments","transactions_title":"Your Transactions","transaction_title":"Transaction Receipt","transaction_period":"Plan Period","arrangements_title":"Your Plans","arrangement_title":"Manage Plan","arrangement_details":"Plan Details","arrangement_id_title":"Plan ID","arrangement_payment_method_title":"Payment Method","arrangement_amount_title":"Plan Amount","arrangement_renewal_title":"Next renewal date","arrangement_action_cancel":"Cancel Plan","arrangement_action_cant_cancel":"Cancelling is currently not available.","arrangement_action_cancel_double":"Are you sure you'd like to cancel?","arrangement_cancelling":"Cancelling Plan...","arrangement_cancelled":"Plan Cancelled","arrangement_failed_to_cancel":"Failed to cancel plan","back_to_plans":"\u2190 Back to Plans","update_payment_method_verb":"Update","sca_auth_description":"Your have a pending renewal payment which requires authorization.","sca_auth_verb":"Authorize renewal payment","sca_authing_verb":"Authorizing payment","sca_authed_verb":"Payment successfully authorized!","sca_auth_failed":"Unable to authorize! Please try again.","login_button_text":"Log in","login_form_has_an_error":"Please check and fix the errors above","uppercase_search":"Search","lowercase_search":"search","uppercase_page":"Page","lowercase_page":"page","uppercase_items":"Items","lowercase_items":"items","uppercase_per":"Per","lowercase_per":"per","uppercase_of":"Of","lowercase_of":"of","back":"Back to plans","zip_code_placeholder":"Zip\/Postal Code","download_file_button_text":"Download File","input_field_instructions":{"tip_amount":{"placeholder_text":"How much would you like to tip?","initial":{"instruction_type":"normal","instruction_message":"How much would you like to tip? Choose any currency."},"empty":{"instruction_type":"error","instruction_message":"How much would you like to tip? Choose any currency."},"invalid_curency":{"instruction_type":"error","instruction_message":"Please choose a valid currency."}},"recurring":{"placeholder_text":"Recurring","initial":{"instruction_type":"normal","instruction_message":"How often would you like to give this?"},"success":{"instruction_type":"success","instruction_message":"How often would you like to give this?"},"empty":{"instruction_type":"error","instruction_message":"How often would you like to give this?"}},"name":{"placeholder_text":"Name on Credit Card","initial":{"instruction_type":"normal","instruction_message":"What is the name on your credit card?"},"success":{"instruction_type":"success","instruction_message":"Enter the name on your card."},"empty":{"instruction_type":"error","instruction_message":"Please enter the name on your card."}},"privacy_policy":{"terms_title":"Terms and conditions","terms_body":null,"terms_show_text":"View Terms","terms_hide_text":"Hide Terms","initial":{"instruction_type":"normal","instruction_message":"I agree to the terms."},"unchecked":{"instruction_type":"error","instruction_message":"Please agree to the terms."},"checked":{"instruction_type":"success","instruction_message":"I agree to the terms."}},"email":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"What is your email address?"},"success":{"instruction_type":"success","instruction_message":"Enter your email address"},"blank":{"instruction_type":"error","instruction_message":"Enter your email address"},"not_an_email_address":{"instruction_type":"error","instruction_message":"Make sure you have entered a valid email address"}},"note_with_tip":{"placeholder_text":"Your note here...","initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"empty":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"not_empty_initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"saving":{"instruction_type":"normal","instruction_message":"Saving note..."},"success":{"instruction_type":"success","instruction_message":"Note successfully saved!"},"error":{"instruction_type":"error","instruction_message":"Unable to save note note at this time. Please try again."}},"email_for_login_code":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email to log in."},"success":{"instruction_type":"success","instruction_message":"Enter your email to log in."},"blank":{"instruction_type":"error","instruction_message":"Enter your email to log in."},"empty":{"instruction_type":"error","instruction_message":"Enter your email to log in."}},"login_code":{"initial":{"instruction_type":"normal","instruction_message":"Check your email and enter the login code."},"success":{"instruction_type":"success","instruction_message":"Check your email and enter the login code."},"blank":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."},"empty":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."}},"stripe_all_in_one":{"initial":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"empty":{"instruction_type":"error","instruction_message":"Enter your credit card details here."},"success":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"invalid_number":{"instruction_type":"error","instruction_message":"The card number is not a valid credit card number."},"invalid_expiry_month":{"instruction_type":"error","instruction_message":"The card's expiration month is invalid."},"invalid_expiry_year":{"instruction_type":"error","instruction_message":"The card's expiration year is invalid."},"invalid_cvc":{"instruction_type":"error","instruction_message":"The card's security code is invalid."},"incorrect_number":{"instruction_type":"error","instruction_message":"The card number is incorrect."},"incomplete_number":{"instruction_type":"error","instruction_message":"The card number is incomplete."},"incomplete_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incomplete."},"incomplete_expiry":{"instruction_type":"error","instruction_message":"The card's expiration date is incomplete."},"incomplete_zip":{"instruction_type":"error","instruction_message":"The card's zip code is incomplete."},"expired_card":{"instruction_type":"error","instruction_message":"The card has expired."},"incorrect_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incorrect."},"incorrect_zip":{"instruction_type":"error","instruction_message":"The card's zip code failed validation."},"invalid_expiry_year_past":{"instruction_type":"error","instruction_message":"The card's expiration year is in the past"},"card_declined":{"instruction_type":"error","instruction_message":"The card was declined."},"missing":{"instruction_type":"error","instruction_message":"There is no card on a customer that is being charged."},"processing_error":{"instruction_type":"error","instruction_message":"An error occurred while processing the card."},"invalid_request_error":{"instruction_type":"error","instruction_message":"Unable to process this payment, please try again or use alternative method."},"invalid_sofort_country":{"instruction_type":"error","instruction_message":"The billing country is not accepted by SOFORT. Please try another country."}}}},"fetched_oembed_html":false}