Six months ago I gave an overview of Perl’s list processing fundamentals, briefly describing what lists are and then introducing the built-in map and grep functions for transforming and filtering them. Later on, I compiled a list (how appropriate) of list processing modules available via CPAN, noting there’s some confusing duplication of effort. But you’re a busy developer, and you just want to know the Right Thing To Do™ when faced with a list processing challenge.
First, some credit is due: these are all restatements of several Perl::Critic policies which in turn codify standards described in Damian Conway’s Perl Best Practices (2005). I’ve repeatedly recommended the latter as a starting point for higher-quality Perl development. Over the years these practices continue to be re-evaluated (including by the author himself) and various authors release new policy modules, but perlcritic remains a great tool for ensuring you (and your team or other contributors) maintain a consistent high standard in your code.
With that said, on to the recommendations!
Don’t use grep to check if any list elements match
It might sound weird to lead off by recommending not to use grep, but sometimes it’s not the right tool for the job. If you’ve got a list and want to determine if a condition matches any item in it, you might try:
if (grep { some_condition($_) } @my_list) {
... # don't do this!
}
Yes, this works because (in scalar context) grep returns the number of matches found, but it’s wasteful, checking every element of @my_list (which could be lengthy) before finally providing a result. Use the standard List::Util module’s any function, which immediately returns (“short-circuits”) on the first match:
use List::Util 1.33 qw(any);
if (any { some_condition($_) } @my_list) { ... # do something }
As a side note for web developers, the Perl Dancer framework also includes an any keyword for declaring multiple HTTP routes, so if you’re mixing List::Util in there don’t import it. Instead, call it explicitly like this or you’ll get an error about a redefined function:
use List::Util 1.33;
if (List::Util::any { some_condition($_) } @my_list) { ... # do something }
I mentioned this back in March, but it bears repeating: map and grep are intended as pure functions, not mutators with side effects. This means that the original list should remain unchanged. Yes, each element aliases in turn to the $_ special variable, but that’s for speed and can have surprising results if changed even if it’s technically allowed. If you need to modify an array in-place use something like:
for (@my_array) { $_ = ...; # make your changes here }
If you want something that looks like map but won’t change the original list (and don’t mind a few CPAN dependencies), consider List::SomeUtils’ apply function:
use List::SomeUtils qw(apply);
my @doubled_array = apply {$_ *= 2} @old_array;
Lastly, side effects also include things like manipulating other variables or doing input and output. Don’t use map or grep in a void context (i.e., without a resulting array or list); do something with the results or use a for or foreach loop:
map { print foo($_) } @my_array; # don't do this print map { foo($_) } @my_array; # do this instead
map { push @new_array, foo($_) } @my_array; # don't do this @new_array = map { foo($_) } @my_array; # do this instead
my @new_array = map foo($_), @old_array; # don't do this my @new_array2 = grep !/^#/, @old_array; # don't do this
Or like this:
my @new_array = map { foo($_) } @old_array; my @new_array2 = grep {!/^#/} @old_array;
Do it the second way. It’s easier to read, especially if you’re passing in a literal list or multiple arrays, and the expression forms can conceal bugs. This recommendation is codified by the BuiltinFunctions::RequireBlockGrep and BuiltinFunctions::RequireBlockMap Perl::Critic policies and comes from Perl Best Practices.
Refactor multi-statement maps, greps, and other list functions
map, grep, and friends should follow the Unix philosophy of “Do One Thing and Do It Well.” Your readability and maintainability drop with every statement you place inside one of their blocks. Consider junior developers and future maintainers (this includes you!) and refactor anything with more than one statement into a separate subroutine or at least a for loop. This goes for list processing functions (like the aforementioned any) imported from other modules, too.
I think this is the real point about Perl code readability: it gives you enough flexibility to do things however you like, and as a result many programmers are faced with a mirror that reflects their own bad practices back at them.
The other conformance tool in a Perl developer’s belt is perltidy, and it too has a Conway-compatible configuration as well as its default Perl Style Guide settings. I’ve found that more than anything else, perltidy helps settle arguments both between developers and between their code in helping to avoid excessive merge conflicts.
But apart from extra tools, Perl the language itself can be bent and even broken to suit just about anyone’s agenda. Those used to more bondage-and-discipline languages(hi, Java!) might feel revulsion at the lengths to which this has sometimes been taken, but per the quote above this is less an indictment of the language and more of its less methodical programmers.
Some of this behavior can be rehabilitated with perlcritic and perltidy, but what about other sins attributed to Perl? Here are a few perennial “favorites”:
The good news is that since 2007 we’ve had a sophisticated metaobject-protocol-based layer on top of them called Moose, since 2010 a lightweight but forward-compatible system called Moo, and a couple of even tinier options as described in the Perl OO Tutorial. Waiting in the wings is Corinna, an effort to bring next-generation object capabilities into the Perl core itself, and Object::Pad, a testbed for some of the ideas in Corinna that you can use today in current code. (Really, please try it—the author needs feedback!)
All this is to say that 99% of the time you never need trouble yourself with bless, constructors, or writing accessors for class or object attributes. Smarter people than me have done the work for you, and you might even find a concept or three that you wish other languages had.
There are two major ones: list and scalar. Another way to think of it is “plural” vs. “singular” in English, which is hopefully a thing you’re familiar with as you’re reading this blog.
Some functions in Perl act differently depending on whether the expected return value is a list or a scalar, and a function will provide a list or scalar context to its arguments. Mostly these act just as you would expect or would like them to, and you can find out how a function behaves by reading its documentation. Your own functions can behave like this too, but there’s usually no need as “both scalars and lists are automatically interpreted into lists.” Again, Perl’s DWIMmery at work.
Subroutine and Method Arguments
I’ve already written about this. Twice. And presented about it. Twice. The short version: Perl has signatures, but they’ve been considered experimental for a while. In the meantime, there are alternatives on CPAN. You can even have type constraints if you want.
I’ll leave you with this: Over the past month, Neil Bowers of the Perl Steering Council has been collecting quirks like these from Perl developers. The PSC is reviewing this collection for potential documentation fixes, bug fixes, further discussion, etc. I wouldn’t expect to see any fundamental changes to the language out of this effort, but it’s a good sign that potentially confusing features are being addressed.
Perl is said (sometimes frustratingly) to be a do-what-I-mean programming language. Many of its statements and constructions are designed to be forgiving or have analogies to natural languages. Still others are said to be “magic,” behaving differently depending on how they’re used. Adept use of Perl asks you to not only understand this magic, but to embrace it and the expressiveness it enables. Here, then, are five ways you can bring some magic to your code.
Perl has many special variables, and first among them (literally, it’s the first documented) is $_. Also spelled $ARG if you use the English module, the documentation describes it as “the default input and pattern-matching space.” Many, many functions and statements will assume it as the default or implicit argument; you can find the full list in the documentation. Here’s an example that uses it implicitly to output the numbers from 1 to 5:
say for 1 .. 5;
Output:
1
2
3
4
5
Where some languages require an iterator variable in a for or foreach loop, in the absence of one Perl assigns it to $_.
Statement modifiers
We then use our second trick; where some other languages require a block to enclose every loop or conditional (whether denoted by braces { } or indentation), Perl allows you to put said looping or conditional statement after a single other statement, in this case the say which prints its argument(s) followed by a newline.
However, above we have no arguments passed to say and so once again the default $_ is used, now containing a number from 1 to 5 which is then printed out. It’s a very powerful and expressive idiom, enabling both the writer and reader of code to concentrate on the important thing that’s happening. It’s also entirely optional. You can just as easily type:
We mentioned the $_ variable above, and that it could also be spelled $ARG if you add use English to your code. It can be hard to read code with large amounts of punctuation, though, and even harder to remember what each variable does. Thankfully the English module provides aliases, and the perlvar man page lists them in order. It’s much easier to read and write things like $LIST_SEPARATOR, $PROCESS_ID, or $MATCH rather than $", $$, and $&, and goes a long way towards reducing Perl’s reputation as a write-only language.
Like natural languages, Perl has a concept of “context” in which words mean different things depending on their surroundings. In Perl’s case, expressions may behave differently depending on whether they expect to produce a list of values or a single value, called a scalar. Here’s a trivial example:
my @foo = (1, 2, 3); # list context, @foo contains the list
my $bar = (1, 2, 3); # scalar context, $bar contains 3
In the first line, we assign the list of numbers (1, 2, 3) to the array @foo. But in the second line, we’re assigning to the scalar variable $bar, which now contains the last item in the list.
Here’s another example, using the reverse function:
my @foo = ('one', 'two', 'three');
my @bar = reverse @foo; # @bar contains ('three', 'two', 'one')
my $baz = reverse @foo; # $baz contains 'eerhtowteno'
In list context, reverse takes its arguments and returns them in the opposite order. But in scalar context, it concatenates all of the arguments together and returns a string with the characters in opposite order.
In general, “there is no general rule for deducing a function’s behavior in scalar context from its behavior in list context.” (Dominus 1998) You’ll just have to look up the function to determine what it does, though in general, it does what you want, but if you want to force scalar context use the scalar operator:
my @foo = ('aa', 'aab', 'bbc');
my @bar = scalar grep /aa/, @foo; # returns a list (2), counting the number of matches
One of Perl’s three built-in data types is the hash, also known as an associative array. It’s an unordered collection of scalars indexed by string, rather than the numbers used by normal arrays. It’s a useful construct, and you can develop complicated data structures using just scalars, arrays, and hashes. What’s not widely known is that you can access several elements of of a hash using a hash slice, using syntax that’s similar to array slices. Here’s an example:
my ($who, $home) = @ENV{'USER', 'HOME'};
It works the other way, too: you can assign to a slice.
use v5.24; # for postfix dereferencing
use Types::Standard qw(Str Int);
use Type::Params 'compile_named';
foo('hello', 42);
sub foo {
state $check = compile_named(
param1 => Str,
param2 => Int, {optional => 1},
);
my ($param1, $param2) =
$check->(@_)->@{'param1', 'param2'};
say $param1, $param2;
}
In the example above, $check->(@_) returns the type-checked arguments to the foo() function courtesy of Type::Params’ compile_named() function. It’s returned as a hash reference, and since hashes are unordered, we specify the order in which we want the values by dereferencing and then slicing the resulting hash. The postfix dereferencing syntax was added in Perl 5.20 and made a default feature in 5.24, and reduces the number of nested brackets and braces we have to deal with.
Conclusion
I hope this article has given you a taste of some of the magic available in the Perl language. It’s these sort of features that make programming in it a bit more joyful. As always, check the documentation for complete information on these and other topics, or look for answers and ask questions on PerlMonks or Stack Overflow.
{"id":"11","mode":"button","open_style":"in_modal","currency_code":"USD","currency_symbol":"$","currency_type":"decimal","blank_flag_url":"https:\/\/phoenixtrap.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/blank.gif","flag_sprite_url":"https:\/\/phoenixtrap.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/flags.png","default_amount":500,"top_media_type":"featured_image","featured_image_url":"https:\/\/phoenixtrap.com\/wp-content\/uploads\/2021\/02\/image-200x200.jpg","featured_embed":"","header_media":null,"file_download_attachment_data":null,"recurring_options_enabled":true,"recurring_options":{"never":{"selected":true,"after_output":"One time only"},"weekly":{"selected":false,"after_output":"Every week"},"monthly":{"selected":false,"after_output":"Every month"},"yearly":{"selected":false,"after_output":"Every year"}},"strings":{"current_user_email":"","current_user_name":"","link_text":"Leave a tip!","complete_payment_button_error_text":"Check info and try again","payment_verb":"Pay","payment_request_label":"The Phoenix Trap","form_has_an_error":"Please check and fix the errors above","general_server_error":"Something isn't working right at the moment. Please try again.","form_title":"The Phoenix Trap","form_subtitle":"Do you like what you see? Leave a one-time or recurring tip!","currency_search_text":"Country or Currency here","other_payment_option":"Other payment option","manage_payments_button_text":"Manage your payments","thank_you_message":"Thank you for being a supporter!","payment_confirmation_title":"The Phoenix Trap","receipt_title":"Your Receipt","print_receipt":"Print Receipt","email_receipt":"Email Receipt","email_receipt_sending":"Sending receipt...","email_receipt_success":"Email receipt successfully sent","email_receipt_failed":"Email receipt failed to send. Please try again.","receipt_payee":"Paid to","receipt_statement_descriptor":"This will show up on your statement as","receipt_date":"Date","receipt_transaction_id":"Transaction ID","receipt_transaction_amount":"Amount","refund_payer":"Refund from","login":"Log in to manage your payments","manage_payments":"Manage Payments","transactions_title":"Your Transactions","transaction_title":"Transaction Receipt","transaction_period":"Plan Period","arrangements_title":"Your Plans","arrangement_title":"Manage Plan","arrangement_details":"Plan Details","arrangement_id_title":"Plan ID","arrangement_payment_method_title":"Payment Method","arrangement_amount_title":"Plan Amount","arrangement_renewal_title":"Next renewal date","arrangement_action_cancel":"Cancel Plan","arrangement_action_cant_cancel":"Cancelling is currently not available.","arrangement_action_cancel_double":"Are you sure you'd like to cancel?","arrangement_cancelling":"Cancelling Plan...","arrangement_cancelled":"Plan Cancelled","arrangement_failed_to_cancel":"Failed to cancel plan","back_to_plans":"\u2190 Back to Plans","update_payment_method_verb":"Update","sca_auth_description":"Your have a pending renewal payment which requires authorization.","sca_auth_verb":"Authorize renewal payment","sca_authing_verb":"Authorizing payment","sca_authed_verb":"Payment successfully authorized!","sca_auth_failed":"Unable to authorize! Please try again.","login_button_text":"Log in","login_form_has_an_error":"Please check and fix the errors above","uppercase_search":"Search","lowercase_search":"search","uppercase_page":"Page","lowercase_page":"page","uppercase_items":"Items","lowercase_items":"items","uppercase_per":"Per","lowercase_per":"per","uppercase_of":"Of","lowercase_of":"of","back":"Back to plans","zip_code_placeholder":"Zip\/Postal Code","download_file_button_text":"Download File","input_field_instructions":{"tip_amount":{"placeholder_text":"How much would you like to tip?","initial":{"instruction_type":"normal","instruction_message":"How much would you like to tip? Choose any currency."},"empty":{"instruction_type":"error","instruction_message":"How much would you like to tip? Choose any currency."},"invalid_curency":{"instruction_type":"error","instruction_message":"Please choose a valid currency."}},"recurring":{"placeholder_text":"Recurring","initial":{"instruction_type":"normal","instruction_message":"How often would you like to give this?"},"success":{"instruction_type":"success","instruction_message":"How often would you like to give this?"},"empty":{"instruction_type":"error","instruction_message":"How often would you like to give this?"}},"name":{"placeholder_text":"Name on Credit Card","initial":{"instruction_type":"normal","instruction_message":"What is the name on your credit card?"},"success":{"instruction_type":"success","instruction_message":"Enter the name on your card."},"empty":{"instruction_type":"error","instruction_message":"Please enter the name on your card."}},"privacy_policy":{"terms_title":"Terms and conditions","terms_body":null,"terms_show_text":"View Terms","terms_hide_text":"Hide Terms","initial":{"instruction_type":"normal","instruction_message":"I agree to the terms."},"unchecked":{"instruction_type":"error","instruction_message":"Please agree to the terms."},"checked":{"instruction_type":"success","instruction_message":"I agree to the terms."}},"email":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"What is your email address?"},"success":{"instruction_type":"success","instruction_message":"Enter your email address"},"blank":{"instruction_type":"error","instruction_message":"Enter your email address"},"not_an_email_address":{"instruction_type":"error","instruction_message":"Make sure you have entered a valid email address"}},"note_with_tip":{"placeholder_text":"Your note here...","initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"empty":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"not_empty_initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"saving":{"instruction_type":"normal","instruction_message":"Saving note..."},"success":{"instruction_type":"success","instruction_message":"Note successfully saved!"},"error":{"instruction_type":"error","instruction_message":"Unable to save note note at this time. Please try again."}},"email_for_login_code":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email to log in."},"success":{"instruction_type":"success","instruction_message":"Enter your email to log in."},"blank":{"instruction_type":"error","instruction_message":"Enter your email to log in."},"empty":{"instruction_type":"error","instruction_message":"Enter your email to log in."}},"login_code":{"initial":{"instruction_type":"normal","instruction_message":"Check your email and enter the login code."},"success":{"instruction_type":"success","instruction_message":"Check your email and enter the login code."},"blank":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."},"empty":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."}},"stripe_all_in_one":{"initial":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"empty":{"instruction_type":"error","instruction_message":"Enter your credit card details here."},"success":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"invalid_number":{"instruction_type":"error","instruction_message":"The card number is not a valid credit card number."},"invalid_expiry_month":{"instruction_type":"error","instruction_message":"The card's expiration month is invalid."},"invalid_expiry_year":{"instruction_type":"error","instruction_message":"The card's expiration year is invalid."},"invalid_cvc":{"instruction_type":"error","instruction_message":"The card's security code is invalid."},"incorrect_number":{"instruction_type":"error","instruction_message":"The card number is incorrect."},"incomplete_number":{"instruction_type":"error","instruction_message":"The card number is incomplete."},"incomplete_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incomplete."},"incomplete_expiry":{"instruction_type":"error","instruction_message":"The card's expiration date is incomplete."},"incomplete_zip":{"instruction_type":"error","instruction_message":"The card's zip code is incomplete."},"expired_card":{"instruction_type":"error","instruction_message":"The card has expired."},"incorrect_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incorrect."},"incorrect_zip":{"instruction_type":"error","instruction_message":"The card's zip code failed validation."},"invalid_expiry_year_past":{"instruction_type":"error","instruction_message":"The card's expiration year is in the past"},"card_declined":{"instruction_type":"error","instruction_message":"The card was declined."},"missing":{"instruction_type":"error","instruction_message":"There is no card on a customer that is being charged."},"processing_error":{"instruction_type":"error","instruction_message":"An error occurred while processing the card."},"invalid_request_error":{"instruction_type":"error","instruction_message":"Unable to process this payment, please try again or use alternative method."},"invalid_sofort_country":{"instruction_type":"error","instruction_message":"The billing country is not accepted by SOFORT. Please try another country."}}}},"fetched_oembed_html":false}