Tag: context

  • Better Perl: Four list processing best practices with map, grep, and more

    Better Perl: Four list processing best practices with map, grep, and more

    Six months ago I gave an overview of Perl’s list pro­cess­ing fun­da­men­tals, briefly describ­ing what lists are and then intro­duc­ing the built-​in mapand grepfunc­tions for trans­form­ing and fil­ter­ing them. Later on, I com­piled a list (how appro­pri­ate) of list pro­cess­ing mod­ules avail­able via CPAN, not­ing there’s some con­fus­ing dupli­ca­tion of effort. But you’re a busy devel­op­er, and you just want to know the Right Thing To Do™ when faced with a list pro­cess­ing challenge.

    First, some cred­it is due: these are all restate­ments of sev­er­al Perl::Critic poli­cieswhich in turn cod­i­fy stan­dards described in Damian Conway’s Perl Best Practices(2005). I’ve repeat­ed­ly rec­om­mend­edthe lat­ter as a start­ing point for higher-​quality Perl devel­op­ment. Over the years these prac­tices con­tin­ue to be re-​evaluated (includ­ing by the author him­self) and var­i­ous authors release new pol­i­cy mod­ules, but perlcriticremains a great tool for ensur­ing you (and your team or oth­er con­trib­u­tors) main­tain a con­sis­tent high stan­dard in your code.

    With that said, on to the recommendations!

    Don’t use grepto check if anylist elements match

    It might sound weird to lead off by rec­om­mend­ing notto use grep, but some­times it’s not the right tool for the job. If you’ve got a list and want to deter­mine if a con­di­tion match­es any item in it, you might try:

    if (grep { some_condition($_) } @my_list) {
        ... # don't do this!
    }

    Yes, this works because (in scalar con­text) grepreturns the num­ber of match­es found, but it’s waste­ful, check­ing every ele­ment of @my_list(which could be lengthy) before final­ly pro­vid­ing a result. Use the stan­dard List::Utilmodule’s anyfunc­tion, which imme­di­ate­ly returns (“short-​circuits”) on the first match:

    use List::Util 1.33 qw(any);

    if (any { some_condition($_) } @my_list) {
    ... # do something
    }

    Perl has includ­ed the req­ui­site ver­sion of this mod­ule since ver­sion 5.20in 2014; for ear­li­er releas­es, you’ll need to update from CPAN. List::Util has many oth­er great list-​reduction, key/​value pair, and oth­er relat­ed func­tionsyou can import into your code, so check it out before you attempt to re-​invent any wheels.

    As a side note for web devel­op­ers, the Perl Dancer frame­workalso includes an anykey­wordfor declar­ing mul­ti­ple HTTP routes, so if you’re mix­ing List::Util in there don’t import it. Instead, call it explic­it­ly like this or you’ll get an error about a rede­fined function:

    use List::Util 1.33;

    if (List::Util::any { some_condition($_) } @my_list) {
    ... # do something
    }

    This rec­om­men­da­tion is cod­i­fied in the BuiltinFunctions::ProhibitBooleanGrepPerl::Critic pol­i­cy, comes direct­ly from Perl Best Practices, and is rec­om­mend­edby the Software Engineering Institute Computer Emergency Response Team (SEI CERT)’s Perl Coding Standard.

    Don’t change $_in mapor grep

    I men­tioned this back in March, but it bears repeat­ing: mapand grepare intend­ed as pure func­tions, not muta­torswith side effects. This means that the orig­i­nal list should remain unchanged. Yes, each ele­ment alias­es in turn to the $_spe­cial vari­able, but that’s for speed and can have sur­pris­ing results if changed even if it’s tech­ni­cal­ly allowed. If you need to mod­i­fy an array in-​place use some­thing like:

    for (@my_array) {
    $_ = ...; # make your changes here
    }

    If you want some­thing that lookslike mapbut won’t change the orig­i­nal list (and don’t mind a few CPAN depen­den­cies), con­sid­er List::SomeUtils’ applyfunction:

    use List::SomeUtils qw(apply);
    
    my @doubled_array = apply {$_ *= 2} @old_array;

    Lastly, side effects also include things like manip­u­lat­ing oth­er vari­ables or doing input and out­put. Don’t use mapor grepin a void con­text (i.e., with­out a result­ing array or list); do some­thing with the results or use a foror foreachloop:

    map { print foo($_) } @my_array; # don't do this
    print map { foo($_) } @my_array; # do this instead

    map { push @new_array, foo($_) } @my_array; # don't do this
    @new_array = map { foo($_) } @my_array; # do this instead

    This rec­om­men­da­tion is cod­i­fied by the BuiltinFunctions::ProhibitVoidGrep, BuiltinFunctions::ProhibitVoidMap, and ControlStructures::ProhibitMutatingListFunctionsPerl::Critic poli­cies. The lat­ter comes from Perl Best Practicesand is an SEI CERT Perl Coding Standard rule.

    Use blocks with mapand grep, not expressions

    You can call mapor greplike this (paren­the­ses are option­alaround built-​in functions):

    my @new_array  = map foo($_), @old_array; # don't do this
    my @new_array2 = grep !/^#/, @old_array; # don't do this

    Or like this:

    my @new_array  = map { foo($_) } @old_array;
    my @new_array2 = grep {!/^#/} @old_array;

    Do it the sec­ond way. It’s eas­i­er to read, espe­cial­ly if you’re pass­ing in a lit­er­al list or mul­ti­ple arrays, and the expres­sion forms can con­ceal bugs. This rec­om­men­da­tion is cod­i­fied by the BuiltinFunctions::RequireBlockGrepand BuiltinFunctions::RequireBlockMapPerl::Critic poli­cies and comes from Perl Best Practices.

    Refactor multi-​statement maps, greps, and other list functions

    map, grep, and friends should fol­low the Unix phi­los­o­phy of ​“Do One Thing and Do It Well.”Your read­abil­i­ty and main­tain­abil­i­ty drop with every state­ment you place inside one of their blocks. Consider junior devel­op­ers and future main­tain­ers (this includes you!) and refac­tor any­thing with more than one state­ment into a sep­a­rate sub­rou­tine or at least a forloop. This goes for list pro­cess­ing func­tions (like the afore­men­tioned any) import­ed from oth­er mod­ules, too.

    This rec­om­men­da­tion is cod­i­fied by the Perl Best Practices-inspired BuiltinFunctions::ProhibitComplexMappingsand BuiltinFunctions::RequireSimpleSortBlockPerl::Critic poli­cies, although those only cov­er mapand sortfunc­tions, respectively.


    Do you have any oth­er sug­ges­tions for list pro­cess­ing best prac­tices? Feel free to leave them in the com­ments or bet­ter yet, con­sid­er cre­at­ing new Perl::Critic poli­ciesfor them or con­tact­ing the Perl::Critic teamto devel­op them for your organization.

  • 5 Ways You Can Embrace the Magic of Perl

    5 Ways You Can Embrace the Magic of Perl

    Perl is said (some­times frus­trat­ing­ly) to be a do-​what-​I-​mean pro­gram­ming lan­guage. Many of its state­ments and con­struc­tions are designed to be for­giv­ingor have analo­gies to nat­ur­al lan­guages. Still oth­ers are said to be ​“mag­ic,” behav­ing dif­fer­ent­ly depend­ing on how they’re used. Adept use of Perl asks you to not only under­standthis mag­ic, but to embraceit and the expres­sive­ness it enables. Here, then, are five ways you can bring some mag­ic to your code.

    $_

    Perl has many spe­cial vari­ables, and first among them (lit­er­al­ly, it’s the first doc­u­ment­ed) is $_. Also spelled $ARGif you usethe Englishmod­ule, the doc­u­men­ta­tion describes it as ​“the default input and pattern-​matching space.” Many, many func­tions and state­ments will assume it as the default or implic­it argu­ment; you can find the full list in the doc­u­men­ta­tion. Here’s an exam­ple that uses it implic­it­ly to out­put the num­bers from 1 to 5:

    say for 1 .. 5;
    

    Output:

    1
    2
    3
    4
    5
    

    Where some lan­guages require an iter­a­tor vari­able in a foror foreachloop, in the absence of one Perl assigns it to $_.

    Statement modifiers

    We then use our sec­ond trick; where some oth­er lan­guages require a block to enclose every loop or con­di­tion­al (whether denot­ed by braces { }or inden­ta­tion), Perl allows you to put said loop­ing or con­di­tion­al state­ment after a sin­gle oth­er state­ment, in this case the saywhich prints its argument(s) fol­lowed by a newline.

    However, above we have noargu­ments passed to sayand so once again the default $_is used, now con­tain­ing a num­ber from 1 to 5 which is then print­ed out. It’s a very pow­er­ful and expres­sive idiom, enabling both the writer and read­er of code to con­cen­trate on the impor­tant thing that’s hap­pen­ing. It’s also entire­ly option­al.You can just as eas­i­ly type:

    for my $foo (1..5) {
        say $foo;
    }
    

    But where’s the mag­ic in that?

    Magic variables and use English

    We men­tioned the $_vari­able above, and that it could also be spelled $ARGif you add use Englishto your code. It can be hard to read code with large amounts of punc­tu­a­tion, though, and even hard­er to remem­ber what each vari­able does. Thankfully the Englishmod­ule pro­vides alias­es, and the per­l­varman page lists them in order. It’s much eas­i­er to read and write things like $LIST_SEPARATOR, $PROCESS_ID, or $MATCHrather than $", $$, and $&, and goes a long way towards reduc­ing Perl’s rep­u­ta­tion as a write-​only language.

    List and scalar contexts

    Like nat­ur­al lan­guages, Perl has a con­cept of ​“con­text” in which words mean dif­fer­ent things depend­ing on their sur­round­ings. In Perl’s case, expres­sions may behave dif­fer­ent­ly depend­ing on whether they expect to pro­duce a list of val­ues or a sin­gle val­ue, called a scalar. Here’s a triv­ial example:

    my @foo = (1, 2, 3); # list context, @foo contains the list
    my $bar = (1, 2, 3); # scalar context, $bar contains 3
    

    In the first line, we assign the list of num­bers (1, 2, 3)to the array @foo. But in the sec­ond line, we’re assign­ing to the scalar vari­able $bar, which now con­tains the last item in the list.

    Here’s anoth­er exam­ple, using the reversefunction:

    my @foo = ('one', 'two', 'three');
    my @bar = reverse @foo; # @bar contains ('three', 'two', 'one')
    my $baz = reverse @foo; # $baz contains 'eerhtowteno'
    

    In list con­text, reversetakes its argu­ments and returns them in the oppo­site order. But in scalar con­text, it con­cate­natesall of the argu­ments togeth­er and returns a string with the char­ac­tersin oppo­site order.

    In gen­er­al, ​“there is no gen­er­al rule for deduc­ing a func­tion’s behav­ior in scalar con­text from its behav­ior in list con­text.” (Dominus 1998)You’ll just have to look up the func­tionto deter­mine what it does, though in gen­er­al, it does what you want, but if you want to force scalar con­text use the scalaroperator:

    my @foo = ('aa', 'aab', 'bbc');
    my @bar = scalar grep /aa/, @foo; # returns a list (2), counting the number of matches
    

    Hash slices

    One of Perl’s three built-​in data types is the hash, also known as an asso­cia­tive array. It’s an unordered col­lec­tion of scalars indexed by string, rather than the num­bers used by nor­mal arrays. It’s a use­ful con­struct, and you can devel­op com­pli­cat­ed data struc­turesusing just scalars, arrays, and hash­es. What’s not wide­ly known is that you can access sev­er­al ele­ments of of a hash using a hash slice, using syn­tax that’s sim­i­lar to array slices. Here’s an example:

    my ($who, $home) = @ENV{'USER', 'HOME'};
    

    It works the oth­er way, too: you can assign to a slice.

    @colors{'red', 'green', 'blue'} = (0xff0000, 0x00ff00, 0x0000ff);
    

    I use this a lot when assign­ing argu­ments received from func­tions or meth­ods (see my pre­vi­ous arti­cle on sub­rou­tine sig­na­tures):

    use v5.24; # for postfix dereferencing
    use Types::Standard qw(Str Int);
    use Type::Params 'compile_named';
    
    foo('hello', 42);
    
    sub foo {
        state $check = compile_named(
            param1 => Str,
            param2 => Int, {optional => 1},
        );
        my ($param1, $param2) =
            $check->(@_)->@{'param1', 'param2'};
    
        say $param1, $param2;
    }
    

    In the exam­ple above, $check->(@_)returns the type-​checked argu­ments to the foo()func­tion cour­tesy of Type::Params’ compile_named()func­tion. It’s returned as a hash ref­er­ence, and since hash­es are unordered, we spec­i­fy the order in which we want the val­ues by deref­er­enc­ing and then slic­ing the result­ing hash. The post­fix deref­er­enc­ing syn­taxwas added in Perl 5.20 and made a default fea­ture in 5.24, and reduces the num­ber of nest­ed brack­ets and braces we have to deal with.

    Conclusion

    I hope this arti­cle has giv­en you a taste of some of the mag­ic avail­able in the Perl lan­guage. It’s these sort of fea­tures that make pro­gram­ming in it a bit more joy­ful. As always, check the doc­u­men­ta­tionfor com­plete infor­ma­tion on these and oth­er top­ics, or look for answers and ask ques­tions on PerlMonksor Stack Overflow.