Tag: grep

  • Perl lightning talk: ​“Don’t Fear map and grep”

    This week’s Perl and Raku Conference 2022in Houston was packed with great pre­sen­ta­tions, and I humbly added to them with a five-​ish minute light­ning talkon two of Perl’s more mis­un­der­stood func­tions: mapand grep.

    Sorry about the ​”um”s and ​”ah”s…

    I’ve writ­ten much about list pro­cess­ing in Perl, and this talk was based on the fol­low­ing blog posts:

    Overall I loved attend­ing the con­fer­ence, and it real­ly invig­o­rat­ed my par­tic­i­pa­tion in the Perl com­mu­ni­ty. Stay tuned as I resume reg­u­lar posting!

    Update for Raku

    On Twitter I nudgedpromi­nent Raku hack­er (and recov­ered Perl hack­er) Elizabeth Mattijsen to write about the Raku pro­gram­ming language’s mapand grepfunc­tion­al­i­ty. Check out her five-​part series on DEV.to.

  • Perl list processing is for hashes, too

    Perl list processing is for hashes, too

    This month I start­ed a new job at Alert Logic, a cyber­se­cu­ri­ty provider with Perl (among many oth­er things) at its beat­ing heart. I’ve been learn­ing a lot, and part of the process has been under­stand­ing the APIs in the code base. To that end, I’ve been writ­ing small test scripts to tease apart data struc­tures, using Perl array-​processing, list-​processing, and hash- (i.e., asso­cia­tive array)-processing func­tions.

    I’ve cov­ered map, grep, and friends a cou­ple timesbefore. Most recent­ly, I described using List::Util’s anyfunc­tionto check if a con­di­tion is true for any item in a list. In the sim­plest case, you can use it to check to see if a giv­en val­ue is in the list at all:

    use feature 'say';
    use List::Util 'any';
    my @colors =
      qw(red orange yellow green blue indigo violet);
    say 'matched' if any { /^red$/ } @colors;

    However, if you’re going to be doing this a lot with arbi­trary strings, Perl FAQ sec­tion 4 advis­esturn­ing the array into the keys of a hash and then check­ing for mem­ber­ship there. For exam­ple, here’s a sim­ple script to check if the col­ors input (either from the key­board or from files passed as argu­ments) are in the rainbow:

    #!/usr/bin/env perl
    
    use v5.22; # introduced <<>> for safe opening of arguments
    use warnings;
     
    my %in_colors = map {$_ => 1}
      qw(red orange yellow green blue indigo violet);
    
    while (<<>>) {
      chomp;
      say "$_ is in the rainbow" if $in_colors{$_};
    }

    List::Util has a bunch of func­tions for pro­cess­ing lists of pairsthat I’ve found use­ful when paw­ing through hash­es. pairgrep, for exam­ple, acts just like grepbut instead assigns $aand $bto each key and val­ue passed in and returns the result­ing pairs that match. I’ve used it as a quick way to search for hash entries match­ing cer­tain val­ue conditions:

    use List::Util 'pairgrep';
    my %numbers = (zero => 0, one => 1, two => 2, three => 3);
    my %odds = pairgrep {$b % 2} %numbers;

    Sure, you could do this by invok­ing a mix of plain grep, keys, and a hash slice, but it’s nois­i­er and more repetitive:

    use v5.20; # for key/value hash slice 
    my %odds = %numbers{grep {$numbers{$_} % 2} keys %numbers};

    pairgrep​’s com­piled C‑based XScode can also be faster, as evi­denced by this Benchmarkscript that works through a hash made of the Unix wordsfile(479,828 entries on my machine):

    #!/usr/bin/env perl
    
    use v5.20;
    use warnings;
    use List::Util 'pairgrep';
    use Benchmark 'cmpthese';
    
    my (%words, $count);
    open my $fh, '<', '/usr/share/dict/words'
      or die "can't open words: $!";
    while (<$fh>) {
      chomp;
      $words{$_} = $count++;
    }
    close $fh;
    
    cmpthese(100, {
      grep => sub {
        my %odds = %words{grep {$words{$_} % 2} keys %words};
      },
      pairgrep => sub {
        my %odds = pairgrep {$b % 2} %words;
      },
    } );

    Benchmark out­put:

               Rate     grep pairgrep
    grep     1.47/s       --     -20%
    pairgrep 1.84/s      25%       --

    In gen­er­al, I urge you to work through the Perl doc­u­men­ta­tion​’s tuto­ri­als on ref­er­ences, lists of lists, the data struc­tures cook­book, and the FAQs on arrayand hash manip­u­la­tion. Then dip into the var­i­ous list-​processing mod­ules(espe­cial­ly the includ­ed List::Utiland CPAN’s List::SomeUtils) for ready-​made func­tions for com­mon oper­a­tions. You’ll find a wealth of tech­niques for cre­at­ing, man­ag­ing, and pro­cess­ing the data struc­tures that your pro­grams need.

  • Sweeter Perl exception classes

    Sweeter Perl exception classes

    What about My::Favorite::Module?

    I men­tioned at the Ephemeral Miniconflast month that as soon as I write about one Perl mod­ule (or five), some­one inevitably brings up anoth­er (or sev­en) I’ve missed. And of course, it hap­pened again last week: no soon­er had I writ­ten in pass­ing that I was using Exception::Classthan the denizens of the Libera Chat IRC #perl chan­nelinsist­ed I should use Throwableinstead for defin­ing my excep­tions. (I’ve already blogged about var­i­ous ways of catch­ing excep­tions.)

    Why Throwable? Aside from Exception::Class’s author rec­om­mend­ing itover his own work due to a “nicer, more mod­ern inter­face,”Throwable is a Moo role, so it’s com­pos­able into class­es along with oth­er rolesinstead of muck­ing about with mul­ti­ple inher­i­tance. This means that if your excep­tions need to do some­thing reusable in your appli­ca­tion like log­ging, you can also con­sume a role that does that and not have so much dupli­cate code. (No, I’m not going to pick a favorite log­ging mod­ule; I’ll prob­a­bly get that wrong too.)

    However, since Throwable is a role instead of a class, I would have to define sev­er­al addi­tion­al packages in my tiny mod­uli­noscript from last week, one for each excep­tion class I want. The beau­ty of Exception::Class is its sim­ple declar­a­tive nature: just useit and pass a listof desired class names along with options for attrib­ut­es and what­not. What’s need­ed for sim­ple use cas­es like mine is a declar­a­tive syn­tax for defin­ing sev­er­al excep­tion class­es with­out the noise of mul­ti­ple packages.

    Enter Throwable::SugarFactory, a mod­ule that enables you to do just that by adding an exceptionfunc­tion for declar­ing excep­tion class­es. (There’s also the similarly-​named Throwable::Factory; see the above dis­cus­sion about nev­er being able to cov­er everybody’s favorites.) The exceptionfunc­tion takes three argu­ments: the name of the desired excep­tion class as a string, a descrip­tion, and an option­al list of instruc­tions Moo uses to build the class. It might look some­thing like this:

    package Local::My::Exceptions;
    use Throwable::SugarFactory;
    
    exception GenericError  => 'something bad happened';
    exception DetailedError => 'something specific happened' =>
      ( has => [ message => ( is => 'ro' ) ] );
    
    1;

    Throwable::SugarFactory takes care of cre­at­ing con­struc­tor func­tions in Perl-​style snake_caseas well as func­tions for detect­ing what kind of excep­tion is being caught, so you can use your new excep­tion library like this:

    #!/usr/bin/env perl
    
    use experimental qw(isa);
    use Feature::Compat::Try;
    use JSON::MaybeXS;
    use Local::My::Exceptions;
    
    try {
        die generic_error();
    }
    catch ($e) {
        warn 'whoops!';
    }
    
    try {
        die detailed_error( message => 'you got me' );
    }
    catch ($e) {
        die encode_json( $e->to_hash )
          if $e isa DetailedError and defined $e->message;
        $e->throw if $e->does('Throwable');
        die $e;
    }

    The above also demon­strates a cou­ple of oth­er Throwable::SugarFactory fea­tures. First, you get a to_hashmethod that returns a hash ref­er­ence of all excep­tion data, suit­able for seri­al­iz­ing to JSON. Second, you get all of Throwable’s meth­ods, includ­ing throwfor re-​throwing exceptions. 

    So where does this leave last week’s FOAAS.commod­uli­no client demon­stra­tion of object mock­ingtests? With a lit­tle bit of rewrit­ing to define and then use our sweet­er excep­tion library, it looks like this. You can reviewfor a descrip­tion of the rest of its workings.

    #!/usr/bin/env perl
    
    package Local::CallFOAAS::Exceptions;
    use Throwable::SugarFactory;
    
    BEGIN {
        exception NoMethodError =>
          'no matching WebService::FOAAS method' =>
          ( has => [ method => ( is => 'ro' ) ] );
        exception ServiceError =>
          'error from WebService::FOAAS' =>
          ( has => [ message => ( is => 'ro' ) ] );
    }
    
    package Local::CallFOAAS;  # this is a modulino
    use Test2::V0;             # enables strict, warnings, utf8
    
    # declare all the new stuff we're using
    use feature qw(say state);
    use experimental qw(isa postderef signatures);
    use Feature::Compat::Try;
    use Syntax::Construct qw(non-destructive-substitution);
    
    use WebService::FOAAS ();
    use Package::Stash;
    BEGIN { Local::CallFOAAS::Exceptions->import() }
    
    my $foaas = Package::Stash->new('WebService::FOAAS');
    
    my $run_as =
        !!$ENV{CPANTEST}       ? 'test'
      : !defined scalar caller ? 'run'
      :                          undef;
    __PACKAGE__->$run_as(@ARGV) if defined $run_as;
    
    sub run ( $class, @args ) {
        try { say $class->call_method(@args) }
        catch ($e) {
            die 'No method ', $e->method, "\n"
              if $e isa NoMethodError;
            die 'Service error: ', $e->message, "\n"
              if $e isa ServiceError;
            die "$e\n";
        }
        return;
    }
    
    # Utilities
    
    sub methods ($) {
        state @methods = sort map s/^foaas_(.+)/$1/r,
          grep /^foaas_/, $foaas->list_all_symbols('CODE');
        return @methods;
    }
    
    sub call_method ( $class, $method = '', @args ) {
        state %methods = map { $_ => 1 } $class->methods();
        die no_method_error( method => $method )
          unless $methods{$method};
        return do {
            try { $foaas->get_symbol("&$method")->(@args) }
            catch ($e) { die service_error( message => $e ) }
        };
    }
    
    # Testing
    
    sub test ( $class, @ ) {
        state $stash = Package::Stash->new($class);
        state @tests = sort grep /^_test_/,
          $stash->list_all_symbols('CODE');
    
        for my $test (@tests) {
            subtest $test => sub {
                try { $class->$test() }
                catch ($e) { diag $e }
            };
        }
        done_testing();
        return;
    }
    
    sub _test_can ($class) {
        state @subs = qw(run call_method methods test);
        can_ok $class, \@subs, "can do: @subs";
        return;
    }
    
    sub _test_methods ($class) {
        my $mock = mock 'WebService::FOAAS' => ( track => 1 );
    
        for my $method ( $class->methods() ) {
            $mock->override( $method => 1 );
    
            ok lives { $class->call_method($method) },
              "$method lives";
            ok scalar $mock->sub_tracking->{$method}->@*,
              "$method called";
        }
        return;
    }
    
    sub _test_service_failure ($class) {
        my $mock = mock 'WebService::FOAAS';
    
        for my $method ( $class->methods() ) {
            $mock->override( $method => sub { die 'mocked' } );
    
            my $exception =
              dies { $class->call_method($method) };
            isa_ok $exception, [ServiceError],
              "$method throws ServiceError on failure";
            like $exception->message, qr/^mocked/,
              "correct error in $method exception";
        }
        return;
    }
    
    1;

    [Updated, thanks to Dan Book, Karen Etheridge, and Bob Kleemann]The only goofy bit above is the need to put the exceptioncalls in a BEGINblock and then explic­it­ly call BEGIN { Local::CallFOAAS::Exceptions->import() }. Since the two pack­ages are in the same file, I can’t do a usestate­ment since the implied requirewould look for a cor­re­spond­ing file or entry in %INC. (You can get around this by mess­ing with %INCdirect­ly or through a mod­ule like me::inlinedthat does that mess­ing for you, but for a single-​purpose mod­uli­no like this it’s fine.)


  • Vicious (test) mockery of a Perl modulino

    Vicious (test) mockery of a Perl modulino

    Over the past two years, I’ve got­ten back into play­ing Dungeons & Dragons, the famous table­top fan­ta­sy role-​playing game. As a soft­ware devel­op­er and musi­cian, one of my favorite char­ac­ter class­es to play is the bard, a mag­i­cal and inspir­ing per­former or word­smith. The list of basic bardic spells includes Vicious Mockery, enchant­i­ng ver­bal barbs that have the pow­er to psy­chi­cal­ly dam­age and dis­ad­van­tage an oppo­nent even if they don’t under­stand the words. (Can you see why this is so appeal­ing to a coder?)

    Mocking has a role to play in soft­ware test­ing as well, in the form of mock objectsthat sim­u­late parts of a sys­tem that are too brit­tle, too slow, too com­pli­cat­ed, or oth­er­wise too finicky to use in real­i­ty. They enable dis­crete unit test­ingwith­out rely­ing on depen­den­cies exter­nal to the code being test­ed. Mocks are great for data­bas­es, web ser­vices, or oth­er net­work resources where the goal is to test what you wrote, not what’s out in ​“the cloud” somewhere.

    Speaking of web ser­vices and mock­ing, one of my favorites is the long-​running FOAAS(link has lan­guage not safe for work), a sur­pris­ing­ly expan­sive RESTfulinsult ser­vice. There’s a cor­re­spond­ing Perl client API, of course, but what I was miss­ing was a handy Perl script to call that API from the ter­mi­nal com­mand line. So I wrote the fol­low­ing over Thanksgivingbreak, try­ing to keep it sim­ple while also show­ing the basics of mock­ing such an API. It also demon­strates some new­er Perl syn­tax and test­ing tech­niques as well as bri­an d foy​’s mod­uli­nocon­cept from Mastering Perl(sec­ond edi­tion, 2014)that mar­ries script and mod­ule into a self-​contained exe­cutable library.

    #!/usr/bin/env perl
    
    package Local::CallFOAAS;  # this is a modulino
    use Test2::V0;             # enables strict, warnings, utf8
    
    # declare all the new stuff we're using
    use feature qw(say state);
    use experimental qw(isa postderef signatures);
    use Feature::Compat::Try;
    use Syntax::Construct qw(non-destructive-substitution);
    
    use WebService::FOAAS ();
    use Package::Stash;
    use Exception::Class (
        NoMethodException => {
            alias  => 'throw_no_method',
            fields => 'method',
        },
        ServiceException => { alias => 'throw_service' },
    );
    
    my $foaas = Package::Stash->new('WebService::FOAAS');
    
    my $run_as =
        !!$ENV{CPANTEST}       ? 'test'
      : !defined scalar caller ? 'run'
      :                          undef;
    __PACKAGE__->$run_as(@ARGV) if defined $run_as;
    
    sub run ( $class, @args ) {
        try { say $class->call_method(@args) }
        catch ($e) {
            die 'No method ', $e->method, "\n"
              if $e isa NoMethodException;
            die 'Service error: ', $e->error, "\n"
              if $e isa ServiceException;
            die "$e\n";
        }
        return;
    }
    
    # Utilities
    
    sub methods ($) {
        state @methods = sort map s/^foaas_(.+)/$1/r,
          grep /^foaas_/, $foaas->list_all_symbols('CODE');
        return @methods;
    }
    
    sub call_method ( $class, $method = '', @args ) {
        state %methods = map { $_ => 1 } $class->methods();
        throw_no_method( method => $method )
          unless $methods{$method};
        return do {
            try { $foaas->get_symbol("&$method")->(@args) }
            catch ($e) { throw_service( error => $e ) }
        };
    }
    
    # Testing
    
    sub test ( $class, @ ) {
        state $stash = Package::Stash->new($class);
        state @tests = sort grep /^_test_/,
          $stash->list_all_symbols('CODE');
    
        for my $test (@tests) {
            subtest $test => sub {
                try { $class->$test() }
                catch ($e) { diag $e }
            };
        }
        done_testing();
        return;
    }
    
    sub _test_can ($class) {
        state @subs = qw(run call_method methods test);
        can_ok( $class, \@subs, "can do: @subs" );
        return;
    }
    
    sub _test_methods ($class) {
        my $mock = mock 'WebService::FOAAS' => ( track => 1 );
    
        for my $method ( $class->methods() ) {
            $mock->override( $method => 1 );
    
            ok lives { $class->call_method($method) },
              "$method lives";
            ok scalar $mock->sub_tracking->{$method}->@*,
              "$method called";
        }
        return;
    }
    
    sub _test_service_failure ($class) {
        my $mock = mock 'WebService::FOAAS';
    
        for my $method ( $class->methods() ) {
            $mock->override( $method => sub { die 'mocked' } );
    
            my $exception =
              dies { $class->call_method($method) };
            isa_ok $exception, ['ServiceException'],
              "$method throws ServiceException on failure";
            like $exception->error, qr/^mocked/,
              "correct error in $method exception";
        }
        return;
    }
    
    1;

    Let’s walk through the code above.

    Preliminaries

    First, there’s a gener­ic she­bang lineto indi­cate that Unix and Linux sys­tems should use the perlexe­cutable found in the user’s PATH via the envcom­mand. I declare a pack­age name (in the Local:: name­space) so as not to pol­lute the default mainpack­age of oth­er scripts that might want to requirethis as a mod­ule. Then I use the Test2::V0 bun­dlefrom Test2::Suitesince the embed­ded test­ing code uses many of its func­tions. This also has the side effect of enabling the strict, warn­ings, and utf8prag­mas, so there’s no need to explic­it­ly usethem here.

    (Why Test2instead of Test::Moreand its deriv­a­tives and add-​ons? Both are main­tained by the same author, who rec­om­mends the for­mer. I’m see­ing more and more mod­ules using it, so I thought this would be a great oppor­tu­ni­ty to learn.)

    I then declare all the new-​ish Perl fea­tures I’d like to use that need to be explic­it­ly enabled so as not to sac­ri­fice back­ward com­pat­i­bil­i­tywith old­er ver­sions of Perl 5. As of this writ­ing, some of these fea­tures (the isaclass instance oper­a­tor, named argu­ment sub­rou­tine sig­na­tures, and try/​catchexcep­tion han­dlingsyn­tax) are con­sid­ered experimental, with the lat­ter enabled in old­er ver­sions of Perl via the Feature::Compat::Trymod­ule. The friend­lier post­fix deref­er­enc­ingsyn­tax was main­lined in Perl ver­sion 5.24, but ver­sions 5.20 and 5.22 still need it exper­i­men­tal. Finally, I use Syntax::Constructto announce the /rflag for non-​destructive reg­u­lar expres­sion text sub­sti­tu­tions intro­duced in ver­sion 5.14.

    Next, I bring in the afore­men­tioned FOAAS Perl API with­out import­ing any of its func­tions, Package::Stashto make metapro­gram­mingeas­i­er, and a cou­ple of excep­tion class­esso that the com­mand line func­tion and oth­er con­sumers might bet­ter tell what caused a fail­ure. In prepa­ra­tion for the meth­ods below dynam­i­cal­ly dis­cov­er­ing what func­tions are pro­vid­ed by WebService::FOAAS, I gath­er upits sym­bol table (or stash) into the $foaasvariable.

    The next block deter­mines how, if at all, I’m going to run the code as a script. If the CPANTESTenvi­ron­ment vari­able is set, I’ll call the testclass method sub, but if there’s no sub­rou­tine call­ingme I’ll exe­cute the runclass method. Either will receive the com­mand line argu­ments from @ARGV. If nei­ther of these con­di­tions is true, do noth­ing; the rest of the code is method declarations.

    Modulino methods, metaprogramming, and exceptions

    The first of these is the runmethod. It’s a thin wrap­per around the call_methodclass method detailed below, either out­putting its result or dieing with an appro­pri­ate error depend­ing on the class of excep­tion thrown. Although I chose not to write tests for this out­put, future tests might call this method and catch these rethrown excep­tions to match against them. The mes­sages end with a \nnew­line char­ac­ter so dieknows not to append the cur­rent script line number.

    Next is a util­i­ty method called methodsthat uses Package::Stash’s list_all_symbolsto retrieve the names of all named CODEblocks (i.e., subs) from WebService::FOAAS’s sym­bol table. Reading from right to left, these are then fil­tered with grepto only find those begin­ning in foaas_and then trans­formed with mapto remove that pre­fix. The list is then sorted and stored in a statevari­able and returned so it need not be ini­tial­ized again.

    (As an aside, although perlcriticstern­ly warns against it I’ve cho­sen the expres­sion forms of grepand maphere over their block formsfor sim­plic­i­ty’s sake. It’s OK to bend the rulesif you have a good reason.)

    sub call_methodis where the real action takes place. Its para­me­ters are the class that called it, the name of a FOAAS $method(default­ed to the emp­ty string), and an array of option­al argu­ments in @args. I build a hash or asso­cia­tive array from the ear­li­er methodsmethod which I then use to see if the passed method name is one I know about. If not, I throw a NoMethodExceptionusing the throw_no_methodalias func­tion cre­at­ed when I used Exception::Classat the begin­ning. Using a func­tion instead of NoMethodException->throw()means that it’s checked at com­pile time rather than run­time, catch­ing typos.

    I get the sub­rou­tine (denot­ed by a &sig­il) named by $methodfrom the $foaasstash and pass it any fur­ther received argu­ments from @args. If that WebService::FOAAS sub­rou­tine throws an excep­tion it’ll be caught and re-​thrown as a ServiceException; oth­er­wise call_methodreturns the result. It’s up to the caller to deter­mine what, if any­thing, to do with that result or any thrown exceptions.

    Testing the modulino with mocks

    This is where I start using those Test2::Suite toolsI men­tioned at the begin­ning. The testclass method starts by build­ing a fil­tered list of all subs begin­ning with _test_in the cur­rent class, much like methodsdid above with WebService::FOAAS. I then loop through that list of subs, run­ning each as a subtestcon­tain­ing a class method with any excep­tions report­ed as diag­nos­tics.

    The rest of the mod­uli­no is sub­test meth­ods, start­ing with a sim­ple _test_cansan­i­ty check for the pub­lic meth­ods in the class. Following that is _test_methods, which starts by mocking the WebService::FOAAS pack­age and telling Test2::Mock I want to trackany added, over­rid­den, or set subs. I then loop through all the method names returned by the methodsclass method, overrideing each one to return a sim­ple true val­ue. I then test pass­ing those names to call_methodand use the hash ref­er­ence returned by sub_trackingto check that the over­rid­den subwas called. This seems a lot sim­pler than the Test::Builder-based mock­ing libraries I’ve tried like Test::MockModuleand Test::MockObject.

    _test_service_failureacts in much the same way, check­ing that call_methodcor­rect­ly throws ServiceExceptions if the wrapped WebService::FOAAS func­tion dies. The main dif­fer­ence is that the mocked WebService::FOAAS subs are now over­rid­den with a code ref­er­ence (sub { die 'mocked' }), which call_methoduses to pop­u­late the rethrown ServiceException​’s errorfield.

    Wrapping up

    With luck, this arti­cle has giv­en you some ideas, whether it’s in mak­ing scripts (per­haps lega­cy code) testable to improve them, or writ­ing bet­ter unit tests that mock depen­den­cies, or delv­ing a lit­tle into metapro­gram­ming so you can dynam­i­cal­ly sup­port and test new fea­tures of said depen­den­cies. I hope you haven’t come away too offend­ed, at least. Let me know in the com­ments what you think.

  • Better Perl: Four list processing best practices with map, grep, and more

    Better Perl: Four list processing best practices with map, grep, and more

    Six months ago I gave an overview of Perl’s list pro­cess­ing fun­da­men­tals, briefly describ­ing what lists are and then intro­duc­ing the built-​in mapand grepfunc­tions for trans­form­ing and fil­ter­ing them. Later on, I com­piled a list (how appro­pri­ate) of list pro­cess­ing mod­ules avail­able via CPAN, not­ing there’s some con­fus­ing dupli­ca­tion of effort. But you’re a busy devel­op­er, and you just want to know the Right Thing To Do™ when faced with a list pro­cess­ing challenge.

    First, some cred­it is due: these are all restate­ments of sev­er­al Perl::Critic poli­cieswhich in turn cod­i­fy stan­dards described in Damian Conway’s Perl Best Practices(2005). I’ve repeat­ed­ly rec­om­mend­edthe lat­ter as a start­ing point for higher-​quality Perl devel­op­ment. Over the years these prac­tices con­tin­ue to be re-​evaluated (includ­ing by the author him­self) and var­i­ous authors release new pol­i­cy mod­ules, but perlcriticremains a great tool for ensur­ing you (and your team or oth­er con­trib­u­tors) main­tain a con­sis­tent high stan­dard in your code.

    With that said, on to the recommendations!

    Don’t use grepto check if anylist elements match

    It might sound weird to lead off by rec­om­mend­ing notto use grep, but some­times it’s not the right tool for the job. If you’ve got a list and want to deter­mine if a con­di­tion match­es any item in it, you might try:

    if (grep { some_condition($_) } @my_list) {
        ... # don't do this!
    }

    Yes, this works because (in scalar con­text) grepreturns the num­ber of match­es found, but it’s waste­ful, check­ing every ele­ment of @my_list(which could be lengthy) before final­ly pro­vid­ing a result. Use the stan­dard List::Utilmodule’s anyfunc­tion, which imme­di­ate­ly returns (“short-​circuits”) on the first match:

    use List::Util 1.33 qw(any);

    if (any { some_condition($_) } @my_list) {
    ... # do something
    }

    Perl has includ­ed the req­ui­site ver­sion of this mod­ule since ver­sion 5.20in 2014; for ear­li­er releas­es, you’ll need to update from CPAN. List::Util has many oth­er great list-​reduction, key/​value pair, and oth­er relat­ed func­tionsyou can import into your code, so check it out before you attempt to re-​invent any wheels.

    As a side note for web devel­op­ers, the Perl Dancer frame­workalso includes an anykey­wordfor declar­ing mul­ti­ple HTTP routes, so if you’re mix­ing List::Util in there don’t import it. Instead, call it explic­it­ly like this or you’ll get an error about a rede­fined function:

    use List::Util 1.33;

    if (List::Util::any { some_condition($_) } @my_list) {
    ... # do something
    }

    This rec­om­men­da­tion is cod­i­fied in the BuiltinFunctions::ProhibitBooleanGrepPerl::Critic pol­i­cy, comes direct­ly from Perl Best Practices, and is rec­om­mend­edby the Software Engineering Institute Computer Emergency Response Team (SEI CERT)’s Perl Coding Standard.

    Don’t change $_in mapor grep

    I men­tioned this back in March, but it bears repeat­ing: mapand grepare intend­ed as pure func­tions, not muta­torswith side effects. This means that the orig­i­nal list should remain unchanged. Yes, each ele­ment alias­es in turn to the $_spe­cial vari­able, but that’s for speed and can have sur­pris­ing results if changed even if it’s tech­ni­cal­ly allowed. If you need to mod­i­fy an array in-​place use some­thing like:

    for (@my_array) {
    $_ = ...; # make your changes here
    }

    If you want some­thing that lookslike mapbut won’t change the orig­i­nal list (and don’t mind a few CPAN depen­den­cies), con­sid­er List::SomeUtils’ applyfunction:

    use List::SomeUtils qw(apply);
    
    my @doubled_array = apply {$_ *= 2} @old_array;

    Lastly, side effects also include things like manip­u­lat­ing oth­er vari­ables or doing input and out­put. Don’t use mapor grepin a void con­text (i.e., with­out a result­ing array or list); do some­thing with the results or use a foror foreachloop:

    map { print foo($_) } @my_array; # don't do this
    print map { foo($_) } @my_array; # do this instead

    map { push @new_array, foo($_) } @my_array; # don't do this
    @new_array = map { foo($_) } @my_array; # do this instead

    This rec­om­men­da­tion is cod­i­fied by the BuiltinFunctions::ProhibitVoidGrep, BuiltinFunctions::ProhibitVoidMap, and ControlStructures::ProhibitMutatingListFunctionsPerl::Critic poli­cies. The lat­ter comes from Perl Best Practicesand is an SEI CERT Perl Coding Standard rule.

    Use blocks with mapand grep, not expressions

    You can call mapor greplike this (paren­the­ses are option­alaround built-​in functions):

    my @new_array  = map foo($_), @old_array; # don't do this
    my @new_array2 = grep !/^#/, @old_array; # don't do this

    Or like this:

    my @new_array  = map { foo($_) } @old_array;
    my @new_array2 = grep {!/^#/} @old_array;

    Do it the sec­ond way. It’s eas­i­er to read, espe­cial­ly if you’re pass­ing in a lit­er­al list or mul­ti­ple arrays, and the expres­sion forms can con­ceal bugs. This rec­om­men­da­tion is cod­i­fied by the BuiltinFunctions::RequireBlockGrepand BuiltinFunctions::RequireBlockMapPerl::Critic poli­cies and comes from Perl Best Practices.

    Refactor multi-​statement maps, greps, and other list functions

    map, grep, and friends should fol­low the Unix phi­los­o­phy of ​“Do One Thing and Do It Well.”Your read­abil­i­ty and main­tain­abil­i­ty drop with every state­ment you place inside one of their blocks. Consider junior devel­op­ers and future main­tain­ers (this includes you!) and refac­tor any­thing with more than one state­ment into a sep­a­rate sub­rou­tine or at least a forloop. This goes for list pro­cess­ing func­tions (like the afore­men­tioned any) import­ed from oth­er mod­ules, too.

    This rec­om­men­da­tion is cod­i­fied by the Perl Best Practices-inspired BuiltinFunctions::ProhibitComplexMappingsand BuiltinFunctions::RequireSimpleSortBlockPerl::Critic poli­cies, although those only cov­er mapand sortfunc­tions, respectively.


    Do you have any oth­er sug­ges­tions for list pro­cess­ing best prac­tices? Feel free to leave them in the com­ments or bet­ter yet, con­sid­er cre­at­ing new Perl::Critic poli­ciesfor them or con­tact­ing the Perl::Critic teamto devel­op them for your organization.