Template proces­sors and engines are one of those pieces of soft­ware where it seems every devel­op­er wants to rein­vent the wheel. Goodness knows I’ve done it ear­li­er in my career. Tell me if this sounds familiar:

  1. You need to mix data into a doc­u­ment so you start with Perl’s string inter­po­la­tion in "dou­ble quotes" or sprintf for­mats. (Or maybe you inves­ti­gate formats, but the less said about them the bet­ter.)
  2. You real­ize your doc­u­ments need to dis­play things based on cer­tain con­di­tions, or you want to loop over a list or some oth­er structure.
  3. You add these fea­tures via key­word pars­ing and escape char­ac­ters, think­ing it’s OK since this is just a small bespoke project.
  4. Before you know it you’ve invent­ed anoth­er domain-​specific lan­guage (DSL) and have to sup­port it on top of the appli­ca­tion you were try­ing to deliv­er in the first place.

Stop. Just stop. Decades of oth­ers who have walked this same path have already done this for you. Especially if you’re using a web frame­work like Dancer, Mojolicious, or Catalyst, where the tem­plate proces­sor is either built-​in or plug­gable from CPAN. Even if you’re not devel­op­ing a web appli­ca­tion there are sev­er­al general-​purpose options of var­i­ous capa­bil­i­ties like Template Toolkit and Template::Mustache. Investigate the alter­na­tives and deter­mine if they have the fea­tures, per­for­mance, and sup­port you need. If you’re sure none of them tru­ly meet your unique require­ments, then maybe, maybe con­sid­er rolling your own.

Whatever you decide, real­ize that as your appli­ca­tion or web­site grows your invest­ment in that selec­tion will only deep­en. Porting to a new tem­plate proces­sor can be as chal­leng­ing as port­ing any source code to a new pro­gram­ming language.

Unfortunately, there are about as many opin­ions on how to choose a tem­plate proces­sor as there are tem­plate proces­sors. For exam­ple, in 2013 Roland Koehler wrote a good Python-​oriented arti­cle on sev­er­al con­sid­er­a­tions and the dif­fer­ent approach­es avail­able. Although he end­ed up devel­op­ing his own (quelle sur­prise), he makes a good case that a tem­plate proces­sor ought to at least pro­vide var­i­ous log­ic con­structs as well as embed­ded expres­sions, if not a full pro­gram­ming lan­guage. Koehler specif­i­cal­ly warns against the lat­ter, though, as a tem­plate devel­op­er might change an application’s data mod­el, to say noth­ing of the pos­si­bil­i­ty of exe­cut­ing arbi­trary destruc­tive code.

I can appre­ci­ate this rea­son­ing. I’ve suc­cess­ful­ly used Perl tem­plate proces­sors like the afore­men­tioned Template::Toolkit (which has both log­ic direc­tives and an option­al facil­i­ty for eval­u­at­ing Perl code) and Text::Xslate (which sup­ports sev­er­al tem­plate syn­tax­es includ­ing a sub­set of Template::Toolkits, but with­out the abil­i­ty to embed Perl code). We use the lat­ter at work com­bined with Text::Xslate::Bridge::TT2Likes emu­la­tion of var­i­ous Template::Toolkit vir­tu­al meth­ods and it’s served us well.

But using those mod­ules’ DSLs means more sophis­ti­cat­ed tasks need extra time and effort find­ing the cor­rect log­ic and expres­sions. This also assumes that their designer(s) have antic­i­pat­ed my needs either through built-​in fea­tures or exten­sions. I’m already writ­ing Perl; why should I switch to anoth­er, more lim­it­ed lan­guage and envi­ron­ment pro­vid­ed I can remain dis­ci­plined enough to avoid issues like those described above by Koehler?

So for my per­son­al projects, I favor tem­plate proces­sors that use the full pow­er of the Perl lan­guage like Mojolicious’ embed­ded Perl ren­der­er or the ven­er­a­ble Text::Template for non-​web appli­ca­tions. It saves me time and I’ll like­ly want more than any DSL can pro­vide. This may not apply to your sit­u­a­tion, though, and I’m open to counter-arguments.

What’s your favorite tem­plate proces­sor and why? Let me know in the comments.

woman looking at the map

Six months ago I gave an overview of Perl’s list pro­cess­ing fun­da­men­tals, briefly describ­ing what lists are and then intro­duc­ing the built-​in map and grep func­tions for trans­form­ing and fil­ter­ing them. Later on, I com­piled a list (how appro­pri­ate) of list pro­cess­ing mod­ules avail­able via CPAN, not­ing there’s some con­fus­ing dupli­ca­tion of effort. But you’re a busy devel­op­er, and you just want to know the Right Thing To Do™ when faced with a list pro­cess­ing challenge.

First, some cred­it is due: these are all restate­ments of sev­er­al Perl::Critic poli­cies which in turn cod­i­fy stan­dards described in Damian Conway’s Perl Best Practices (2005). I’ve repeat­ed­ly rec­om­mend­ed the lat­ter as a start­ing point for higher-​quality Perl devel­op­ment. Over the years these prac­tices con­tin­ue to be re-​evaluated (includ­ing by the author him­self) and var­i­ous authors release new pol­i­cy mod­ules, but perlcritic remains a great tool for ensur­ing you (and your team or oth­er con­trib­u­tors) main­tain a con­sis­tent high stan­dard in your code.

With that said, on to the recommendations!

Don’t use grep to check if any list elements match

It might sound weird to lead off by rec­om­mend­ing not to use grep, but some­times it’s not the right tool for the job. If you’ve got a list and want to deter­mine if a con­di­tion match­es any item in it, you might try:

if (grep { some_condition($_) } @my_list) {
    ... # don't do this!

Yes, this works because (in scalar con­text) grep returns the num­ber of match­es found, but it’s waste­ful, check­ing every ele­ment of @my_list (which could be lengthy) before final­ly pro­vid­ing a result. Use the stan­dard List::Util module’s any func­tion, which imme­di­ate­ly returns (“short-​circuits”) on the first match:

use List::Util 1.33 qw(any);

if (any { some_condition($_) } @my_list) {
... # do something

Perl has includ­ed the req­ui­site ver­sion of this mod­ule since ver­sion 5.20 in 2014; for ear­li­er releas­es, you’ll need to update from CPAN. List::Util has many oth­er great list-​reduction, key/​value pair, and oth­er relat­ed func­tions you can import into your code, so check it out before you attempt to re-​invent any wheels.

As a side note for web devel­op­ers, the Perl Dancer frame­work also includes an any key­word for declar­ing mul­ti­ple HTTP routes, so if you’re mix­ing List::Util in there don’t import it. Instead, call it explic­it­ly like this or you’ll get an error about a rede­fined function:

use List::Util 1.33;

if (List::Util::any { some_condition($_) } @my_list) {
... # do something

This rec­om­men­da­tion is cod­i­fied in the BuiltinFunctions::ProhibitBooleanGrep Perl::Critic pol­i­cy, comes direct­ly from Perl Best Practices, and is rec­om­mend­ed by the Software Engineering Institute Computer Emergency Response Team (SEI CERT)’s Perl Coding Standard.

Don’t change $_ in map or grep

I men­tioned this back in March, but it bears repeat­ing: map and grep are intend­ed as pure func­tions, not muta­tors with side effects. This means that the orig­i­nal list should remain unchanged. Yes, each ele­ment alias­es in turn to the $_ spe­cial vari­able, but that’s for speed and can have sur­pris­ing results if changed even if it’s tech­ni­cal­ly allowed. If you need to mod­i­fy an array in-​place use some­thing like:

for (@my_array) {
$_ = ...; # make your changes here

If you want some­thing that looks like map but won’t change the orig­i­nal list (and don’t mind a few CPAN depen­den­cies), con­sid­er List::SomeUtilsapply function:

use List::SomeUtils qw(apply);

my @doubled_array = apply {$_ *= 2} @old_array;

Lastly, side effects also include things like manip­u­lat­ing oth­er vari­ables or doing input and out­put. Don’t use map or grep in a void con­text (i.e., with­out a result­ing array or list); do some­thing with the results or use a for or foreach loop:

map { print foo($_) } @my_array; # don't do this
print map { foo($_) } @my_array; # do this instead

map { push @new_array, foo($_) } @my_array; # don't do this
@new_array = map { foo($_) } @my_array; # do this instead

This rec­om­men­da­tion is cod­i­fied by the BuiltinFunctions::ProhibitVoidGrep, BuiltinFunctions::ProhibitVoidMap, and ControlStructures::ProhibitMutatingListFunctions Perl::Critic poli­cies. The lat­ter comes from Perl Best Practices and is an SEI CERT Perl Coding Standard rule.

Use blocks with map and grep, not expressions

You can call map or grep like this (paren­the­ses are option­al around built-​in functions):

my @new_array  = map foo($_), @old_array; # don't do this
my @new_array2 = grep !/^#/, @old_array; # don't do this

Or like this:

my @new_array  = map { foo($_) } @old_array;
my @new_array2 = grep {!/^#/} @old_array;

Do it the sec­ond way. It’s eas­i­er to read, espe­cial­ly if you’re pass­ing in a lit­er­al list or mul­ti­ple arrays, and the expres­sion forms can con­ceal bugs. This rec­om­men­da­tion is cod­i­fied by the BuiltinFunctions::RequireBlockGrep and BuiltinFunctions::RequireBlockMap Perl::Critic poli­cies and comes from Perl Best Practices.

Refactor multi-​statement maps, greps, and other list functions

map, grep, and friends should fol­low the Unix phi­los­o­phy of Do One Thing and Do It Well.” Your read­abil­i­ty and main­tain­abil­i­ty drop with every state­ment you place inside one of their blocks. Consider junior devel­op­ers and future main­tain­ers (this includes you!) and refac­tor any­thing with more than one state­ment into a sep­a­rate sub­rou­tine or at least a for loop. This goes for list pro­cess­ing func­tions (like the afore­men­tioned any) import­ed from oth­er mod­ules, too.

This rec­om­men­da­tion is cod­i­fied by the Perl Best Practices-inspired BuiltinFunctions::ProhibitComplexMappings and BuiltinFunctions::RequireSimpleSortBlock Perl::Critic poli­cies, although those only cov­er map and sort func­tions, respectively.

Do you have any oth­er sug­ges­tions for list pro­cess­ing best prac­tices? Feel free to leave them in the com­ments or bet­ter yet, con­sid­er cre­at­ing new Perl::Critic poli­cies for them or con­tact­ing the Perl::Critic team to devel­op them for your organization.

clear light bulb planter on gray rock

Twitter recent­ly rec­om­mend­ed a tweet to me (all hail the algo­rithm) tout­ing what the author viewed as the top 5 web devel­op­ment stacks.”

JavaScript/​Node.js options dom­i­nat­ed the four-​letter acronyms as expect­ed, but the fifth one sur­prised me: LAMP, the com­bi­na­tion of the Linux oper­at­ing sys­tem, Apache web serv­er, MySQL rela­tion­al data­base, and Perl, PHP, or Python pro­gram­ming lan­guages. A quick web search for sim­i­lar lists yield­ed sim­i­lar results. Clearly, this meme (in the Dawkins sense) has out­last­ed its pop­u­lar­iza­tion by tech pub­lish­er O’Reilly in the 2000s.

Originally coined in 1998 dur­ing the dot-​com” bub­ble, I had thought that the term LAMP” had fad­ed with devel­op­ers in the inter­ven­ing decades with the rise of language-​specific web frame­works for:

Certainly on the Perl side (with which I’m most famil­iar), the com­mu­ni­ty has long since rec­om­mend­ed the use of a frame­work built on the PSGI spec­i­fi­ca­tion, dep­re­cat­ing 1990s-​era CGI scripts and the mod_​perl Apache exten­sion. Although general-​purpose web servers like Apache or Nginx may be part of an over­all sys­tem, they’re typ­i­cal­ly used as prox­ies or load bal­ancers for Perl-​specific servers either pro­vid­ed by the frame­work or a third-​party mod­ule.

Granted, PHP still relies on web server-​specific mod­ules, APIs, or vari­a­tions of the FastCGI pro­to­col for inter­fac­ing with a web serv­er. And Python web appli­ca­tions typ­i­cal­ly make use of its WSGI pro­to­col either as a web serv­er exten­sion or, like the Perl exam­ples above, as a prox­ied stand­alone serv­er. But all of these are deploy­ment details and do lit­tle to describe how devel­op­ers imple­ment and extend a web application’s structure.

Note how the var­i­ous four-​letter JavaScript stacks (e.g., MERN, MEVN, MEAN, PERN) dif­fer­en­ti­ate them­selves most­ly by fron­tend frame­work (e.g., Angular, React, Vue.js) and maybe by the (rela­tion­al or NoSQL) data­base (e.g., MongoDB, MySQL, PostgreSQL). All how­ev­er seem stan­dard­ized on the Node.js run­time and Express back­end web frame­work, which could, in the­o­ry, be replaced with non-​JavaScript options like the more mature LAMP-​associated lan­guages and frame­works. (Or if you pre­fer lan­guages that don’t start with P”, there’s C#, Go, Java, Ruby, etc.)

My point is that LAMP” as the name of a web devel­op­ment stack has out­lived its use­ful­ness. It’s at once too spe­cif­ic (about oper­at­ing sys­tem and web serv­er details that are often abstract­ed away for devel­op­ers) and too broad (cov­er­ing three sep­a­rate pro­gram­ming lan­guages and not the frame­works they favor). It also leaves out oth­er non-​JavaScript back-​end lan­guages and their asso­ci­at­ed frameworks.

The ques­tion is: what can replace it? I’d pro­pose NoJS” as rem­i­nis­cent of NoSQL,” but that inac­cu­rate­ly excludes JavaScript from its nec­es­sary role in the front-​end. NJSB” doesn’t exact­ly roll off the tongue, either, and still has the same ambi­gu­i­ty prob­lem as LAMP.”

How about pithy sort-​of-​acronyms pat­terned like database-​frontend-​backend? Here are some Perl examples:

  • MRDancer: MySQL, React, and Dancer (I use this at work. Yes, the M could also stand for MongoDB. Naming things is hard.)
  • MRMojo: MongoDB, React, and Mojolicious
  • PACat: PostgreSQL, Angular, and Catalyst
  • etc.

Ultimately it comes down to com­mu­ni­ty and indus­try adop­tion. If you’re involved with back-​end web devel­op­ment, please let me know in the com­ments if you agree or dis­agree that LAMP” is still a use­ful term, and if not, what should replace it.