Template processors and engines are one of those pieces of software where it seems every developer wants to reinvent the wheel. Goodness knows I’ve done it earlier in my career. Tell me if this sounds familiar:

  1. You need to mix data into a document so you start with Perl’s string interpolation in "double quotes" or sprintf formats. (Or maybe you investigate formats, but the less said about them the better.)
  2. You realize your documents need to display things based on certain conditions, or you want to loop over a list or some other structure.
  3. You add these features via keyword parsing and escape characters, thinking it’s OK since this is just a small bespoke project.
  4. Before you know it you’ve invented another domain-​specific language (DSL) and have to support it on top of the application you were trying to deliver in the first place.

Stop. Just stop. Decades of others who have walked this same path have already done this for you. Especially if you’re using a web framework like Dancer, Mojolicious, or Catalyst, where the template processor is either built-​in or pluggable from CPAN. Even if you’re not developing a web application there are several general-​purpose options of various capabilities like Template Toolkit and Template::Mustache. Investigate the alternatives and determine if they have the features, performance, and support you need. If you’re sure none of them truly meet your unique requirements, then maybe, maybe consider rolling your own.

Whatever you decide, realize that as your application or website grows your investment in that selection will only deepen. Porting to a new template processor can be as challenging as porting any source code to a new programming language.

Unfortunately, there are about as many opinions on how to choose a template processor as there are template processors. For example, in 2013 Roland Koehler wrote a good Python-​oriented article on several considerations and the different approaches available. Although he ended up developing his own (quelle surprise), he makes a good case that a template processor ought to at least provide various logic constructs as well as embedded expressions, if not a full programming language. Koehler specifically warns against the latter, though, as a template developer might change an application’s data model, to say nothing of the possibility of executing arbitrary destructive code.

I can appreciate this reasoning. I’ve successfully used Perl template processors like the aforementioned Template::Toolkit (which has both logic directives and an optional facility for evaluating Perl code) and Text::Xslate (which supports several template syntaxes including a subset of Template::Toolkits, but without the ability to embed Perl code). We use the latter at work combined with Text::Xslate::Bridge::TT2Likes emulation of various Template::Toolkit virtual methods and it’s served us well.

But using those modules’ DSLs means more sophisticated tasks need extra time and effort finding the correct logic and expressions. This also assumes that their designer(s) have anticipated my needs either through built-​in features or extensions. I’m already writing Perl; why should I switch to another, more limited language and environment provided I can remain disciplined enough to avoid issues like those described above by Koehler?

So for my personal projects, I favor template processors that use the full power of the Perl language like Mojolicious’ embedded Perl renderer or the venerable Text::Template for non-​web applications. It saves me time and I’ll likely want more than any DSL can provide. This may not apply to your situation, though, and I’m open to counter-arguments.

What’s your favorite template processor and why? Let me know in the comments.

woman looking at the map

Six months ago I gave an overview of Perl’s list processing fundamentals, briefly describing what lists are and then introducing the built-​in map and grep functions for transforming and filtering them. Later on, I compiled a list (how appropriate) of list processing modules available via CPAN, noting there’s some confusing duplication of effort. But you’re a busy developer, and you just want to know the Right Thing To Do™ when faced with a list processing challenge.

First, some credit is due: these are all restatements of several Perl::Critic policies which in turn codify standards described in Damian Conway’s Perl Best Practices (2005). I’ve repeatedly recommended the latter as a starting point for higher-​quality Perl development. Over the years these practices continue to be re-​evaluated (including by the author himself) and various authors release new policy modules, but perlcritic remains a great tool for ensuring you (and your team or other contributors) maintain a consistent high standard in your code.

With that said, on to the recommendations!

Don’t use grep to check if any list elements match

It might sound weird to lead off by recommending not to use grep, but sometimes it’s not the right tool for the job. If you’ve got a list and want to determine if a condition matches any item in it, you might try:

if (grep { some_condition($_) } @my_list) {
    ... # don't do this!
}

Yes, this works because (in scalar context) grep returns the number of matches found, but it’s wasteful, checking every element of @my_list (which could be lengthy) before finally providing a result. Use the standard List::Util module’s any function, which immediately returns (“short-​circuits”) on the first match:

use List::Util 1.33 qw(any);

if (any { some_condition($_) } @my_list) {
... # do something
}

Perl has included the requisite version of this module since version 5.20 in 2014; for earlier releases, you’ll need to update from CPAN. List::Util has many other great list-​reduction, key/​value pair, and other related functions you can import into your code, so check it out before you attempt to re-​invent any wheels.

As a side note for web developers, the Perl Dancer framework also includes an any keyword for declaring multiple HTTP routes, so if you’re mixing List::Util in there don’t import it. Instead, call it explicitly like this or you’ll get an error about a redefined function:

use List::Util 1.33;

if (List::Util::any { some_condition($_) } @my_list) {
... # do something
}

This recommendation is codified in the BuiltinFunctions::ProhibitBooleanGrep Perl::Critic policy, comes directly from Perl Best Practices, and is recommended by the Software Engineering Institute Computer Emergency Response Team (SEI CERT)’s Perl Coding Standard.

Don’t change $_ in map or grep

I mentioned this back in March, but it bears repeating: map and grep are intended as pure functions, not mutators with side effects. This means that the original list should remain unchanged. Yes, each element aliases in turn to the $_ special variable, but that’s for speed and can have surprising results if changed even if it’s technically allowed. If you need to modify an array in-​place use something like:

for (@my_array) {
$_ = ...; # make your changes here
}

If you want something that looks like map but won’t change the original list (and don’t mind a few CPAN dependencies), consider List::SomeUtilsapply function:

use List::SomeUtils qw(apply);

my @doubled_array = apply {$_ *= 2} @old_array;

Lastly, side effects also include things like manipulating other variables or doing input and output. Don’t use map or grep in a void context (i.e., without a resulting array or list); do something with the results or use a for or foreach loop:

map { print foo($_) } @my_array; # don't do this
print map { foo($_) } @my_array; # do this instead

map { push @new_array, foo($_) } @my_array; # don't do this
@new_array = map { foo($_) } @my_array; # do this instead

This recommendation is codified by the BuiltinFunctions::ProhibitVoidGrep, BuiltinFunctions::ProhibitVoidMap, and ControlStructures::ProhibitMutatingListFunctions Perl::Critic policies. The latter comes from Perl Best Practices and is an SEI CERT Perl Coding Standard rule.

Use blocks with map and grep, not expressions

You can call map or grep like this (parentheses are optional around built-​in functions):

my @new_array  = map foo($_), @old_array; # don't do this
my @new_array2 = grep !/^#/, @old_array; # don't do this

Or like this:

my @new_array  = map { foo($_) } @old_array;
my @new_array2 = grep {!/^#/} @old_array;

Do it the second way. It’s easier to read, especially if you’re passing in a literal list or multiple arrays, and the expression forms can conceal bugs. This recommendation is codified by the BuiltinFunctions::RequireBlockGrep and BuiltinFunctions::RequireBlockMap Perl::Critic policies and comes from Perl Best Practices.

Refactor multi-​statement maps, greps, and other list functions

map, grep, and friends should follow the Unix philosophy of Do One Thing and Do It Well.” Your readability and maintainability drop with every statement you place inside one of their blocks. Consider junior developers and future maintainers (this includes you!) and refactor anything with more than one statement into a separate subroutine or at least a for loop. This goes for list processing functions (like the aforementioned any) imported from other modules, too.

This recommendation is codified by the Perl Best Practices-inspired BuiltinFunctions::ProhibitComplexMappings and BuiltinFunctions::RequireSimpleSortBlock Perl::Critic policies, although those only cover map and sort functions, respectively.


Do you have any other suggestions for list processing best practices? Feel free to leave them in the comments or better yet, consider creating new Perl::Critic policies for them or contacting the Perl::Critic team to develop them for your organization.

clear light bulb planter on gray rock

Twitter recently recommended a tweet to me (all hail the algorithm) touting what the author viewed as the top 5 web development stacks.”

JavaScript/​Node.js options dominated the four-​letter acronyms as expected, but the fifth one surprised me: LAMP, the combination of the Linux operating system, Apache web server, MySQL relational database, and Perl, PHP, or Python programming languages. A quick web search for similar lists yielded similar results. Clearly, this meme (in the Dawkins sense) has outlasted its popularization by tech publisher O’Reilly in the 2000s.

Originally coined in 1998 during the dot-​com” bubble, I had thought that the term LAMP” had faded with developers in the intervening decades with the rise of language-​specific web frameworks for:

Certainly on the Perl side (with which I’m most familiar), the community has long since recommended the use of a framework built on the PSGI specification, deprecating 1990s-​era CGI scripts and the mod_​perl Apache extension. Although general-​purpose web servers like Apache or Nginx may be part of an overall system, they’re typically used as proxies or load balancers for Perl-​specific servers either provided by the framework or a third-​party module.

Granted, PHP still relies on web server-​specific modules, APIs, or variations of the FastCGI protocol for interfacing with a web server. And Python web applications typically make use of its WSGI protocol either as a web server extension or, like the Perl examples above, as a proxied standalone server. But all of these are deployment details and do little to describe how developers implement and extend a web application’s structure.

Note how the various four-​letter JavaScript stacks (e.g., MERN, MEVN, MEAN, PERN) differentiate themselves mostly by frontend framework (e.g., Angular, React, Vue.js) and maybe by the (relational or NoSQL) database (e.g., MongoDB, MySQL, PostgreSQL). All however seem standardized on the Node.js runtime and Express backend web framework, which could, in theory, be replaced with non-​JavaScript options like the more mature LAMP-​associated languages and frameworks. (Or if you prefer languages that don’t start with P”, there’s C#, Go, Java, Ruby, etc.)

My point is that LAMP” as the name of a web development stack has outlived its usefulness. It’s at once too specific (about operating system and web server details that are often abstracted away for developers) and too broad (covering three separate programming languages and not the frameworks they favor). It also leaves out other non-​JavaScript back-​end languages and their associated frameworks.

The question is: what can replace it? I’d propose NoJS” as reminiscent of NoSQL,” but that inaccurately excludes JavaScript from its necessary role in the front-​end. NJSB” doesn’t exactly roll off the tongue, either, and still has the same ambiguity problem as LAMP.”

How about pithy sort-​of-​acronyms patterned like database-​frontend-​backend? Here are some Perl examples:

  • MRDancer: MySQL, React, and Dancer (I use this at work. Yes, the M could also stand for MongoDB. Naming things is hard.)
  • MRMojo: MongoDB, React, and Mojolicious
  • PACat: PostgreSQL, Angular, and Catalyst
  • etc.

Ultimately it comes down to community and industry adoption. If you’re involved with back-​end web development, please let me know in the comments if you agree or disagree that LAMP” is still a useful term, and if not, what should replace it.