Tag: work

Posted on
by
man cutting tress using chainsaw

The Java world had an… inter­est­ing week­end when secu­ri­ty researchers revealed on December 9 a vul­ner­a­bil­i­ty in the pop­u­lar Apache Log4j 2 soft­ware library for record­ing and debug­ging events. Systems as diverse as Amazon Web Services, Apple iCloud, and the Minecraft video game could be exploit­ed to run arbi­trary code on a serv­er mere­ly by send­ing a specially-​crafted string of text. Information tech­nol­o­gy pro­fes­sion­als have been scram­bling ever since the ini­tial dis­clo­sure to patch, upgrade, recon­fig­ure, or oth­er­wise pro­tect affect­ed servers. It’s bad, and past unpatched vul­ner­a­bil­i­ties like this have been respon­si­ble for the expo­sure of mil­lions of people’s sen­si­tive data.

Many Perl appli­ca­tions use the similarly-​named and ‑designed Log::Log4perl library, and the good news is that as far as I can tell the lat­ter doesn’t suf­fer from the type of vul­ner­a­bil­i­ty described above. This doesn’t mean poorly-​written or ‑con­fig­ured Perl-​based sys­tems are immune to all exploits, just this par­tic­u­lar one. You should be safe to con­tin­ue using Log4perl unless some­one has delib­er­ate­ly con­fig­ured it oth­er­wise, and in fact, my work uses it extensively.

You might be sur­prised to read me sug­gest­ing a log­ging frame­work after writ­ing mul­ti­ple arti­cles espous­ing the Perl step debug­ger as an alter­na­tive. Log4perl devel­op­er Mike Schilli’s 2002 intro­duc­tion to the pack­age for Perl.com came down on the oppo­site side of the argu­ment. It can seem like one of those pro­gram­mer reli­gious issues like tabs vs. spaces, vim vs. Emacs, or Linux vs. Windows. (For the record, the cor­rect answers are spaces, BBEdit, and macOS. 😉)

But in this case, you can and should have the best of both worlds — log­ging at dif­fer­ent lev­els to appro­pri­ate des­ti­na­tions while still drop­ping into the inter­ac­tive debug­ger when you need to do some­thing trick­i­er like exam­ine pro­gram state or tweak a data struc­ture on the fly. I use both tech­niques and only empha­size the advo­ca­cy of step debug­ging because it’s under­stood less.

Posted on
by
woman using a laptop with her daughter

Inspired by my par­ents com­ing to vis­it at the end of the week, I thought I’d write about how Perl class­es can have ​“par­ents” as well, from which they inher­it meth­ods. Although it might seem on the sur­face as though there’s more than one way to do it, these tech­niques all share the same under­ly­ing mechanism.

Where it all BEGINs: @ISA

Perl class­es are just repur­posed packages, i.e., a name­space for vari­ables and sub­rou­tines. The two key dif­fer­ences are:

If you want­ed to do every­thing by hand at the low­est lev­el, you could make a sub­class at com­pile time like this:

package Local::MyChildClass;
BEGIN { # don't do this:
    require Local::MyParentClass;
    push @ISA, 'Local::MyParentClass';
}

Don’t do that though, because we have…

base and parent

In 1997 Perl 5.004_04 intro­duced the base prag­ma (back when Perl used that kind of ver­sion­ing scheme; in these days of seman­tic ver­sion­ing we’d call it ver­sion 5.4.4). It does the above BEGIN block in a sin­gle line:

use base 'Local::MyParentClass'; # don't do this unless you're also using fields

You might see use base in old­er code espe­cial­ly if it’s also using the fields prag­ma. However, Perl devel­op­ers dis­cour­age both as the for­mer silences cer­tain mod­ule load­ing errors while the lat­ter is at odds with the object-​oriented pro­gram­ming prin­ci­ple of encap­su­la­tion.

So use parent instead, which Perl has includ­ed since ver­sion 5.10.1 in 2009:

use parent 'Local::MyParentClass';

A cou­ple of years ago my Newfold Digital col­league David Oswald cre­at­ed a fork of par­ent called parent::versioned that sup­ports spec­i­fy­ing the low­est ver­sion for super­class­es. You call it like this:

use parent::versioned ['Local::MyParentClass' => 1.23];

Within an OO system

There are dozens of object-​oriented pro­gram­ming sys­tems on CPAN that pro­vide syn­tac­tic sug­ar and extra fea­tures to Perl’s min­i­mal but flex­i­ble basics. Two of the more pop­u­lar ones, Moose and Moo, offer an extends key­word that you should use instead of use parent so that your sub­class­es may take advan­tage of their features:

package Local::MyChildClass;
use Moo;
extends 'Local::MyParentClass';

Moose can also spec­i­fy a required super­class version:

package Local::MyChildClass;
use Moose;
extends 'Local::MyParentClass' => {-version => 1.23};

Also, use the MooseX::NonMoose mod­ule when extend­ing non-​Moose class­es, again so you get Moose fea­tures even though your meth­ods are com­ing from some­where else:

package Local::MyMooseClass;
use Moose;
use MooseX::NonMoose;
extends 'Local::MyPlainParentClass';

The exper­i­men­tal Object::Pad mod­ule spec­i­fies a sin­gle super­class while defin­ing the class name with an option­al ver­sion. Per the author’s sug­gest­ed file lay­out, includ­ing a required min­i­mum ver­sion, it would look like:

use Object::Pad 0.41;
package Local::MyChildClass;
class Local::MyChildClass isa Local::MyParentClass 1.23;

Object::Pad and Corinna, its inspi­ra­tion, are works in progress so this syn­tax isn’t set in stone. The latter’s design­er Curtis ​“Ovid” Poe blogged ear­li­er this week about con­sid­er­ing a more self-​consistent syntax.

Multiple inheritance vs. roles

To quote the Perl doc­u­men­ta­tion, ​“mul­ti­ple inher­i­tance often indi­cates a design prob­lem, but Perl always gives you enough rope to hang your­self with if you ask for it.” All the tech­niques described above except for Object::Pad sup­port mul­ti­ple inher­i­tance by spec­i­fy­ing a list of super­class­es. For example:

package Local::MyChildClass;
use parent qw(Local::MyParentClass1 Local::MyParentClass2);

If you’re using roles instead of or on top of super­class­es (I’ve seen both sit­u­a­tions) and your OO sys­tem doesn’t sup­port them on its own, you can use the Role::Tiny mod­ule, first by describ­ing your role in one pack­age and then con­sum­ing it in another:

package Local::DoesSomething;
use Role::Tiny;

...

1;
package Local::MyConsumer;
use Role::Tiny::With;
with 'Local::DoesSomething';

...

1;

Moo::Role uses Role::Tiny under the hood and Moo can com­pose roles from either. The syn­tax for both Moo and Moose is similar:

package Local::DoesSomething;
use Moo::Role; # or "use Moose::Role;"

...

1;
package Local::MyConsumer;
use Moo; # or "use Moose;"
with 'Local::DoesSomething';

...

1;

Object::Pad spec­i­fies roles with the role key­word, and both class­es and roles use does to con­sume them:

use Object::Pad 0.56;
package Local::DoesSomething;
role Local::DoesSomething does Local::DoesSomethingElse;

...

1;
use Object::Pad 0.56;
package Local::MyConsumer;
class Local::MyConsumer does Local::DoesSomething;

...

1;

The pre­vi­ous caveat about pos­si­ble changes to this syn­tax applies.

Like parent, (sort of) like child

Of course, the whole point of inher­i­tance or role con­sump­tion is so your child or con­sumer class can reuse func­tions and meth­ods. Each of the tech­niques above has its ways of over­rid­ing that code, from the Perl built-​in SUPER pseudo-​class to Moose’s override and super key­words, to Moose’s and Moo’s method mod­i­fiers. (You can use the lat­ter out­side of Moo since it’s pro­vid­ed by Class::Method::Modifiers.)

I’ve writ­ten about choos­ing between over­rid­ing and mod­i­fy­ing meth­ods before, and when it comes to Moose and Moo code I’m now on the side of using the around method mod­i­fi­er if a method needs to call an inher­it­ed or con­sumed method of the same name. Object::Pad doesn’t have method mod­i­fiers (yet), so classes that use it will have to sat­is­fy them­selves with SUPER in their methods with an :override attribute that will throw an error if a par­ent doesn’t also pro­vide the same method.

The Parent Wrap

In the end, your choice of Perl OO sys­tem will deter­mine how (or whether) you han­dle inher­i­tance and may even be a decid­ing fac­tor. Which would you choose? And more impor­tant­ly, have I made my par­ents proud with this post?

Posted on
by
animal antler big close up

At my work, we exten­sive­ly use the Moose object sys­tem to take care of what would ordi­nar­i­ly be very tedious boil­er­plate object-​oriented Perl code. In one part of the code­base, we have a fam­i­ly of class­es that, among oth­er things, map Perl meth­ods to the names of var­i­ous calls in a third-​party API with­in our larg­er orga­ni­za­tion. Those pri­vate Perl meth­ods are in turn called from pub­lic meth­ods pro­vid­ed by roles con­sumed by these class­es so that oth­er areas aren’t con­cerned with said API’s details.

Without going into too many specifics, I had a bunch of class­es all with sec­tions that looked like this:

sub _create_method    { return 'api_add'     }
sub _retrieve_method  { return 'api_info'    }
sub _search_method    { return 'api_list'    }
sub _update_method    { return 'api_update'  }
sub _cancel_method    { return 'api_remove'  }
sub _suspend_method   { return 'api_disable' }
sub _unsuspend_method { return 'api_restore' }

... # etc.

The val­ues returned by these very sim­ple meth­ods might dif­fer from class to class depend­ing on the API call need­ed, and dif­fer­ent class­es might have a dif­fer­ent mix of these meth­ods depend­ing on what roles they consume.

These meth­ods had built up over time as devel­op­ers had expand­ed the class­es’ func­tion­al­i­ty, and this week it was my turn. I decid­ed to apply the DRY (don’t repeat your­self) prin­ci­ple and cre­ate them from a sim­ple hash table like so:

my %METHOD_MAP = (
  _create_method    => 'api_add',
  _retrieve_method  => 'api_info',
  _search_method    => 'api_list',
  _update_method    => 'api_update',
  _cancel_method    => 'api_remove',
  _suspend_method   => 'api_disable',
  _unsuspend_method => 'api_restore',
);

At first, I thought to myself, ​“These look like pri­vate read-​only attrib­ut­es!” So I wrote:

use Moose;

...

has $_ => (
  is       => 'ro',
  init_arg => undef,
  default  => $METHOD_MAP{$_},
) for keys %METHOD_MAP;

Of course, I’d have to move the class­es’ with state­ments after these def­i­n­i­tions so the roles they con­sume could ​“see” these runtime-​defined attrib­ut­es. But some of the meth­ods used to read these are class meth­ods (e.g., called as ClassName->foo() rather than $object->foo()), and Moose attrib­ut­es are only set after the con­struc­tion of a class instance.

Then I thought, ​“Hey, Moose has a MOP (meta-​object pro­to­col)! I’ll use that to gen­er­ate these meth­ods at runtime!”

my $meta = __PACKAGE__->meta;

while (my ($method, $api_call) = each %METHOD_MAP) {
    $meta->add_method( $method => sub {$api_call} );
}

The add_method doc­u­men­ta­tion ​“strong­ly encourage[s]” you to pass a metamethod object rather than a code ref­er­ence, though, so that would look like:

use Moose::Meta::Method;

my $meta = __PACKAGE__->meta;

while (my ($method, $api_call) = each %METHOD_MAP) {
    $meta->add_method( $method = Moose::Meta::Method->wrap(
      sub {$api_call}, __PACKAGE__, $meta,
    );
}

This was get­ting ugly. There had to be a bet­ter way, and for­tu­nate­ly there was in the form of Dave Rolsky​’s MooseX::ClassAttribute mod­ule. It sim­pli­fies the above to:

use MooseX::ClassAttribute;

class_has $_ => (
  is      => 'ro',
  default => $METHOD_MAP{$_},
) for keys %METHOD_MAP;

Note there’s no need for init_arg => undef to pre­vent set­ting the attribute in the con­struc­tor. Although they’re still Moose attrib­ut­es, they act like class meth­ods so long as the class con­sumes the roles that require them after the attribute definitions.

Lastly, if we were using Moo as a light­weight alter­na­tive to Moose, I could have instead select­ed Toby Inkster​’s MooX::ClassAttribute. Although it has some caveats, it’s pret­ty much the only alter­na­tive to our ini­tial class method def­i­n­i­tions as Moo lacks a meta-​object pro­to­col.

The les­son as always is to check CPAN (or the appro­pri­ate mix of your language’s soft­ware repos­i­to­ry, forums like Stack Overflow, etc.) for any­thing that could con­ceiv­ably have appli­ca­tion out­side of your par­tic­u­lar cir­cum­stances. Twenty-​five years into my career and I’m still leap­ing into code with­out first con­sid­er­ing that some­one smarter than me has already done the work.

Posted on
by

Template proces­sors and engines are one of those pieces of soft­ware where it seems every devel­op­er wants to rein­vent the wheel. Goodness knows I’ve done it ear­li­er in my career. Tell me if this sounds familiar:

  1. You need to mix data into a doc­u­ment so you start with Perl’s string inter­po­la­tion in "dou­ble quotes" or sprintf for­mats. (Or maybe you inves­ti­gate formats, but the less said about them the bet­ter.)
  2. You real­ize your doc­u­ments need to dis­play things based on cer­tain con­di­tions, or you want to loop over a list or some oth­er structure.
  3. You add these fea­tures via key­word pars­ing and escape char­ac­ters, think­ing it’s OK since this is just a small bespoke project.
  4. Before you know it you’ve invent­ed anoth­er domain-​specific lan­guage (DSL) and have to sup­port it on top of the appli­ca­tion you were try­ing to deliv­er in the first place.

Stop. Just stop. Decades of oth­ers who have walked this same path have already done this for you. Especially if you’re using a web frame­work like Dancer, Mojolicious, or Catalyst, where the tem­plate proces­sor is either built-​in or plug­gable from CPAN. Even if you’re not devel­op­ing a web appli­ca­tion there are sev­er­al general-​purpose options of var­i­ous capa­bil­i­ties like Template Toolkit and Template::Mustache. Investigate the alter­na­tives and deter­mine if they have the fea­tures, per­for­mance, and sup­port you need. If you’re sure none of them tru­ly meet your unique require­ments, then maybe, maybe con­sid­er rolling your own.

Whatever you decide, real­ize that as your appli­ca­tion or web­site grows your invest­ment in that selec­tion will only deep­en. Porting to a new tem­plate proces­sor can be as chal­leng­ing as port­ing any source code to a new pro­gram­ming language.

Unfortunately, there are about as many opin­ions on how to choose a tem­plate proces­sor as there are tem­plate proces­sors. For exam­ple, in 2013 Roland Koehler wrote a good Python-​oriented arti­cle on sev­er­al con­sid­er­a­tions and the dif­fer­ent approach­es avail­able. Although he end­ed up devel­op­ing his own (quelle sur­prise), he makes a good case that a tem­plate proces­sor ought to at least pro­vide var­i­ous log­ic con­structs as well as embed­ded expres­sions, if not a full pro­gram­ming lan­guage. Koehler specif­i­cal­ly warns against the lat­ter, though, as a tem­plate devel­op­er might change an application’s data mod­el, to say noth­ing of the pos­si­bil­i­ty of exe­cut­ing arbi­trary destruc­tive code.

I can appre­ci­ate this rea­son­ing. I’ve suc­cess­ful­ly used Perl tem­plate proces­sors like the afore­men­tioned Template::Toolkit (which has both log­ic direc­tives and an option­al facil­i­ty for eval­u­at­ing Perl code) and Text::Xslate (which sup­ports sev­er­al tem­plate syn­tax­es includ­ing a sub­set of Template::Toolkit​’s, but with­out the abil­i­ty to embed Perl code). We use the lat­ter at work com­bined with Text::Xslate::Bridge::TT2Like​’s emu­la­tion of var­i­ous Template::Toolkit vir­tu­al meth­ods and it’s served us well.

But using those mod­ules’ DSLs means more sophis­ti­cat­ed tasks need extra time and effort find­ing the cor­rect log­ic and expres­sions. This also assumes that their designer(s) have antic­i­pat­ed my needs either through built-​in fea­tures or exten­sions. I’m already writ­ing Perl; why should I switch to anoth­er, more lim­it­ed lan­guage and envi­ron­ment pro­vid­ed I can remain dis­ci­plined enough to avoid issues like those described above by Koehler?

So for my per­son­al projects, I favor tem­plate proces­sors that use the full pow­er of the Perl lan­guage like Mojolicious’ embed­ded Perl ren­der­er or the ven­er­a­ble Text::Template for non-​web appli­ca­tions. It saves me time and I’ll like­ly want more than any DSL can pro­vide. This may not apply to your sit­u­a­tion, though, and I’m open to counter-arguments.

What’s your favorite tem­plate proces­sor and why? Let me know in the comments.

Posted on
by

Look, I get it. You don’t like the Perl pro­gram­ming lan­guage or have oth­er­wise dis­re­gard­ed it as ​“dead.” (Or per­haps you haven’t, in which case please check out my oth­er blog posts!) It has weird noisy syn­tax, mix­ing reg­u­lar expres­sions, sig­ils on vari­able names, var­i­ous braces and brack­ets for data struc­tures, and a menagerie of cryp­tic spe­cial vari­ables. It’s old: 34 years in December, with a his­to­ry of (some­times ama­teur) devel­op­ers that have used and abused that syn­tax to ship code of ques­tion­able qual­i­ty. Maybe you grudg­ing­ly accept its util­i­ty but think it should die grace­ful­ly, main­tained only to run lega­cy applications.

But you know what? Perl’s still going. It’s had a steady cadence of year­ly releas­es for the past decade, intro­duc­ing new fea­tures and fenc­ing in bad behav­ior while main­tain­ing an admirable lev­el of back­ward com­pat­i­bil­i­ty. Yes, there was a too-​long adven­ture devel­op­ing what start­ed as Perl 6, but that lan­guage now has its own iden­ti­ty as Raku and even has facil­i­ties for mix­ing Perl with its native code or vice versa.

And then there’s CPAN, the Comprehensive Perl Archive Network: a continually-​updated col­lec­tion of over 200,000 open-​source mod­ules writ­ten by over 14,000 authors, the best of which are well-​tested and ‑doc­u­ment­ed (apply­ing peer pres­sure to those that fall short), pre­sent­ed through a search engine and front-​end built by scores of con­trib­u­tors. Through CPAN you can find dis­tri­b­u­tions for things like:

All of this is avail­able through a mature instal­la­tion tool­chain that doesn’t break from month to month.

Finally and most impor­tant­ly, there’s the glob­al Perl com­mu­ni­ty. The COVID-​19 pan­dem­ic has put a damper on the hun­dreds of glob­al Perl Mongers groups’ mee­tups, but that hasn’t stopped the year­ly Perl and Raku Conference from meet­ing vir­tu­al­ly. (In the past there have also been year­ly European and Asian con­fer­ences, occa­sion­al for­ays into South America and Russia, as well as hackathons and work­shops world­wide.) There are IRC servers and chan­nels for chat, mail­ing lists galore, blogs (yes, apart from this one), and a quirky social net­work that pre­dates Facebook and Twitter.

So no, Perl isn’t dead or even dying, but if you don’t like it and favor some­thing new­er, that’s OK! Technologies can coex­ist on their own mer­its and advo­cates of one don’t have to beat down their con­tem­po­raries to be suc­cess­ful. Perl hap­pens to be battle-​tested (to bor­row a term from my friend Curtis ​“Ovid” Poe), it runs large parts of the Web (speak­ing from direct and ongo­ing expe­ri­ence in the host­ing busi­ness here), and it’s still evolv­ing to meet the needs of its users.

Posted on
by
iceberg in body of water

We have a huge code­base of over 700,000 lines of Perl spread across a cou­ple dozen Git repos­i­to­ries at work. Sometimes refac­tor­ing is easy if the class­es and meth­ods involved are con­fined to one of those repos, but last week we want­ed to rename a method that was poten­tial­ly used across many of them with­out hav­ing to QA and launch so many changes. After get­ting some help from Dan Book and Ryan Voots on the #perl libera.chat IRC chan­nel, I arrived at the fol­low­ing solution.

First, if all you want to do is alias the new method call to the old while mak­ing the least amount of changes, you can just do this:

*new_method = \&old_method;

This takes advan­tage of Perl’s type­globs by assign­ing to the new method­’s name in the sym­bol table a ref­er­ence (indi­cat­ed by the \ char­ac­ter) to the old method. Methods are just sub­rou­tines in Perl, and although you don’t need the & char­ac­ter when call­ing one, you do need it if you’re pass­ing a sub­rou­tine as an argu­ment or cre­at­ing a ref­er­ence, as we’re doing above.

I want­ed to do a bit more, though. First, I want­ed to log the calls to the old method name so that I could track just how wide­ly it’s used and have a head start on renam­ing it else­where in our code­base. Also, I did­n’t want to fill our logs with those calls — we have enough noise in there already. And last­ly, I want­ed future calls to go direct­ly to the new method name with­out adding anoth­er stack frame when using caller or Carp.

With all that in mind, here’s the result:

sub old_method {
    warn 'old_method is deprecated';
    no warnings 'redefine';
    *old_method = \&new_method;
    goto &new_method;
}

sub new_method {
    # code from old_method goes here
}

Old (and not-​so-​old) hands at pro­gram­ming are prob­a­bly leap­ing out of their seats right now yelling, ​“YOU’RE USING GOTO! GOTO IS CONSIDERED HARMFUL!” And they’re right, but this isn’t Dijkstra’s goto. From the Perl manual:

The goto &NAME form is quite dif­fer­ent from the oth­er forms of goto. In fact, it isn’t a goto in the nor­mal sense at all, and does­n’t have the stig­ma asso­ci­at­ed with oth­er gotos. Instead, it exits the cur­rent sub­rou­tine (los­ing any changes set by local) and imme­di­ate­ly calls in its place the named sub­rou­tine using the cur­rent val­ue of @_. […] After the goto, not even caller will be able to tell that this rou­tine was called first.

perl­func man­u­al page

Computer sci­en­tists call this tail call elim­i­na­tion. The bot­tom line is that this achieves our third goal above: imme­di­ate­ly jump­ing to the new method as if it were orig­i­nal­ly called.

The oth­er tricky bit is in the line before, when we’re redefin­ing old_method to point to new_method while we’re still inside old_method. (Yes, you can do this.) If you’re run­ning under use warnings (and we are, and you should), you first need to dis­able that warn­ing. Later calls to old_method will go straight to new_method with­out log­ging anything.

And that’s it. The next step after launch­ing this change is to add a sto­ry to our back­log to mon­i­tor our logs for calls to the old method, and grad­u­al­ly refac­tor our oth­er repos­i­to­ries. Then we can final­ly remove the old method wrapper.

Posted on
by

This week my main task for this sprint was can­celed. While not as momen­tous as the can­cel­la­tion of an entire project (I’ve been there too), delet­ing the past week’s work still stung. This isn’t the first time, though, so I know that there are a few things to keep in mind:

You didn’t waste your time

Bottom line: Were you paid for your work? Then your employ­er still sees it as valu­able, if only to make sure that a giv­en line of devel­op­ment was suf­fi­cient­ly explored before deter­min­ing it was­n’t worth con­tin­u­ing. Developing a prod­uct or ser­vice often means say­ing ​“no” to things, and some­times that means cut­ting loss­es before the sunk cost fal­la­cy takes hold.

You probably learned something

Over the past week’s work, I learned about man­ag­ing TLS con­nec­tions (includ­ing sup­port­ing ciphers that are no longer con­sid­ered secure), para­me­ter val­i­da­tion, and XML name­space sup­port in XPath. You prob­a­bly learned a lot more if your project extend­ed longer, and you can use that knowl­edge fur­ther on in your career. Put it on your résumé or CV, and you may get an oppor­tu­ni­ty to work on the same things in the future.

You could continue if you want

Okay, maybe you’re not going to sneak into the office for months to fin­ish things. But as long as you have the time and incli­na­tion, you could con­tin­ue to work on your project, espe­cial­ly if you think it could be valu­able to the com­pa­ny lat­er on. Consider this care­ful­ly, though — you don’t want off-​the-​books work tak­ing time and ener­gy away from your main job.

There’s no shame

Lastly, you should­n’t feel ashamed about being part of a can­celed project. They hap­pen all the time, and prob­a­bly should hap­pen more — his­to­ry is lit­tered with failed soft­ware projects that like­ly could have cost less if only their prob­lems were rec­og­nized ear­li­er. By its nature, soft­ware devel­op­ment is explorato­ry and dif­fi­cult, and not every idea pans out. As long as you can find some­thing new to work on, you’ll be fine.