Favorited GitHub - Perl/perl5 by Perl authors (GitHub)
The Perl programming language

Perl has been my pro­gram­ming weapon of choice since 1994 and a con­sis­tent pay­check since 2007. I start­ed [blog­ging about it][tag/perl] in earnest on New Year’s Day 2021.

This post also serves to test Bridgys new abil­i­ty to sup­port favorites as well as likes. Because I like many com­put­er lan­guages, but Perl is my favorite. ❤️

[tag/​perl]: https://phoenixtrap.com/tag/perl/ “#perl” rel=tag

Replied to

I’m tremen­dous­ly grate­ful for Fastlys sup­port of the Perl MetaCPAN search engine over the past ten years, but the for­mer’s page says they only start­ed serv­ing open source” for eight.
logos: Rust, Python, Ruby, Scratch, LF, ASF, curl

And I’m not sure how MetaCPAN is explic­it­ly part of this ini­tia­tive. I don’t see it mentioned.

IKEA BLÅHAJ shark toys

IKEA’s toy BLÅHAJ shark has become a beloved Internet icon over the past sev­er­al years. I thought it might be cute to write a lit­tle Perl to get info about it and even dis­play a cud­dly pic­ture right in the ter­mi­nal where I’m run­ning the code. Maybe this will give you some ideas for your own quick web clients. Of course, you could accom­plish all of these things using a pipeline of indi­vid­ual command-​line util­i­ties like curl, jq, and GNU core­uti­lsbase64. These exam­ples focus on Perl as the glue, though.

Warning: dodgy API ahead

I haven’t found a publicly-​documented and ‑sup­port­ed offi­cial API for query­ing IKEA prod­uct infor­ma­tion but oth­ers have decon­struct­ed the company’s web site AJAX requests so we can use that instead. The alter­na­tive would be to scrape the IKEA web site direct­ly which, although pos­si­ble, would be more tedious and prone to fail­ure should their design change. An unof­fi­cial API is also unre­li­able but the sim­pler client code is eas­i­er to change should any errors surface.

Enter the Mojolicious

My orig­i­nal goal was to do this in a sin­gle line issued to the perl com­mand, and luck­i­ly the Mojolicious framework’s ojo mod­ule is tailor-​made for such things. By adding a -Mojo switch to the perl com­mand, you get over a dozen quick single-​character func­tions for spin­ning up a quick web appli­ca­tion or, in our case, mak­ing and inter­pret­ing web requests with­out a lot of cer­e­mo­ny. Here’s the start of my one-​line request to the IKEA API for infor­ma­tion on their BLÅHAJ prod­uct, using ojo’s g func­tion to per­form an HTTP GET and dis­play­ing the JSON from the response body to the terminal.

$ perl -Mojo -E 'say g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->body'

This cur­rent­ly returns over 2,400 lines of data, so after read­ing it over I’ll con­vert the response body JSON to a Perl data struc­ture and dump only the main prod­uct infor­ma­tion using ojo’s r func­tion:

$ perl -Mojo -E 'say r g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}'
{
  "availability" => [],
  "breathTaking" => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
  "colors" => [
    {
      "hex" => "0058a3",
      "id" => 10007,
      "name" => "blue"
    },
    {
      "hex" => "ffffff",
      "id" => 10156,
      "name" => "white"
    }
  ],
  "contextualImageUrl" => "https://www.ikea.com/us/en/images/products/blahaj-soft-toy-shark__0877371_pe633608_s5.jpg",
  "currencyCode" => "USD",
  "discount" => "",
  "features" => [],
  "gprDescription" => {
    "numberOfVariants" => 0,
    "variants" => []
  },
  "id" => 90373590,
  "itemMeasureReferenceText" => "39 \x{bc} \"",
  "itemNo" => 90373590,
  "itemNoGlobal" => 30373588,
  "itemType" => "ART",
  "lastChance" => $VAR1->{"breathTaking"},
  "mainImageAlt" => "BL\x{c5}HAJ Soft toy, shark, 39 \x{bc} \"",
  "mainImageUrl" => "https://www.ikea.com/us/en/images/products/blahaj-soft-toy-shark__0710175_pe727378_s5.jpg",
  "name" => "BL\x{c5}HAJ",
  "onlineSellable" => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),
  "pipUrl" => "https://www.ikea.com/us/en/p/blahaj-soft-toy-shark-90373590/",
  "price" => {
    "decimals" => 99,
    "isRegularCurrency" => $VAR1->{"breathTaking"},
    "prefix" => "\$",
    "separator" => ".",
    "suffix" => "",
    "wholeNumber" => 19
  },
  "priceNumeral" => "19.99",
  "quickFacts" => [],
  "tag" => "NONE",
  "typeName" => "Soft toy"
}

If I just want the price I can do:

$ perl -Mojo -E 'say g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}->@{qw(currencyCode priceNumeral)}'
USD19.99

That ->@{qw(currencyCode priceNumeral)} towards the end uses the post­fix ref­er­ence slic­ing syn­tax intro­duced exper­i­men­tal­ly in Perl v5.20 and made offi­cial in v5.24. If you’re using an old­er perl, you’d say:

$ perl -Mojo -E 'say @{g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}}{qw(currencyCode priceNumeral)}'
USD19.99

I pre­fer the for­mer, though, because it’s eas­i­er to read left-to-right.

But I’m not in the United States! Where’s my native currency?

You can either replace the us/en” in the URL above or use the core I18N::LangTags::Detect mod­ule added in Perl v5.8.5 if you’re real­ly deter­mined to be portable across dif­fer­ent users’ locales. This is real­ly stretch­ing the def­i­n­i­tion of one-​liner,” though.

$ LANG=de_DE.UTF-8 perl -Mojo -MI18N::LangTags::Detect -E 'my @lang = (split /-/, I18N::LangTags::Detect::detect)[1,0]; say g("https://sik.search.blue.cdtapps.com/" . join("/", @lang == 2 ? @lang : ("us", "en")) . "/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}->@{qw(currencyCode priceNumeral)}'
EUR27.99

Window dressing

It’s hard to envi­sion cud­dling a num­ber, but luck­i­ly the prod­uct infor­ma­tion returned above links to a JPEG file in the mainImageUrl key. My favorite ter­mi­nal app iTerm2 can dis­play images inline from either a file or Base64 encod­ed data, so adding an extra HTTP request and encod­ing from the core MIME::Base64 mod­ule yields:

$ perl -Mojo -MMIME::Base64 -E 'say "\c[]1337;File=inline=1;width=100%:", encode_base64(g(g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl})->body), "\cG"'

(You could just send the image URL to iTerm2’s bun­dled imgcat util­i­ty, but where’s the fun in that?)

$ imgcat --url `perl -Mojo -E 'print g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl}'`

But I don’t have iTerm2 or a Mac!

I got you. At the expense of a num­ber of oth­er depen­den­cies, here’s a ver­sion that will work on any ter­mi­nal that sup­ports 256-​color mode with ANSI codes using Image::Term256Color from CPAN and a Unicode font with block char­ac­ters. I’ll also use Term::ReadKey to size the image for the width of your win­dow. (Again, this stretch­es the def­i­n­i­tion of one-​liner.”)

$ perl -Mojo -MImage::Term256Color -MTerm::ReadKey -E 'say for Image::Term256Color::convert(g(g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl})->body, {scale_x => (GetTerminalSize)[0], utf8 => 1})'

I hate Mojolicious! Can’t you just use core modules?

Fine. Here’s retriev­ing the prod­uct price using HTTP::Tiny and the pure-​Perl JSON pars­er JSON::PP, which were added to core in ver­sion 5.14.

$ perl -MHTTP::Tiny -MJSON::PP -E 'say @{decode_json(HTTP::Tiny->new->get("https://sik.search.blue.cdtapps.com/us/en/search-result-page?types=PRODUCT&q=BLÅHAJ")->{content})->{searchResultPage}{products}{main}{items}[0]{product}}{qw(currencyCode priceNumeral)}'
USD19.99

Fetching and dis­play­ing a pic­ture of the hug­gable shark using MIME::Base64 or Image::Term256Color as above is left as an exer­cise to the reader.

plate of eggs and hash browns

This month I start­ed a new job at Alert Logic, a cyber­se­cu­ri­ty provider with Perl (among many oth­er things) at its beat­ing heart. I’ve been learn­ing a lot, and part of the process has been under­stand­ing the APIs in the code base. To that end, I’ve been writ­ing small test scripts to tease apart data struc­tures, using Perl array-​processing, list-​processing, and hash- (i.e., asso­cia­tive array)-processing func­tions.

I’ve cov­ered map, grep, and friends a cou­ple times before. Most recent­ly, I described using List::Util’s any func­tion to check if a con­di­tion is true for any item in a list. In the sim­plest case, you can use it to check to see if a giv­en val­ue is in the list at all:

use feature 'say';
use List::Util 'any';
my @colors =
  qw(red orange yellow green blue indigo violet);
say 'matched' if any { /^red$/ } @colors;

However, if you’re going to be doing this a lot with arbi­trary strings, Perl FAQ sec­tion 4 advis­es turn­ing the array into the keys of a hash and then check­ing for mem­ber­ship there. For exam­ple, here’s a sim­ple script to check if the col­ors input (either from the key­board or from files passed as argu­ments) are in the rainbow:

#!/usr/bin/env perl

use v5.22; # introduced <<>> for safe opening of arguments
use warnings;
 
my %in_colors = map {$_ => 1}
  qw(red orange yellow green blue indigo violet);

while (<<>>) {
  chomp;
  say "$_ is in the rainbow" if $in_colors{$_};
}

List::Util has a bunch of func­tions for pro­cess­ing lists of pairs that I’ve found use­ful when paw­ing through hash­es. pairgrep, for exam­ple, acts just like grep but instead assigns $a and $b to each key and val­ue passed in and returns the result­ing pairs that match. I’ve used it as a quick way to search for hash entries match­ing cer­tain val­ue conditions:

use List::Util 'pairgrep';
my %numbers = (zero => 0, one => 1, two => 2, three => 3);
my %odds = pairgrep {$b % 2} %numbers;

Sure, you could do this by invok­ing a mix of plain grep, keys, and a hash slice, but it’s nois­i­er and more repetitive:

use v5.20; # for key/value hash slice 
my %odds = %numbers{grep {$numbers{$_} % 2} keys %numbers};

pairgreps com­piled C‑based XS code can also be faster, as evi­denced by this Benchmark script that works through a hash made of the Unix words file (479,828 entries on my machine):

#!/usr/bin/env perl

use v5.20;
use warnings;
use List::Util 'pairgrep';
use Benchmark 'cmpthese';

my (%words, $count);
open my $fh, '<', '/usr/share/dict/words'
  or die "can't open words: $!";
while (<$fh>) {
  chomp;
  $words{$_} = $count++;
}
close $fh;

cmpthese(100, {
  grep => sub {
    my %odds = %words{grep {$words{$_} % 2} keys %words};
  },
  pairgrep => sub {
    my %odds = pairgrep {$b % 2} %words;
  },
} );

Benchmark out­put:

           Rate     grep pairgrep
grep     1.47/s       --     -20%
pairgrep 1.84/s      25%       --

In gen­er­al, I urge you to work through the Perl doc­u­men­ta­tions tuto­ri­als on ref­er­ences, lists of lists, the data struc­tures cook­book, and the FAQs on array and hash manip­u­la­tion. Then dip into the var­i­ous list-​processing mod­ules (espe­cial­ly the includ­ed List::Util and CPAN’s List::SomeUtils) for ready-​made func­tions for com­mon oper­a­tions. You’ll find a wealth of tech­niques for cre­at­ing, man­ag­ing, and pro­cess­ing the data struc­tures that your pro­grams need.

I men­tioned in pass­ing last week that the next major release of Perl, v5.36, is set to enable warnings by default for code that opts in to use v5.35; or above. Commemorating Perl’s 34th birth­day the week before that, I not­ed that the warn­ings sys­tem has been get­ting ever finer-​grained since its intro­duc­tion in 2000. And fel­low Perl blog­ger and CPAN author Tom Wyant has been cat­a­loging his favorites over the past sev­er­al months — the lat­est as of this writ­ing was on the ambigu­ous” cat­e­go­ry of warn­ings, and you can find links to pre­vi­ous entries in his series at the bot­tom of that post.

It occurred to me after­ward that there may be some con­fu­sion between the warnings prag­ma and the relat­ed warn func­tion for report­ing arbi­trary run­time errors. warn out­puts its argu­ments to the stan­dard error (STDERR) stream, or if it’s not giv­en any then you get a string with any excep­tion from [email protected] ($EVAL_ERROR under use English) fol­lowed by a tab and then “...caught at <file> line x.” If that’s emp­ty too, a plain warn just says, Warning: something's wrong at <file> line x.”, which isn’t exact­ly help­ful, but then again you didn’t give it much to go on.

warn out­put doesn’t have to go to STDERR, and this is where the rela­tion to the warn­ings prag­ma comes in because both are gov­erned by the __WARN__ sig­nal han­dler in the %SIG hash. Normally, you might opt to only dis­play run­time warn­ings if a debug­ging flag is set, like so:

#!/usr/bin/env perl

use strict;
use warnings;

my $DEBUG = 0;
$SIG{__WARN__} = sub { warn @_ if $DEBUG };
warn 'shhh'; # silenced

$DEBUG = 1;
warn 'hello warnings';

But if you set that sig­nal han­dler in a BEGIN block, it catch­es compile-​time warn­ings too, in which case flip­ping a flag after the fact has no effect — the compiler’s already run:

#!/usr/bin/env perl

use strict;
use warnings;

my $DEBUG = 0;
BEGIN { $SIG{__WARN__} = sub { warn @_ if $DEBUG } }
my $foo = 'hello';
my $foo = 'world'; # no warning issued here

$DEBUG = 1;
my $foo = 'howdy'; # still nothing

By the way, both __WARN__ and __DIE__ hooks are also used by the Carp mod­ule and its friends, so you can use the same tech­nique with their enhanced output:

#!/usr/bin/env perl

use strict;
use warnings;
use Carp qw(carp cluck);

my $DEBUG = 0;
BEGIN { $SIG{__WARN__} = sub { warn @_ if $DEBUG } }
carp 'quiet fish';

$DEBUG = 1;
loud_chicken();

sub loud_chicken {
    cluck 'here comes a stack trace';
}

You could use these as step­ping stones towards a debug log for larg­er appli­ca­tions, but at that point, I’d sug­gest look­ing into one of the log­ging mod­ules on CPAN like Log::Log4perl (not to be con­fused with that lately-​problematic Java library), Log::Dispatch (which can be wired into Log4perl), or some­thing else to suit your needs.