The Perl programming language
Perl has been my programming weapon of choice since 1994 and a consistent paycheck since 2007. I started [blogging about it][tag/perl] in earnest on New Year’s Day 2021.
This post also serves to test Bridgy’s new ability to support favorites as well as likes. Because I like many computer languages, but Perl is my favorite. ❤️
[tag/perl]: https://phoenixtrap.com/tag/perl/ “#perl” rel=tag
I’m tremendously grateful for Fastly’s support of the Perl MetaCPAN search engine over the past ten years, but the former’s #FastForward page says they only started “serving open source” for eight.
And I’m not sure how MetaCPAN is explicitly part of this initiative. I don’t see it mentioned.
IKEA’s toy BLÅHAJ shark has become a beloved Internet icon over the past several years. I thought it might be cute to write a little Perl to get info about it and even display a cuddly picture right in the terminal where I’m running the code. Maybe this will give you some ideas for your own quick web clients. Of course, you could accomplish all of these things using a pipeline of individual command-line utilities like curl, jq, and GNU coreutils’ base64. These examples focus on Perl as the glue, though.
I haven’t found a publicly-documented and ‑supported official API for querying IKEA product information but others have deconstructed the company’s web site AJAX requests so we can use that instead. The alternative would be to scrape the IKEA web site directly which, although possible, would be more tedious and prone to failure should their design change. An unofficial API is also unreliable but the simpler client code is easier to change should any errors surface.
My original goal was to do this in a single line issued to the perl command, and luckily the Mojolicious framework’s ojo module is tailor-made for such things. By adding a -Mojo switch to the perl command, you get over a dozen quick single-character functions for spinning up a quick web application or, in our case, making and interpreting web requests without a lot of ceremony. Here’s the start of my one-line request to the IKEA API for information on their BLÅHAJ product, using ojo’s g function to perform an HTTP GET and displaying the JSON from the response body to the terminal.
$ perl -Mojo -E 'say g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->body'This currently returns over 2,400 lines of data, so after reading it over I’ll convert the response body JSON to a Perl data structure and dump only the main product information using ojo’s r function:
$ perl -Mojo -E 'say r g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}'
{
"availability" => [],
"breathTaking" => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
"colors" => [
{
"hex" => "0058a3",
"id" => 10007,
"name" => "blue"
},
{
"hex" => "ffffff",
"id" => 10156,
"name" => "white"
}
],
"contextualImageUrl" => "https://www.ikea.com/us/en/images/products/blahaj-soft-toy-shark__0877371_pe633608_s5.jpg",
"currencyCode" => "USD",
"discount" => "",
"features" => [],
"gprDescription" => {
"numberOfVariants" => 0,
"variants" => []
},
"id" => 90373590,
"itemMeasureReferenceText" => "39 \x{bc} \"",
"itemNo" => 90373590,
"itemNoGlobal" => 30373588,
"itemType" => "ART",
"lastChance" => $VAR1->{"breathTaking"},
"mainImageAlt" => "BL\x{c5}HAJ Soft toy, shark, 39 \x{bc} \"",
"mainImageUrl" => "https://www.ikea.com/us/en/images/products/blahaj-soft-toy-shark__0710175_pe727378_s5.jpg",
"name" => "BL\x{c5}HAJ",
"onlineSellable" => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),
"pipUrl" => "https://www.ikea.com/us/en/p/blahaj-soft-toy-shark-90373590/",
"price" => {
"decimals" => 99,
"isRegularCurrency" => $VAR1->{"breathTaking"},
"prefix" => "\$",
"separator" => ".",
"suffix" => "",
"wholeNumber" => 19
},
"priceNumeral" => "19.99",
"quickFacts" => [],
"tag" => "NONE",
"typeName" => "Soft toy"
}If I just want the price I can do:
$ perl -Mojo -E 'say g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}->@{qw(currencyCode priceNumeral)}'
USD19.99That ->@{qw(currencyCode priceNumeral)} towards the end uses the postfix reference slicing syntax introduced experimentally in Perl v5.20 and made official in v5.24. If you’re using an older perl, you’d say:
$ perl -Mojo -E 'say @{g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}}{qw(currencyCode priceNumeral)}'
USD19.99I prefer the former, though, because it’s easier to read left-to-right.
You can either replace the ”us/en” in the URL above or use the core I18N::LangTags::Detect module added in Perl v5.8.5 if you’re really determined to be portable across different users’ locales. This is really stretching the definition of ”one-liner,” though.
$ LANG=de_DE.UTF-8 perl -Mojo -MI18N::LangTags::Detect -E 'my @lang = (split /-/, I18N::LangTags::Detect::detect)[1,0]; say g("https://sik.search.blue.cdtapps.com/" . join("/", @lang == 2 ? @lang : ("us", "en")) . "/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}->@{qw(currencyCode priceNumeral)}'
EUR27.99It’s hard to envision cuddling a number, but luckily the product information returned above links to a JPEG file in the mainImageUrl key. My favorite terminal app iTerm2 can display images inline from either a file or Base64 encoded data, so adding an extra HTTP request and encoding from the core MIME::Base64 module yields:
$ perl -Mojo -MMIME::Base64 -E 'say "\c[]1337;File=inline=1;width=100%:", encode_base64(g(g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl})->body), "\cG"'
(You could just send the image URL to iTerm2’s bundled imgcat utility, but where’s the fun in that?)
$ imgcat --url `perl -Mojo -E 'print g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl}'`I got you. At the expense of a number of other dependencies, here’s a version that will work on any terminal that supports 256-color mode with ANSI codes using Image::Term256Color from CPAN and a Unicode font with block characters. I’ll also use Term::ReadKey to size the image for the width of your window. (Again, this stretches the definition of “one-liner.”)
$ perl -Mojo -MImage::Term256Color -MTerm::ReadKey -E 'say for Image::Term256Color::convert(g(g("https://sik.search.blue.cdtapps.com/us/en/search-result-page", form => {types => "PRODUCT", q => "BLÅHAJ"})->json->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl})->body, {scale_x => (GetTerminalSize)[0], utf8 => 1})'
Fine. Here’s retrieving the product price using HTTP::Tiny and the pure-Perl JSON parser JSON::PP, which were added to core in version 5.14.
$ perl -MHTTP::Tiny -MJSON::PP -E 'say @{decode_json(HTTP::Tiny->new->get("https://sik.search.blue.cdtapps.com/us/en/search-result-page?types=PRODUCT&q=BLÅHAJ")->{content})->{searchResultPage}{products}{main}{items}[0]{product}}{qw(currencyCode priceNumeral)}'
USD19.99Fetching and displaying a picture of the huggable shark using MIME::Base64 or Image::Term256Color as above is left as an exercise to the reader.
This month I started a new job at Alert Logic, a cybersecurity provider with Perl (among many other things) at its beating heart. I’ve been learning a lot, and part of the process has been understanding the APIs in the code base. To that end, I’ve been writing small test scripts to tease apart data structures, using Perl array-processing, list-processing, and hash- (i.e., associative array)-processing functions.
I’ve covered map, grep, and friends a couple times before. Most recently, I described using List::Util’s any function to check if a condition is true for any item in a list. In the simplest case, you can use it to check to see if a given value is in the list at all:
use feature 'say';
use List::Util 'any';
my @colors =
qw(red orange yellow green blue indigo violet);
say 'matched' if any { /^red$/ } @colors;However, if you’re going to be doing this a lot with arbitrary strings, Perl FAQ section 4 advises turning the array into the keys of a hash and then checking for membership there. For example, here’s a simple script to check if the colors input (either from the keyboard or from files passed as arguments) are in the rainbow:
#!/usr/bin/env perl
use v5.22; # introduced <<>> for safe opening of arguments
use warnings;
my %in_colors = map {$_ => 1}
qw(red orange yellow green blue indigo violet);
while (<<>>) {
chomp;
say "$_ is in the rainbow" if $in_colors{$_};
}List::Util has a bunch of functions for processing lists of pairs that I’ve found useful when pawing through hashes. pairgrep, for example, acts just like grep but instead assigns $a and $b to each key and value passed in and returns the resulting pairs that match. I’ve used it as a quick way to search for hash entries matching certain value conditions:
use List::Util 'pairgrep';
my %numbers = (zero => 0, one => 1, two => 2, three => 3);
my %odds = pairgrep {$b % 2} %numbers;Sure, you could do this by invoking a mix of plain grep, keys, and a hash slice, but it’s noisier and more repetitive:
use v5.20; # for key/value hash slice
my %odds = %numbers{grep {$numbers{$_} % 2} keys %numbers};pairgrep’s compiled C‑based XS code can also be faster, as evidenced by this Benchmark script that works through a hash made of the Unix words file (479,828 entries on my machine):
#!/usr/bin/env perl
use v5.20;
use warnings;
use List::Util 'pairgrep';
use Benchmark 'cmpthese';
my (%words, $count);
open my $fh, '<', '/usr/share/dict/words'
or die "can't open words: $!";
while (<$fh>) {
chomp;
$words{$_} = $count++;
}
close $fh;
cmpthese(100, {
grep => sub {
my %odds = %words{grep {$words{$_} % 2} keys %words};
},
pairgrep => sub {
my %odds = pairgrep {$b % 2} %words;
},
} );Benchmark output:
Rate grep pairgrep
grep 1.47/s -- -20%
pairgrep 1.84/s 25% --In general, I urge you to work through the Perl documentation’s tutorials on references, lists of lists, the data structures cookbook, and the FAQs on array and hash manipulation. Then dip into the various list-processing modules (especially the included List::Util and CPAN’s List::SomeUtils) for ready-made functions for common operations. You’ll find a wealth of techniques for creating, managing, and processing the data structures that your programs need.
I mentioned in passing last week that the next major release of Perl, v5.36, is set to enable warnings by default for code that opts in to use v5.35; or above. Commemorating Perl’s 34th birthday the week before that, I noted that the warnings system has been getting ever finer-grained since its introduction in 2000. And fellow Perl blogger and CPAN author Tom Wyant has been cataloging his favorites over the past several months—the latest as of this writing was on the “ambiguous” category of warnings, and you can find links to previous entries in his series at the bottom of that post.
It occurred to me afterward that there may be some confusion between the warnings pragma and the related warn function for reporting arbitrary runtime errors. warn outputs its arguments to the standard error (STDERR) stream, or if it’s not given any then you get a string with any exception from $@ ($EVAL_ERROR under use English) followed by a tab and then “...caught at <file> line x.” If that’s empty too, a plain warn just says, “Warning: something's wrong at <file> line x.”, which isn’t exactly helpful, but then again you didn’t give it much to go on.
warn output doesn’t have to go to STDERR, and this is where the relation to the warnings pragma comes in because both are governed by the __WARN__ signal handler in the %SIG hash. Normally, you might opt to only display runtime warnings if a debugging flag is set, like so:
#!/usr/bin/env perl
use strict;
use warnings;
my $DEBUG = 0;
$SIG{__WARN__} = sub { warn @_ if $DEBUG };
warn 'shhh'; # silenced
$DEBUG = 1;
warn 'hello warnings';But if you set that signal handler in a BEGIN block, it catches compile-time warnings too, in which case flipping a flag after the fact has no effect—the compiler’s already run:
#!/usr/bin/env perl
use strict;
use warnings;
my $DEBUG = 0;
BEGIN { $SIG{__WARN__} = sub { warn @_ if $DEBUG } }
my $foo = 'hello';
my $foo = 'world'; # no warning issued here
$DEBUG = 1;
my $foo = 'howdy'; # still nothingBy the way, both __WARN__ and __DIE__ hooks are also used by the Carp module and its friends, so you can use the same technique with their enhanced output:
#!/usr/bin/env perl
use strict;
use warnings;
use Carp qw(carp cluck);
my $DEBUG = 0;
BEGIN { $SIG{__WARN__} = sub { warn @_ if $DEBUG } }
carp 'quiet fish';
$DEBUG = 1;
loud_chicken();
sub loud_chicken {
cluck 'here comes a stack trace';
}You could use these as stepping stones towards a debug log for larger applications, but at that point, I’d suggest looking into one of the logging modules on CPAN like Log::Log4perl (not to be confused with that lately-problematic Java library), Log::Dispatch (which can be wired into Log4perl), or something else to suit your needs.