Tag: logging

  • Patch-​Perfect: Smarter Homebrew Upgrades on macOS

    Patch-​Perfect: Smarter Homebrew Upgrades on macOS

    This is a sto­ry about patches.

    At first, I just want­ed Homebrewto behave a lit­tle more polite­ly. Formulaeshould upgrade only on patch changes, with­out being dragged through minor and major bumps. That itch became a small(ish) Perl script, brew-patch-upgrade.pl.

    Along the way, I dis­cov­ered anoth­er patch was need­ed. My own log­ging adapter, Log::Any::Adapter::MacOS::OSLog, was­n’t build­ing and installing its bun­dle cor­rect­ly. Before the script can shine in the Mac Console app, I had to fix the adapter itself.

    What fol­lows is how those two threads came togeth­er. One is a tool that keeps Homebrew upgrades patch-​perfect. The oth­er is a log­ging adapter that final­ly behaves as a first-​class Perl module.

    What is Homebrew?

    Not to be con­fused with perlbrew, Homebrewis a free and open-​source pack­age man­ag­er for macOS (and Linux). It makes it easy to install and update soft­ware from the com­mand line. You don’t have to hunt down installers on web­sites. Instead, you can type com­mands like brew install wget. Homebrew will then fetch, build, and link the tool into place.

    Homebrew ≠ Perlbrew

    Everything is orga­nized under /opt/homebrew(on Apple Silicon-​based Macs) or /usr/local(on Intel). Homebrew can even man­age both command‑line util­i­ties (“for­mu­lae”) and desk­top apps (“casks”).

    In short: it’s the miss­ing pack­age man­ag­er Apple nev­er shipped, and it’s become an essen­tial part of many devel­op­ers’ workflows.

    Homebrew makes it easy to stay up to date — some­times tooeasy. By default, brew upgradejumps to the lat­est ver­sion of every­thing, even across minor and major releases.

    That’s fine when you want the newest fea­tures. Yet, it can be dis­rup­tive if all you real­ly need are the qui­et, patch-​level fixes.

    Scripted patch-​only upgrades

    My brew‑patch‑upgrade.pltakes a nar­row­er view: it pars­es brew outdatedcom­mand, com­pares seman­tic ver­sions, and upgrades only when the patch num­ber changes.

    That means:

    • 3.2.1→ 3.2.4will be upgraded.
    • 3.2.1→ 3.3.0will be skipped.
    • 3.2.1→ 4.0.0will be skipped.

    Normally, the scrip­t’s out­put is com­pa­ra­ble to brew upgrade, stream­ing the famil­iar Homebrew out­put to your ter­mi­nal for any patch-​level updates while not­ing any skipped versions:

    % brew-patch-upgrade.pl
    Skipping harfbuzz, needs manual review before upgrade at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 178.
    	main::process_formula(HASH(0x7c075dc90)) called at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 121
    Skipping woodpecker-cli, needs manual review before upgrade at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 178.
    	main::process_formula(HASH(0x7c0c1a180)) called at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 121
    ==> Upgrading 1 outdated package:
    zstd 1.5.6 -> 1.5.7
    ==> Fetching downloads for: zstd
    ==> Fetching zstd
    ==> Downloading https://ghcr.io/v2/homebrew/core/zstd/blobs/sha256:ddb0c145060bc2366ce5d58d95aa205bb15cb4c66948f20bb85e23fdb5eba7e9
    Already downloaded: /Users/mjg/Library/Caches/Homebrew/downloads/eb865576547e163ef6908f1e5762f4dc4d7fb548940f7b896ae12c0d5b202362--zstd--1.5.7.arm64_tahoe.bottle.1.tar.gz
    ==> Upgrading zstd
      1.5.6 -> 1.5.7
    ==> Pouring zstd--1.5.7.arm64_tahoe.bottle.1.tar.gz
    🍺  /opt/homebrew/Cellar/zstd/1.5.7: 32 files, 2.2MB
    ==> Running `brew cleanup zstd`...
    Disable this behaviour by setting `HOMEBREW_NO_INSTALL_CLEANUP=1`.
    Hide these hints with `HOMEBREW_NO_ENV_HINTS=1` (see `man brew`).
    Removing: /opt/homebrew/Cellar/zstd/1.5.6... (32 files, 2.1MB)
    Removing: /Users/mjg/Library/Caches/Homebrew/zstd_bottle_manifest--1.5.6... (11.9KB)
    Removing: /Users/mjg/Library/Caches/Homebrew/zstd--1.5.6... (753.9KB)
    ==> No outdated dependents to upgrade!

    But since I intend­ed this to run unat­tend­ed on a sched­ule, you can also tell it to log to a dif­fer­ent des­ti­na­tion using the envi­ron­ment vari­able LOG_ANY_DEFAULT_ADAPTER:

    % LOG_ANY_DEFAULT_ADAPTER=Stderr brew-patch-upgrade.pl
    using log adapter Log::Any::Adapter::Stderr
    outdated formulae: {casks => [],formulae => [{current_version => "12.0.0",installed_versions => ["11.5.1"],name => "harfbuzz",pinned => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),pinned_version => undef},{current_version => "3.10.0",installed_versions => ["3.9.0"],name => "woodpecker-cli",pinned => $VAR1->{formulae}[0]{pinned},pinned_version => undef},{current_version => "1.5.7",installed_versions => ["1.5.6","1.5.7"],name => "zstd",pinned => $VAR1->{formulae}[0]{pinned},pinned_version => undef}]}
    Skipping harfbuzz, needs manual review before upgrade at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 178.
    	main::process_formula(HASH(0xa1a01f510)) called at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 121
    Skipping woodpecker-cli, needs manual review before upgrade at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 178.
    	main::process_formula(HASH(0xa1acbcf30)) called at /Users/mjg/.local/bin/brew-patch-upgrade.pl line 121
    Starting brew upgrade for zstd...
    ==> Upgrading 1 outdated package:
    zstd 1.5.6 -> 1.5.7
    Finished brew upgrade for zstd
    Summary: upgraded 1, skipped 2, failed 0

    That’s use­ful for debug­ging. But, the real pay­off comes when you send logs to a more sophis­ti­cat­ed adapter. A good exam­ple is my Log::Any::Adapter::MacOS::OSLogpack­age. It tags dif­fer­ent lev­els of log mes­sages in macOS’ uni­fied log­ging sys­tem. These mes­sages can then be fil­tered and searched in the Console util­i­ty app.

    The com­mand is like the Stderrexam­ple above:

    % LOG_ANY_DEFAULT_ADAPTER=MacOS::OSLog brew-patch-upgrade.pl

    And pro­duces results that can be viewed in Console:

    Screenshot of macOS' Console app showing messages from the com.phoenixtrap.brew-patch-upgrade subsystem

    Or using the macOS log showcommand:

    Screenshot of macOS' iTerm2 app showing messages from the com.phoenixtrap.brew-patch-upgrade subsystem of the unified log

    At the end, the script logs a sum­ma­ry of upgrad­ed, skipped, and failed for­mu­lae. On fail­ure, it exits non-​zero to make it easy to use in automation.

    Highlights from the script source code

    For those curi­ous about the inter­nals, here are a few high­lights from the source.

    Skipping major and minor version bumps

    use version;
    ...
    sub process_formula ($formula_ref) {
        my $name = $formula_ref->{name};
        my ( $current, $installed ) = map { defined and qv($_) } (
            $formula_ref->{current_version},
            $formula_ref->{installed_versions}->[0] );
        unless ( $current and $installed ) {
            $logger->debug(
                'no current and/or installed version detected for',
                $name );
            return 'skipped';
        }
        my $result = 'failed';    # default
        my @current   = $current->{version}->@*;
        my @installed = $installed->{version}->@*;
        #<<<
        unless (@current   >= 3
            and @installed >= 3
            and $installed[0] == $current[0]
            and $installed[1] == $current[1]
            and $installed[2] <  $current[2] )
        #>>>
        {
            carp "Skipping $name, needs manual review before upgrade";
            return 'skipped';
        }
        ...

    At the start, process_formulachecks whether a for­mu­la from brew outdatedhas a valid seman­tic ver­sion. Only then does it com­pare patch numbers.

    To that end, it uses the core Perl versionmod­ule’s qvfunc­tion. This pars­es the cur­rent and lat­est installed ver­sion, avoid­ing a com­pli­cat­ed reg­u­lar expres­sion. The result­ing ver­sion objects can be treat­ed as hash ref­er­ences with a versionkey. This key is itself a ref­er­ence to an array con­tain­ing the major, minor, and patch ver­sion num­bers. If the major or minor ver­sion num­bers dif­fer between installed and cur­rent, the script calls for a man­u­al review. Then it returns a ​“skipped” status.

    Setting up the loggers

    use Log::Any qw($logger);
    use Log::Any::Adapter;
    ...
    use constant {
        MAC_LOG_ADAPTER_CLASS => 'Log::Any::Adapter::MacOS::OSLog',
        MAC_LOG_SUBSYSTEM     => 'com.phoenixtrap.brew-patch-upgrade',
    };
    my %brew_log;
    if ( $ENV{LOG_ANY_DEFAULT_ADAPTER} ) {
        my $adapter = Log::Any::Adapter->get(__PACKAGE__);
        Log::Any::Adapter->set( q(+) . ref $adapter,
            subsystem => MAC_LOG_SUBSYSTEM )
            if $adapter->isa(MAC_LOG_ADAPTER_CLASS);
        $logger->debug( 'using log adapter', ref $adapter );
        $SIG{__WARN__} = sub ($message) { $logger->alert($message) };
        $SIG{__DIE__}  = sub ($message) { $logger->fatal($message) };
        foreach my $stdio (qw(stdout stderr)) {
            Log::Any::Adapter->set(
                { category => "brew.$stdio" },
                q(+) . ref $adapter,
                (   subsystem   => MAC_LOG_SUBSYSTEM,
                    os_category => "brew-$stdio",
                )x!!$adapter->isa(MAC_LOG_ADAPTER_CLASS),
            );
            $brew_log{$stdio}
                = Log::Any->get_logger( category => "brew.$stdio" );
        }
    }

    If LOG_ANY_DEFAULT_ADAPTERis set, the script cre­ates three log­gers: one for the pro­gram itself, one for brew.stdout, and one for brew.stderr. All three have log­ic to check if the default log­ger class is Log::Any::Adapter::MacOS::OSLog. If it is, an appro­pri­ate OS-​level log­ging cat­e­go­ry is set. A unique sub­sys­tem name for the script, ​“com.phoenixtrap.brew-patch-upgrade”, is also assigned.

    Unfortunately, while work­ing on this macOS-​specific log­ging code, I made a dis­cov­ery. I real­ized that I need­ed to fix my log adapter class with a patch of its own.

    OSLog revisited

    I was proud of my ear­li­er workon the MacOS::OSLoglog adapter — a clever wrap­per around macOS’ log­ging func­tions using Perl for­eign func­tion inter­face (FFI). I even took a vic­to­ry lap, con­vert­ing the CPAN dis­tri­b­u­tion to use Dist::Zilla for plug­gable installer cre­ation and documentation.

    But out­side the pack­aged maclogscript, the mod­ule did­n’t work. Oops.

    Once I tried using the MacOS::OSLogadapter in brew-patch-upgrade.pl, the cracks became obvi­ous. The bun­dle was­n’t built or installed cor­rect­ly. My _find_my_bundlehackwas brit­tle — not a foun­da­tion to build on.

    The solu­tion was to stop fak­ing it and let the FFI::Platypustool­chain do its job. Using Dist::Zilla::Plugin::MakeMaker::Awesome in my dist.inifile, I rewrote the build process to use FFI::Build::MMin the gen­er­at­ed Makefile.PLinstaller. This meant that the bun­dle would com­pile and install in the right place instead of rely­ing on my _find_my_bundlehack:

    [MakeMaker::Awesome]
    delimiter = |
    ...
    header = |use FFI::Build::MM;
    header = |my $fbmm = FFI::Build::MM->new();
    header = |my %fbmm_args = $fbmm->mm_args(
    header = |  DISTNAME     => 'Log-Any-Adapter-MacOS-OSLog',
    header = |  NAME         => 'Log::Any::Adapter::MacOS::OSLog',
    header = |  VERSION_FROM => 'lib/Log/Any/Adapter/MacOS/OSLog.pm',
    header = |);
    header = |$fbmm_args{CCFLAGS} .= ' -mmacosx-version-min=10.12';
    footer = |sub MY::postamble { $fbmm->mm_postamble }
    WriteMakefile_arg = %fbmm_args

    The loca­tion of the C source code as well as com­pi­la­tion flags moved to a sep­a­rate ffi/OSLog.fbxfile, which FFI::Build::MMwould infer the name of based on the argu­ments passed to its mm_argsmethod.

    {
      source => [ 'ffi/OSLog.c' ],
      cflags => [ '-mmacosx-version-min=10.12' ],
      libs   => [ '-framework', 'OSLog' ],
    }

    I also updat­ed the C source codewith an #include <ffi_platypus_bundle.h>line, giv­ing the result­ing code bun­dle a prop­er entry point that FFI::Platypuscould recognize.

    Finally, I set a default sub­sys­tem name (“com.example.perl”). Now users can make it their default Log::Anyadapter and start log­ging immediately.

    The fix is in

    With those fix­es, the adapter now works as a native-​grade com­po­nent. No hacks. No guess­ing. Just clean, struc­tured logs flow­ing into the uni­fied log­ging system.

    Each run of brew-patch-upgrade.plnow shows a clear sub­sys­tem, with cat­e­gories for STDOUT, STDERR, and the script itself. This makes it easy to fil­ter and review. And because the script exits with a sum­ma­ry line and prop­er exit code, it slots neat­ly into automa­tion as well.

    In oth­er words, the log­ging side of brew-patch-upgrade.plis now as patch-​perfect as the upgrade log­ic it supports.

    Looking back, I real­ized that this project was­n’t just about tam­ing Homebrew’s upgrades or fix­ing a stub­born Perl mod­ule. It was about cre­at­ing a stronger con­nec­tion between the tools I rely on and their expect­ed behav­ior. I want them to be pre­dictable, leg­i­ble, and resilient. Both pack­ages work togeth­er, keep­ing my upgrades qui­et and inten­tion­al while sur­fac­ing them in the Console app.

    Both the scriptand log adapterare on Codeberg, with the adapter also on CPAN. If you’d like to try them out, file issues, or sug­gest improve­ments, the repos are waiting.

  • Kaiju Boss Battle: A Dist::Zilla Journey from Chaos to Co-Op

    Kaiju Boss Battle: A Dist::Zilla Journey from Chaos to Co-Op

    Last week I wrote about devel­op­ing a Perl mod­ule enabling me to out­put log entriesto macOS’ uni­fied log­ging sys­tem. This week­end’s adven­ture involved port­ing that mod­ule’s man­u­al process­es. These process­es includ­ed depen­den­cy man­age­ment, doc­u­men­ta­tion sync­ing, ver­sion bump­ing, and release. The goal was to make every­thing more auto­mat­ed and repeatable.

    And maybe even… monstrous.

    I was already famil­iar with the Dist::Zilla(DZil to its friends) suite of tools and plu­g­ins. I also knew that some Perl devel­op­ers view it as a huge bar­ri­er to entry. This per­cep­tion affects their will­ing­ness to con­tribute to oth­ers’ projects.

    So even though Log-​Any-​Adapter-​MacOS-​OSLogwas a small mod­ule of inter­est to a lim­it­ed cod­ing audi­ence (macOS Perl users), I thought it wise to have both a mainbranch and sep­a­rate build/mainbranch for those pro­gram­mers who want­ed to work as though things had bare­ly changed:

    • Entire source code with full POD-​formatted documentation;
    • A Makefile.PL script to gen­er­ate a portable build­ing, test­ing, and installing Makefile;
    • Plus, every Perl dis­tri­b­u­tion should sup­ply the typ­i­cal README, MANIFEST, LICENSE, and CONTRIBUTING doc­u­men­ta­tion. This is essen­tial if it’s meant for pub­lic consumption.

    A small dis­tri­b­u­tion would also give me a mod­el I can scale up for larg­er projects. At the very least, it was anoth­er learn­ing oppor­tu­ni­ty for me.

    Boy, was it.

    Why Dist::Zilla?

    Because I know it can auto­mate away the boil­er­plate code and rep­e­ti­tion of infor­ma­tion that’s unfor­tu­nate­ly nec­es­sary in a mod­ern Perl mod­ule distribution:

    • the ver­sion num­bers in every mod­ule and script
    • the README that often includes the same intro­duc­to­ry text as the main mod­ule’s documentation
    • the nam­ing, order, and con­tent of POD sec­tions (via DZil’s sis­ter suite, Pod::Weaver), some of which repeat dis­tri­b­u­tion meta­da­ta like author, ver­sion, sup­port, copy­right, license, and so on

    If you need fur­ther detail, Dan Book’s Dist::Zilla::Starteris, as its name sug­gests. an excel­lent and remark­ably thor­ough guide to the how and why of Dist::Zilla. It even cov­ers the basic struc­ture of CPAN dis­tri­b­u­tionsand the his­to­ry of Perl mod­ule author­ingtools.

    Why not something else?

    Because just as in the sto­ry of Goldilocks and the three bears, every­thing else I looked at seemed want­i­ng in some way:

    • Module::Build/​Module::Build::Tiny: Although min­i­mal, pure-​Perl, and easy to install, Module::Build prop­er still requires ship­ping extra boil­er­plate and dupli­cate meta­da­ta. ::Tiny shaves that down fur­ther but drops whole swaths of func­tion­al­i­ty. As an exam­ple, you can’t spec­i­fy at set­up time that users need a spe­cif­ic oper­at­ing sys­tem. It’s a deal-​breaker for a mod­ule that requires macOS 10.12 Sierra or newer.
    • Minilla: Opinionated con­ven­tion over con­fig­u­ra­tion, but I don’t share its opin­ions. Overriding direc­to­ry lay­out, test struc­ture, and README/​license han­dling would mean fight­ing Minilla’s defaults, DZil-​config style, with­out DZil’s plu­g­in ecosystem.
    • ShipIt: Simple, one-​file con­fig­u­ra­tion. But too sim­ple: it’s most­ly release automa­tion and not author­ing automa­tion. Everything I want­ed to tool away is still manual.
    • No or min­i­mal tool­chain: That was how I start­ed this mod­ule, with ExtUtils::MakeMaker. Totally man­u­al, with every con­trib­u­tor, includ­ing yours tru­ly, need­ing to remem­ber all the mov­ing parts by hand.

    Off to see the lizard!

    My DZil dist.inicon­fig­u­ra­tion file start­ed off sim­ply enough:

    name    = Log-Any-Adapter-MacOS-OSLog
    author  = Mark Gardner <[email protected]>
    license = Perl_5
    copyright_holder = Mark Gardner
    version = 0.0.5 ; bump as appropriate
    
    [MetaResources]
    repository.type = git
    bugtracker.web  = https://codeberg.org/mjgardner/perl-Log-Any-Adapter-MacOS-OSLog/issues
    
    [Repository]
    web = https://codeberg.org/mjgardner/perl-Log-Any-Adapter-MacOS-OSLog

    That [MetaResources]is the first evi­dence of using plu­g­ins. It pro­vides a nice tidy way to spec­i­fy the meta­da­ta. Automated tools can use this infor­ma­tion to index, exam­ine, pack­age, or install Perl distributions.

    So far so good. But then the mon­ster attacked.

    [@Filter]
    -bundle = @Basic
    -remove = GatherDir
    -remove = MakeMaker
    -remove = Readme
    ; License plugin still active here
    
    [GatherDir]
    include_dotfiles = 1
    exclude_filename = .gitignore
    exclude_filename = .build
    
    [ReadmeAnyFromPod]
    type = markdown
    filename = README.md
    location = build
    
    [CopyFilesFromBuild]
    copy = README.md
    
    ; various MakeMaker::*, Git::*, Pod::Weaver, etc. plugins

    I was get­ting errors from the dzil --buildcom­mand as it attempt­ed to add the same file mul­ti­ple times. These were gen­er­at­ed files com­mit­ted to the mainver­sion con­trol branch and a build/mainbranch for non-​DZil contributors.

    This brought me to a dead stop. I went down a rabbit-​hole exam­in­ing built files from rsync(1)with [Run::AfterBuild]., com­par­ing them to the branch I had set up.

    Plus my .perlcriticrcfile was dis­ap­pear­ing from the build arti­facts despite that instruc­tion to [GatherDir]to include dot­files.

    “Let them fight.”

    LICENSE to kill (files)

    Eventually I traced the LICENSE file dupli­ca­tion to duel­ing DZil plu­g­ins. The afore­men­tioned [GatherDir]was duti­ful­ly copy­ing the file from my work­ing root even as the [License]plu­g­in was gen­er­at­ing it. I did­n’t want to lose the lat­ter source of what­ev­er license I hap­pened to be using to dis­trib­ute this code.

    And .perlcriticrc? It turns out the [PruneCruft]plu­g­in does­n’t lis­ten to its friend [GatherDir]and was hoover­ing it away.

    In the end, I had to expand my manually-​configured plu­g­in ros­ter a lit­tle to keep them from fight­ing over what files came from where:

    [@Filter]
    -bundle = @Basic
    -remove = GatherDir
    -remove = PruneCruft
    -remove = MakeMaker
    -remove = Readme
    
    [PruneCruft]
    except = .perlcriticrc
    
    [GatherDir]
    include_dotfiles = 1
    exclude_filename = .gitignore
    exclude_filename = .build
    exclude_filename = LICENSE
    
    [ReadmeAnyFromPod]
    type            = markdown
    filename        = README.md
    
    [CopyFilesFromBuild]
    copy = README.md
    copy = LICENSE

    With these tweaks to Log-​Any-​Adapter-​MacOS-​OSLog’s dist.ini:

    • [License]gen­er­ates its file,
    • [CopyFilesFromBuild]copies it along with the README into the root for my commit,
    • And [GatherDir]does­n’t stomp on it like so many Tokyo city blocks.

    The build/mainbranch for non-​DZil con­trib­u­tors would con­tain exact­ly what they and I expect­ed. It would be a match of the DZil work­ing copy plus arti­facts, git cloneable with no surprises.

    Lessons learned, and what’s next?

    I want­ed to serve two styles of devel­op­ment and had signed myself up for a com­pli­cat­ed author­ing process. I now have more knowl­edge of which plu­g­in runs in each phaseof the build. This allows me to decide between gen­er­at­ed and version-​controlled files.

    CPAN has a rich vari­ety of per­son­al Dist::Zilla::PluginBundle::s fac­tor­ing indi­vid­ual authors’ pref­er­ences into a sin­gle place. Those authors don’t have to copy-​and-​paste DZil con­fig­u­ra­tions around. It’s past time for me to mint one of my own bun­dles for more con­sis­tent and well-​understood Perl dis­tri­b­u­tions. Then I can start spin­ning up new projects with­out revis­it­ing the same pain.


    This week­end has brought on a men­tal shift. I moved from fight­ing DZil’s defaults to mak­ing it work myway. I also found sat­is­fac­tion in a pre­dictable, minimal-​effort Perl release pipeline.

    The mon­ster was just a guy in a rub­ber suit all along.

  • Logging from Perl to macOS’ unified log with FFI and Log::Any

    Logging from Perl to macOS’ unified log with FFI and Log::Any

    Part 1: The elephant in the room

    A few weeks ago, I start­ed host­ing my own Mastodon instanceon a Mac miniin my home office. I want­ed to join the social Fediverseon my own terms–but it did­n’t take long to notice bal­loon­ing disk usage. Cached media from oth­er users’ posts was pil­ing up fast.

    That got me think­ing: how do I track this growth before it gets out of hand?

    Logging seemed like the obvi­ous answer. On Unix and Linux sys­tems, it’s straight­for­ward enough. But on macOS, find­ing a native, main­tain­able solu­tion takes more digging.

    Part 2: Feeding the Apple

    macOS is Unix-​based, so you’d expect log­ging to be sim­ple. You caninstall logro­tatevia Homebrew, then sched­ule it with cron(8). It works–but it adds lay­ers of con­fig­u­ra­tion files, per­mis­sions, and guess­work. I want­ed some­thing native. Something that felt like it belonged on a Mac.

    Turns out, macOS offers twobuilt-​in options. One is newsys­log, a BSD-​style tool that rotates logs based on size or time. It’s reli­able, but it requires priv­i­leged root-owned con­fig­u­ra­tion files and feels like a holdover from old­er Unix systems.

    The oth­er is Apple’s uni­fied log­ging sys­tem–a mod­ern API used across macOS, iOS, and even watchOS. It’s struc­tured, search­able, and already baked into the plat­form. That’s the one I decid­ed to explore.

    Howard Oakley’s explain­er on the Unified Loghelped me under­stand Apple’s sys­tem for con­sol­i­dat­ing logs. It showed how they are stored in a com­pressed bina­ry for­mat, com­plete with struc­tured meta­da­ta and pri­va­cy con­trols. With that foun­da­tion, I turned to Apple’s OSLog Framework doc­u­men­ta­tion. It showed how to tag entries and fil­ter them with pred­i­cates. macOS han­dles the rest.

    It’s elegant–but you need to use the API to write logs. Yes, read­ing and fil­ter­ing can be done on the com­mand line or in the Console app. But Apple seems to expect log­ging to be the sole province of Swift and Objective‑C devel­op­ers. I’d rather not have to learn a new pro­gram­ming lan­guage just to write logs.

    UPDATE: Howard Oakley’s blow­holeutil­i­typro­vides a sim­ple way to write to the uni­fied log from the com­mand line, but all mes­sages come from the ​“co.eclecticlight.blowhole” sub­sys­tem with a ​“gen­er­al” cat­e­go­ry. We can do better.

    Part 3: A platypus in the key of C

    I doknow Perl. I also know just enough Cto be dan­ger­ous. And I briefly con­sid­ered learn­ing Swift or Objective‑C. Nevertheless, I won­dered about bridg­ing Perl to Apple’s uni­fied log­ging sys­tem with­out switch­ing languages.

    macOS expos­es a C API in <os/log.h>:

    #include <os/log.h>
    
    void
    os_log(os_log_t log, const char *format, ...);
    
    void
    os_log_info(os_log_t log, const char *format, ...);
    
    void
    os_log_debug(os_log_t log, const char *format, ...);
    
    void
    os_log_error(os_log_t log, const char *format, ...);
    
    void
    os_log_fault(os_log_t log, const char *format, ...);

    Perl’s CPANhas a mod­ule called FFI::Platypusthat would let me call for­eign func­tions in C and oth­er lan­guages. It looked promising.

    But there’s a catch: these log­ging func­tions are vari­adic macros, not plain func­tions. That makes them inac­ces­si­ble via FFI. Worse, they expand into pri­vate API calls–unstable across OS updates and risky to rely upon.

    So I wrote a small C wrap­per to con­vert each macro into a prop­er func­tion. This makes them FFI-​safe and lets me con­trol vis­i­bil­i­ty (pub­lic log­ging vs. pri­vate, redact­ed log­ging) using Apple’s for­mat specifiers:

    #include <os/log.h>
    
    #define DEFINE_OSLOG_WRAPPERS(level_macro, suffix)    \
        void os_log_##suffix##_public(os_log_t log,       \
                                      const char *msg) {  \
            level_macro(log, "%{public}s", msg);          \
        }                                                 \
        void os_log_##suffix##_private(os_log_t log,      \
                                       const char *msg) { \
            level_macro(log, "%{private}s", msg);         \
        }
    
    // Generate wrappers for each log level
    DEFINE_OSLOG_WRAPPERS(os_log, default)
    DEFINE_OSLOG_WRAPPERS(os_log_info, info)
    DEFINE_OSLOG_WRAPPERS(os_log_debug, debug)
    DEFINE_OSLOG_WRAPPERS(os_log_error, error)
    DEFINE_OSLOG_WRAPPERS(os_log_fault, fault)

    This macro gen­er­ates two func­tions per log level–one pub­lic, one private–giving down­stream Perl code a choice. It’s ver­bose, but it’s safe, auditable, and future-proof.

    Part 4: Plugging into Log::Any

    With the wrap­per library in place, I began map­ping Apple’s log lev­els to some­thing Perl can use. I chose Log::Any from CPANbecause it’s light­weight, wide­ly sup­port­ed, and its adapters don’t lock you into a spe­cif­ic back-​end. The same code that logs to the screen can also log to a file, or in our case, Apple’s system.

    Admittedly, at this point I’m no longer writ­ing a sim­ple log­ging script for my Mastodon instance. Instead, it’s a full-​fledged log­ging mod­ule. Oh well.

    Some Log::Any lev­elsshare the same under­ly­ing Apple call– OSLog does­n’t dis­tin­guish between noticeand infoor traceand debug. That’s a lit­tle dif­fer­ent from how Unix sys­logdoes things, but that’s fine. The goal here is com­pat­i­bil­i­ty, not per­fect fidelity.

    Building a sim­ple dis­patch table to route log mes­sages based on lev­el, I then used FFI::Platypus to bind each wrap­per function:

    use FFI::Platypus 2.00;
    
    my %OS_LOG_MAP = (
        trace     => 'os_log_debug',
        debug     => 'os_log_debug',
        info      => 'os_log_info',
        notice    => 'os_log_info',
        warning   => 'os_log_fault',
        error     => 'os_log_error',
        critical  => 'os_log_default',
        alert     => 'os_log_default',
        emergency => 'os_log_default',
    );
    
    my $ffi = FFI::Platypus->new(
        api => 2,
        lib => [ './liboslogwrapper.dylib' ],
    );
    
    $ffi->attach(
        [ os_log_create => '_os_log_create' ],
        [ 'string', 'string' ],
        'opaque',
    );
    
    # attach each wrapper function
    my %UNIQUE_OS_LOG = map { $_ => 1 } values %OS_LOG_MAP;
    foreach my $function ( keys %UNIQUE_OS_LOG ) {
        for my $variant (qw(public private)) {
            my $name = "${function}_$variant";
            $ffi->attach(
                [ $name => "_$name" ],
                [ 'opaque', 'string' ],
                'void',
            );
        }
    }

    This set­up gives me a clean way to log from Perl using Apple’s native sys­tem. I can achieve this with­out touch­ing Swift, Objective‑C, or exter­nal tools. Each log lev­el maps to a C wrap­per, and the FFI lay­er han­dles the rest.

    Now I just need an initfunc­tion to cre­ate the os_​log_​tobject and a set of meth­ods for log­ging and detect­ing whether a giv­en log lev­el is enabled:

    use strict;
    use Carp;
    use base qw(Log::Any::Adapter::Base);
    use Log::Any::Adapter::Util qw(
      detection_methods
      numeric_level
    );
    
    sub init {
        my $self = shift;
        $self->{private} ||= 0;
        croak 'subsystem is required'
          unless defined $self->{subsystem};
    
        $self->{_os_log} = _os_log_create(
          @{$self}{qw(subsystem category)},
        );
    
        return;
    }
    
    foreach my $log_level ( keys %OS_LOG_MAP ) {
        no strict 'refs';
        *{$log_level} = sub {
            my ( $self, $message ) = @_;
    
            &{  "_$OS_LOG_MAP{$log_level}_"
                    . ( $self->{private}
                        ? 'private'
                        : 'public'
                    ) }( $self->{_os_log}, $message );
        };
    }
    
    foreach my $method ( detection_methods() ) {
        my $method_level = numeric_level(substr $method 3);
        no strict 'refs';
        *{$method} = sub {
            !!( $method_level <= (
              $_[0]->{log_level} // numeric_level('info')
            ) );
        };
    }

    What’s that ​“sub­sys­tem” bit up there? That’s the term macOS uses for iden­ti­fy­ing process­es in logs. They’re usu­al­ly for­mat­ted in reverse DNS nota­tion (e.g., ​“com.example.perl”). Once again, Howard Oakley has a great explain­er on the top­ic.

    Also, there’s some metapro­gram­minggoing on there:

    • The first fore­achloop cre­ates func­tions called trace, debug, and info. These func­tions call the cor­re­spond­ing FFI::Platypus-created func­tions. It uses the pri­vate vari­ants if the pri­vateattribute for the log adapter was set.
    • The sec­ond fore­achloop cre­ates cre­ates func­tions called is_​trace, is_​debug, is_​info, etc., that return true if the adapter is catch­ing that lev­el of log message.

    Part 5: At long last, logging… mostly

    Once this is pack­aged in a Perl mod­ule, how do you use it? At least that part isn’t toohard:

    use Log::Any '$log', default_adapter => [
      'MacOS::OSLog', subsystem => 'com.phoenixtrap.perl',
    ];
    use English;
    use Carp qw(longmess);
    
    $log->info('Hello from Perl!');
    $log->infof('You are using Perl %s', $PERL_VERSION);
    
    $log->trace( longmess('tracing!') );
    $log->debug(     'debugging!'     );
    $log->info(      'informing!'     );
    $log->notice(    'noticing!'      );
    $log->warning(   'warning!'       );
    $log->error(     'erring!'        );
    $log->critical(  'critiquing!'    );
    $log->alert(     'alerting!'      );
    $log->emergency( 'emerging!'      );

    And then you can run this com­mand line to stream log mes­sages from the sub­sytem used above:

    % log stream --level debug \
      --predicate 'subsystem == "com.phoenixtrap.perl"

    What hap­pened to the traceand debuglog mes­sages that were sup­posed to call os_log_debug(3)? According to macOS’ log(1)man­u­al page, you have to explic­it­ly allow debug­ging out­putfor a giv­en subsystem:

    % sudo log config --mode "level:debug" \
      --subsystem com.phoenixtrap.perl

    Et voilà!

    Hmm, same lack of debug­ging messages.

    I’m still fig­ur­ing this out. Any clues? Drop me a line!

    UPDATE:This is now fixed thanks to some inspi­ra­tion from the source code of Log::Any::Adapter::Syslog. I’ve updat­ed the code on Codeberg; here is the diff.

    Bonus: Fancy output

    Thanks to Log::Any::Proxy, you also get sprintffor­mat­ting vari­ant functions:

    use English;
    $log->infof(
        'You are using Perl %s in %d',
        $PERL_VERSION, (localtime)[5] + 1900,
    );
    You are using Perl v5.40.2 in 2025

    If you out­put an object that over­loads string rep­re­sen­ta­tion, you get that string:

    use DateTime;
    $log->infof('It is now %s', DateTime->now);
    It is now 2025-08-10T20:16:50

    And you get single-​line Data::Dumperout­put of com­plex data struc­tures, plus replac­ing unde­fined val­ues with the string ​“undef”:

    $log->info( {
        foo    => 'hello',
        bar    => 'world',
        colors => [ qw(
            red
            green
            blue
        ) ],
        null => undef,
    } );
    {bar => "world",colors => ["red","green","blue"],foo => "hello",null => undef}

    Conclusion: Build once, use everywhere

    The best tools aren’t always the ones you planned to build. They’re the ones that solve a prob­lem cleanly–and then solve five more you hadn’t thought of yet.

    What start­ed as a quick fix for Mastodon media mon­i­tor­ing became a reusable bridge between Perl and macOS’ Unified Log. Along the way, I got to explore Apple’s log­ging inter­nals, write an FFI-​respecting C wrap­per, and inte­grate clean­ly with Log::Any. The result­ing code is mod­u­lar, auditable, and–most importantly–maintainable.

    I did­n’t set out to write a log­ging adapter. But when you care about clean ops and repro­ducible infra­struc­ture, some­times the best tools are the ones you build your­self. And if they hap­pen to be over-​engineered for the task at hand? All the better–they’ll prob­a­bly out­live it.

    Try it out or contribute!

    The full adapter code is on Codeberg.If you’re log­ging from Perl on macOS, give it a spin. Contributions, bug reports, and real-​world feed­back are welcome–especially if you’re test­ing it in pro­duc­tion or on old­er macOS versions.

    I’ll do my best to stay com­pat­i­ble with past and future macOS and Perl releas­es. Keeping the code auditable and min­i­mal should help it stay use­ful with­out becom­ing a mov­ing target.

  • Perl warnings and the warn function

    Perl warnings and the warn function

    I men­tioned in pass­ing last weekthat the next major release of Perl, v5.36, is set to enable warningsby default for code that opts in to use v5.35;or above. Commemorating Perl’s 34th birth­day the week before that, I not­ed that the warn­ings sys­tem has been get­ting ever finer-​grainedsince its intro­duc­tion in 2000. And fel­low Perl blog­ger and CPAN author Tom Wyanthas been cat­a­loging his favorites over the past sev­er­al months—the lat­est as of this writ­ing was on the ​“ambigu­ous” cat­e­go­ry of warn­ings, and you can find links to pre­vi­ous entries in his series at the bot­tom of that post.

    It occurred to me after­ward that there may be some con­fu­sion between the warningsprag­ma and the relat­ed warnfunc­tionfor report­ing arbi­trary run­time errors. warnout­puts its argu­ments to the stan­dard error (STDERR) stream, or if it’s not giv­en any then you get a string with any excep­tion from $@($EVAL_ERRORunder use English) fol­lowed by a tab and then “...caught at <file> line x.” If that’s emp­ty too, a plain warnjust says, ​“Warning: something's wrong at <file> line x.”, which isn’t exact­ly help­ful, but then again you didn’t give it much to go on.

    warnout­put doesn’t have to go to STDERR, and this is where the rela­tion to the warn­ings prag­ma comes in because both are gov­erned by the __WARN__sig­nal han­dler in the %SIGhash. Normally, you might opt to only dis­play run­time warn­ings if a debug­ging flag is set, like so:

    #!/usr/bin/env perl
    
    use strict;
    use warnings;
    
    my $DEBUG = 0;
    $SIG{__WARN__} = sub { warn @_ if $DEBUG };
    warn 'shhh'; # silenced
    
    $DEBUG = 1;
    warn 'hello warnings';

    But if you set that sig­nal han­dler in a BEGINblock, it catch­es compile-​time warn­ings too, in which case flip­ping a flag after the fact has no effect—the compiler’s already run:

    #!/usr/bin/env perl
    
    use strict;
    use warnings;
    
    my $DEBUG = 0;
    BEGIN { $SIG{__WARN__} = sub { warn @_ if $DEBUG } }
    my $foo = 'hello';
    my $foo = 'world'; # no warning issued here
    
    $DEBUG = 1;
    my $foo = 'howdy'; # still nothing

    By the way, both __WARN__and __DIE__hooks are also used by the Carp mod­uleand its friends, so you can use the same tech­nique with their enhanced output:

    #!/usr/bin/env perl
    
    use strict;
    use warnings;
    use Carp qw(carp cluck);
    
    my $DEBUG = 0;
    BEGIN { $SIG{__WARN__} = sub { warn @_ if $DEBUG } }
    carp 'quiet fish';
    
    $DEBUG = 1;
    loud_chicken();
    
    sub loud_chicken {
        cluck 'here comes a stack trace';
    }

    You coulduse these as step­ping stones towards a debug log for larg­er appli­ca­tions, but at that point, I’d sug­gest look­ing into one of the log­ging mod­ules on CPAN like Log::Log4perl(not to be con­fused with that lately-​problematic Java library), Log::Dispatch(which can be wired into Log4perl), or some­thing elseto suit your needs.

  • A brief note on Log4perl

    A brief note on Log4perl

    The Java world had an… inter­est­ing week­end when secu­ri­ty researchers revealed on December 9 a vul­ner­a­bil­i­tyin the pop­u­lar Apache Log4j 2soft­ware library for record­ing and debug­ging events. Systems as diverse as Amazon Web Services, Apple iCloud, and the Minecraft video gamecould be exploit­ed to run arbi­trary code on a serv­er mere­ly by send­ing a specially-​crafted string of text. Information tech­nol­o­gy pro­fes­sion­als have been scram­bling ever since the ini­tial dis­clo­sure to patch, upgrade, recon­fig­ure, or oth­er­wise pro­tect affect­ed servers. It’s bad, and past unpatched vul­ner­a­bil­i­ties like thishave been respon­si­ble for the expo­sure of mil­lions of people’s sen­si­tive data.

    Many Perl appli­ca­tions use the similarly-​named and ‑designed Log::Log4perllibrary, and the good news is that as far as I can tell the lat­ter doesn’t suf­fer from the type of vul­ner­a­bil­i­ty described above. This doesn’t mean poorly-​written or ‑con­fig­ured Perl-​based sys­tems are immune to all exploits, just this par­tic­u­lar one.You should be safe to con­tin­ue using Log4perl unless some­one has delib­er­ate­ly con­fig­ured it oth­er­wise, and in fact, my workuses it extensively.

    You might be sur­prised to read me sug­gest­ing a log­ging frame­work after writ­ing mul­ti­ple arti­cles espous­ing the Perl step debug­geras an alter­na­tive. Log4perl devel­op­er Mike Schilli’s 2002 intro­duc­tion to the pack­age for Perl.comcame down on the oppo­site side of the argu­ment. It can seem like one of those pro­gram­mer reli­gious issues like tabs vs. spaces, vim vs. Emacs, or Linuxvs. Windows. (For the record, the cor­rect answers are spaces, BBEdit, and macOS. 😉)

    But in this case, you can and should have the best of both worlds—logging at dif­fer­ent lev­elsto appro­pri­ate des­ti­na­tionswhile still drop­ping into the inter­ac­tive debug­gerwhen you need to do some­thing trick­i­er like exam­ine pro­gram state or tweak a data struc­ture on the fly. I use both tech­niques and only empha­size the advo­ca­cy of step debug­ging because it’s under­stood less.