Tag: Apple

  • Logging from Perl to macOS’ unified log with FFI and Log::Any

    Logging from Perl to macOS’ unified log with FFI and Log::Any

    Part 1: The elephant in the room

    A few weeks ago, I start­ed host­ing my own Mastodon instance on a Mac mini in my home office. I want­ed to join the social Fediverse on my own terms–but it did­n’t take long to notice bal­loon­ing disk usage. Cached media from oth­er users’ posts was pil­ing up fast.

    That got me think­ing: how do I track this growth before it gets out of hand?

    Logging seemed like the obvi­ous answer. On Unix and Linux sys­tems, it’s straight­for­ward enough. But on macOS, find­ing a native, main­tain­able solu­tion takes more digging.

    Part 2: Feeding the Apple

    macOS is Unix-​based, so you’d expect log­ging to be sim­ple. You can install logro­tate via Homebrew, then sched­ule it with cron(8). It works–but it adds lay­ers of con­fig­u­ra­tion files, per­mis­sions, and guess­work. I want­ed some­thing native. Something that felt like it belonged on a Mac.

    Turns out, macOS offers two built-​in options. One is newsys­log, a BSD-​style tool that rotates logs based on size or time. It’s reli­able, but it requires priv­i­leged root-owned con­fig­u­ra­tion files and feels like a holdover from old­er Unix systems.

    The oth­er is Apple’s uni­fied log­ging sys­tem–a mod­ern API used across macOS, iOS, and even watchOS. It’s struc­tured, search­able, and already baked into the plat­form. That’s the one I decid­ed to explore.

    Howard Oakley’s explain­er on the Unified Log helped me under­stand Apple’s sys­tem for con­sol­i­dat­ing logs. It showed how they are stored in a com­pressed bina­ry for­mat, com­plete with struc­tured meta­da­ta and pri­va­cy con­trols. With that foun­da­tion, I turned to Apple’s OSLog Framework doc­u­men­ta­tion. It showed how to tag entries and fil­ter them with pred­i­cates. macOS han­dles the rest.

    It’s elegant–but you need to use the API to write logs. Yes, read­ing and fil­ter­ing can be done on the com­mand line or in the Console app. But Apple seems to expect log­ging to be the sole province of Swift and Objective‑C devel­op­ers. I’d rather not have to learn a new pro­gram­ming lan­guage just to write logs.

    UPDATE: Howard Oakley’s blow­hole util­i­ty pro­vides a sim­ple way to write to the uni­fied log from the com­mand line, but all mes­sages come from the co.eclecticlight.blowhole” sub­sys­tem with a gen­er­al” cat­e­go­ry. We can do better.

    Part 3: A platypus in the key of C

    I do know Perl. I also know just enough C to be dan­ger­ous. And I briefly con­sid­ered learn­ing Swift or Objective‑C. Nevertheless, I won­dered about bridg­ing Perl to Apple’s uni­fied log­ging sys­tem with­out switch­ing languages.

    macOS expos­es a C API in <os/log.h>:

    #include <os/log.h>
    
    void
    os_log(os_log_t log, const char *format, ...);
    
    void
    os_log_info(os_log_t log, const char *format, ...);
    
    void
    os_log_debug(os_log_t log, const char *format, ...);
    
    void
    os_log_error(os_log_t log, const char *format, ...);
    
    void
    os_log_fault(os_log_t log, const char *format, ...);

    Perl’s CPAN has a mod­ule called FFI::Platypus that would let me call for­eign func­tions in C and oth­er lan­guages. It looked promising.

    But there’s a catch: these log­ging func­tions are vari­adic macros, not plain func­tions. That makes them inac­ces­si­ble via FFI. Worse, they expand into pri­vate API calls–unstable across OS updates and risky to rely upon.

    So I wrote a small C wrap­per to con­vert each macro into a prop­er func­tion. This makes them FFI-​safe and lets me con­trol vis­i­bil­i­ty (pub­lic log­ging vs. pri­vate, redact­ed log­ging) using Apple’s for­mat specifiers:

    #include <os/log.h>
    
    #define DEFINE_OSLOG_WRAPPERS(level_macro, suffix)    \
        void os_log_##suffix##_public(os_log_t log,       \
                                      const char *msg) {  \
            level_macro(log, "%{public}s", msg);          \
        }                                                 \
        void os_log_##suffix##_private(os_log_t log,      \
                                       const char *msg) { \
            level_macro(log, "%{private}s", msg);         \
        }
    
    // Generate wrappers for each log level
    DEFINE_OSLOG_WRAPPERS(os_log, default)
    DEFINE_OSLOG_WRAPPERS(os_log_info, info)
    DEFINE_OSLOG_WRAPPERS(os_log_debug, debug)
    DEFINE_OSLOG_WRAPPERS(os_log_error, error)
    DEFINE_OSLOG_WRAPPERS(os_log_fault, fault)

    This macro gen­er­ates two func­tions per log level–one pub­lic, one private–giving down­stream Perl code a choice. It’s ver­bose, but it’s safe, auditable, and future-proof.

    Part 4: Plugging into Log::Any

    With the wrap­per library in place, I began map­ping Apple’s log lev­els to some­thing Perl can use. I chose Log::Any from CPAN because it’s light­weight, wide­ly sup­port­ed, and its adapters don’t lock you into a spe­cif­ic back-​end. The same code that logs to the screen can also log to a file, or in our case, Apple’s system.

    Admittedly, at this point I’m no longer writ­ing a sim­ple log­ging script for my Mastodon instance. Instead, it’s a full-​fledged log­ging mod­ule. Oh well.

    Some Log::Any lev­els share the same under­ly­ing Apple call– OSLog does­n’t dis­tin­guish between notice and info or trace and debug. That’s a lit­tle dif­fer­ent from how Unix sys­log does things, but that’s fine. The goal here is com­pat­i­bil­i­ty, not per­fect fidelity.

    Building a sim­ple dis­patch table to route log mes­sages based on lev­el, I then used FFI::Platypus to bind each wrap­per function:

    use FFI::Platypus 2.00;
    
    my %OS_LOG_MAP = (
        trace     => 'os_log_debug',
        debug     => 'os_log_debug',
        info      => 'os_log_info',
        notice    => 'os_log_info',
        warning   => 'os_log_fault',
        error     => 'os_log_error',
        critical  => 'os_log_default',
        alert     => 'os_log_default',
        emergency => 'os_log_default',
    );
    
    my $ffi = FFI::Platypus->new(
        api => 2,
        lib => [ './liboslogwrapper.dylib' ],
    );
    
    $ffi->attach(
        [ os_log_create => '_os_log_create' ],
        [ 'string', 'string' ],
        'opaque',
    );
    
    # attach each wrapper function
    my %UNIQUE_OS_LOG = map { $_ => 1 } values %OS_LOG_MAP;
    foreach my $function ( keys %UNIQUE_OS_LOG ) {
        for my $variant (qw(public private)) {
            my $name = "${function}_$variant";
            $ffi->attach(
                [ $name => "_$name" ],
                [ 'opaque', 'string' ],
                'void',
            );
        }
    }

    This set­up gives me a clean way to log from Perl using Apple’s native sys­tem. I can achieve this with­out touch­ing Swift, Objective‑C, or exter­nal tools. Each log lev­el maps to a C wrap­per, and the FFI lay­er han­dles the rest.

    Now I just need an init func­tion to cre­ate the os_​log_​t object and a set of meth­ods for log­ging and detect­ing whether a giv­en log lev­el is enabled:

    use strict;
    use Carp;
    use base qw(Log::Any::Adapter::Base);
    use Log::Any::Adapter::Util qw(
      detection_methods
      numeric_level
    );
    
    sub init {
        my $self = shift;
        $self->{private} ||= 0;
        croak 'subsystem is required'
          unless defined $self->{subsystem};
    
        $self->{_os_log} = _os_log_create(
          @{$self}{qw(subsystem category)},
        );
    
        return;
    }
    
    foreach my $log_level ( keys %OS_LOG_MAP ) {
        no strict 'refs';
        *{$log_level} = sub {
            my ( $self, $message ) = @_;
    
            &{  "_$OS_LOG_MAP{$log_level}_"
                    . ( $self->{private}
                        ? 'private'
                        : 'public'
                    ) }( $self->{_os_log}, $message );
        };
    }
    
    foreach my $method ( detection_methods() ) {
        my $method_level = numeric_level(substr $method 3);
        no strict 'refs';
        *{$method} = sub {
            !!( $method_level <= (
              $_[0]->{log_level} // numeric_level('info')
            ) );
        };
    }

    What’s that sub­sys­tem” bit up there? That’s the term macOS uses for iden­ti­fy­ing process­es in logs. They’re usu­al­ly for­mat­ted in reverse DNS nota­tion (e.g., com.example.perl”). Once again, Howard Oakley has a great explain­er on the top­ic.

    Also, there’s some metapro­gram­ming going on there:

    • The first fore­ach loop cre­ates func­tions called trace, debug, and info. These func­tions call the cor­re­spond­ing FFI::Platypus-created func­tions. It uses the pri­vate vari­ants if the pri­vate attribute for the log adapter was set.
    • The sec­ond fore­ach loop cre­ates cre­ates func­tions called is_​trace, is_​debug, is_​info, etc., that return true if the adapter is catch­ing that lev­el of log message.

    Part 5: At long last, logging… mostly

    Once this is pack­aged in a Perl mod­ule, how do you use it? At least that part isn’t too hard:

    use Log::Any '$log', default_adapter => [
      'MacOS::OSLog', subsystem => 'com.phoenixtrap.perl',
    ];
    use English;
    use Carp qw(longmess);
    
    $log->info('Hello from Perl!');
    $log->infof('You are using Perl %s', $PERL_VERSION);
    
    $log->trace( longmess('tracing!') );
    $log->debug(     'debugging!'     );
    $log->info(      'informing!'     );
    $log->notice(    'noticing!'      );
    $log->warning(   'warning!'       );
    $log->error(     'erring!'        );
    $log->critical(  'critiquing!'    );
    $log->alert(     'alerting!'      );
    $log->emergency( 'emerging!'      );

    And then you can run this com­mand line to stream log mes­sages from the sub­sytem used above:

    % log stream --level debug \
      --predicate 'subsystem == "com.phoenixtrap.perl"

    What hap­pened to the trace and debug log mes­sages that were sup­posed to call os_log_debug(3)? According to macOS’ log(1) man­u­al page, you have to explic­it­ly allow debug­ging out­put for a giv­en subsystem:

    % sudo log config --mode "level:debug" \
      --subsystem com.phoenixtrap.perl

    Et voilà!

    Hmm, same lack of debug­ging messages.

    I’m still fig­ur­ing this out. Any clues? Drop me a line!

    UPDATE: This is now fixed thanks to some inspi­ra­tion from the source code of Log::Any::Adapter::Syslog. I’ve updat­ed the code on Codeberg; here is the diff.

    Bonus: Fancy output

    Thanks to Log::Any::Proxy, you also get sprintf for­mat­ting vari­ant functions:

    use English;
    $log->infof(
        'You are using Perl %s in %d',
        $PERL_VERSION, (localtime)[5] + 1900,
    );
    You are using Perl v5.40.2 in 2025

    If you out­put an object that over­loads string rep­re­sen­ta­tion, you get that string:

    use DateTime;
    $log->infof('It is now %s', DateTime->now);
    It is now 2025-08-10T20:16:50

    And you get single-​line Data::Dumper out­put of com­plex data struc­tures, plus replac­ing unde­fined val­ues with the string undef”:

    $log->info( {
        foo    => 'hello',
        bar    => 'world',
        colors => [ qw(
            red
            green
            blue
        ) ],
        null => undef,
    } );
    {bar => "world",colors => ["red","green","blue"],foo => "hello",null => undef}

    Conclusion: Build once, use everywhere

    The best tools aren’t always the ones you planned to build. They’re the ones that solve a prob­lem cleanly–and then solve five more you hadn’t thought of yet.

    What start­ed as a quick fix for Mastodon media mon­i­tor­ing became a reusable bridge between Perl and macOS’ Unified Log. Along the way, I got to explore Apple’s log­ging inter­nals, write an FFI-​respecting C wrap­per, and inte­grate clean­ly with Log::Any. The result­ing code is mod­u­lar, auditable, and–most importantly–maintainable.

    I did­n’t set out to write a log­ging adapter. But when you care about clean ops and repro­ducible infra­struc­ture, some­times the best tools are the ones you build your­self. And if they hap­pen to be over-​engineered for the task at hand? All the better–they’ll prob­a­bly out­live it.

    Try it out or contribute!

    The full adapter code is on Codeberg. If you’re log­ging from Perl on macOS, give it a spin. Contributions, bug reports, and real-​world feed­back are welcome–especially if you’re test­ing it in pro­duc­tion or on old­er macOS versions.

    I’ll do my best to stay com­pat­i­ble with past and future macOS and Perl releas­es. Keeping the code auditable and min­i­mal should help it stay use­ful with­out becom­ing a mov­ing target.

  • A brief note on Log4perl

    A brief note on Log4perl

    The Java world had an… inter­est­ing week­end when secu­ri­ty researchers revealed on December 9 a vul­ner­a­bil­i­ty in the pop­u­lar Apache Log4j 2 soft­ware library for record­ing and debug­ging events. Systems as diverse as Amazon Web Services, Apple iCloud, and the Minecraft video game could be exploit­ed to run arbi­trary code on a serv­er mere­ly by send­ing a specially-​crafted string of text. Information tech­nol­o­gy pro­fes­sion­als have been scram­bling ever since the ini­tial dis­clo­sure to patch, upgrade, recon­fig­ure, or oth­er­wise pro­tect affect­ed servers. It’s bad, and past unpatched vul­ner­a­bil­i­ties like this have been respon­si­ble for the expo­sure of mil­lions of people’s sen­si­tive data.

    Many Perl appli­ca­tions use the similarly-​named and ‑designed Log::Log4perl library, and the good news is that as far as I can tell the lat­ter doesn’t suf­fer from the type of vul­ner­a­bil­i­ty described above. This doesn’t mean poorly-​written or ‑con­fig­ured Perl-​based sys­tems are immune to all exploits, just this par­tic­u­lar one. You should be safe to con­tin­ue using Log4perl unless some­one has delib­er­ate­ly con­fig­ured it oth­er­wise, and in fact, my work uses it extensively.

    You might be sur­prised to read me sug­gest­ing a log­ging frame­work after writ­ing mul­ti­ple arti­cles espous­ing the Perl step debug­ger as an alter­na­tive. Log4perl devel­op­er Mike Schilli’s 2002 intro­duc­tion to the pack­age for Perl.com came down on the oppo­site side of the argu­ment. It can seem like one of those pro­gram­mer reli­gious issues like tabs vs. spaces, vim vs. Emacs, or Linux vs. Windows. (For the record, the cor­rect answers are spaces, BBEdit, and macOS. 😉)

    But in this case, you can and should have the best of both worlds—logging at dif­fer­ent lev­els to appro­pri­ate des­ti­na­tions while still drop­ping into the inter­ac­tive debug­ger when you need to do some­thing trick­i­er like exam­ine pro­gram state or tweak a data struc­ture on the fly. I use both tech­niques and only empha­size the advo­ca­cy of step debug­ging because it’s under­stood less.