By 10:25 AM, Iโd entered what Mystery Science Theater 3000 fans call โโDeep Hurting.โ The migraยญtion plan was solยญid. The backยญup disยญciยญpline was comยญpreยญhenยญsive. The exeยญcuยญtion? Chaos.
I run a conยญtainerยญized proยญducยญtion Mastodon instance on an 8 GB Mac mini. (Yes, I know what the cloud peoยญple say, and FYI itโs Cloudflare Tunneled for proยญtecยญtion.) My Docker Desktop installationโs half-โgig RAM footยญprint was eatยญing preยญcious resources. Colima promised the same Docker expeยญriยญence withยญout the GUI overยญhead. I budยญgetยญed a 1.5 hour migraยญtion plan for what shouldโve been a straightยญforยญward runยญtime swap.
Two and a half hours and sevยญen critยญiยญcal issues latยญer, Iโd disยญcovยญered that Docker Hardened Images and Colima donโt play niceยญly togethยญer. And that disยญcovยญery matยญters to anyยญone runยญning hardยญened conยญtainยญers in virยญtuยญalยญized environments.
The Plan (That Didnโt Survive Contact with Reality)
The stratยญeยญgy was textยญbook: mainยญteยญnance winยญdow approach, comยญpreยญhenยญsive backยญups (dataยญbase dumps, volยญume archives, conยญfigยญuยญraยญtion snapยญshots), explicยญit rollยญback proยญceยญdures. Iโd stop Docker Desktop, switch the Docker conยญtext to Colima, update one path in the Makefile I use to autoยญmate tasks, and restart serยญvices. Everything uses bind mounts, so data stays on the host file sysยญtem. What could go wrong?
Everything. Everything could go wrong.
Obsolete Makefile references
First backยญup try:
service "db" is not running
Waitโwhatโs db? I migratยญed from verยญsion 14 to verยญsion 17 of the PostgreSQL relaยญtionยญal dataยญbase sysยญtem weeks ago. Switched and even switched from the default PostgreSQL image to a Docker Hardened Image (DHI), even. My comยญpose files refยญerยญence db-pg17. But the Makefileโs backยญup tarยญgets? Still callยญing the old db serยญvice. The PostgreSQL migraยญtion docยญuยญmenยญtaยญtion lived in the README file that I keep. The Makefile lived inโฆ a difยญferยญent menยญtal conยญtext apparently.
Lesson: When you migrate infraยญstrucยญture comยญpoยญnents, grep for refยญerยญences everyยญwhere. Compose files, Makefiles, scripts, docยญuยญmenยญtaยญtion. โโItโs workยญingโ means โโitโs workยญing right now,โ not โโthe migraยญtion completed.โ
The empty postgres17/ directory
After resolvยญing the dataยญbase restore issues (weโll get there), conยญtainยญers startยญed sucยญcessยญfulยญly. Then I ran a restart test. PostgreSQL came up emptyโno data, no tables, fresh initialization.
% ls -la postgres17/
total 0
drwxr-xr-x@ 2 markandsharon staff 64 Jan 7 16:31 .
64 bytes. An empยญty direcยญtoยญry. That December PostgreSQL 14 โ 17 โโmigraยญtionโ? Created the direcยญtoยญry, nevยญer popยญuยญlatยญed it. PostgreSQL 14 data stayed in postgres14/. Docker Desktop mustโve been using cached or interยญnal storage.
Lesson: Donโt trust that migraยญtions sucยญceedยญed because serยญvices are healthy. Check the actuยญal data files. Persistence isnโt perยญsisยญtence if nothยญingโs persisting.
Wrong database target
After fixยญing the Makefile, serยญvices startยญedโฆ and instantยญly crash-looped:
PG::UndefinedTable: ERROR: relation "users" does not exist
PostgreSQL was healthy. The appliยญcaยญtion disยญagreed. Turns out Iโd restored the dump to the wrong database:
# What I did (wrong):
psql -U mastodon postgres < dump.sql
# What I should have done:
psql -U mastodon mastodon_production < dump.sql
The mastodon_production dataยญbase existedโit was just empยญty. All my data went into the postgres dataยญbase that nothยญing was readยญing. The psql command-โline client defaults to the dataยญbase matchยญing your userยญname or postgres if unspecยญiยญfied. Explicit is betยญter than implicยญit, espeยญcialยญly when youโre in a hurry.
Version-โspecific PGDATA paths
Once data landยญed in the right dataยญbase, I hit a new probยญlem: data didยญnโt perยญsist across restarts. The bind mount direcยญtoยญry stayed empยญty even though PostgreSQL was runยญning and acceptยญing writes.
It turns out that my PostgreSQL DHI uses version-โspecific paths:
# My bind mount:
- ./postgres17:/var/lib/postgresql/data
# Actual DHI PostgreSQL data directory:
# PGDATA=/var/lib/postgresql/17/data
The mount shadยญowed the wrong direcยญtoยญry. PostgreSQL wrote data to /var/lib/postgresql/17/data, which wasยญnโt mountยญed. Data lived in ephemerยญal conยญtainยญer storยญage. Restart? Data gone.
$ docker compose exec db-pg17 psql -U mastodon postgres -c "SHOW data_directory;"
data_directory
-----------------------------
/var/lib/postgresql/17/data
Lesson: Verify assumpยญtions. Every sinยญgle one. Check SHOW data_directory; immeยญdiยญateยญly after conยญtainยญer start. Test a restart before celยญeยญbratยญing success.
I corยญrectยญed the mount path to match DHIโs expectยญed locaยญtion. Thatโs when I found the real problem.
The DHI + Colima Incompatibility Discovery: VirtioFS bind mount ownership failures
After corยญrectยญing the mount path, PostgreSQL entered an immeยญdiยญate crash-loop:
FATAL: data directory "/var/lib/postgresql/17/data" has wrong ownership
HINT: The server must be started by the user that owns the data directory.
Inside the conยญtainยญer, the mountยญed direcยญtoยญry appeared owned by the root user (user ID 0). But PostgreSQL runs as the postgres user. Permission denied.
% docker compose run --rm --entrypoint sh db-pg17 -c "ls -ld /var/lib/postgresql/17/data"
drwxr-xr-x 2 0 0 4096 Jan 10 16:22 /var/lib/postgresql/17/data
# Owner: UID 0 (root), but PostgreSQL requires postgres user ownership
Colima uses the VirtioFS sysยญtem for file sharยญing. VirtioFS hanยญdles UID mapยญping difยญferยญentยญly than Docker Desktopโs virยญtuยญal machine (VM) impleยญmenยญtaยญtion. Bind mounts that work perยญfectยญly on Docker Desktop fail on Colima because the ownยญerยญship mapยญping doesยญnโt translate.
Fine. This is a known issue with Colima and some images. Iโll switch to a named volumeโDocker manยญages those interยญnalยญly, so host filesysยญtem perยญmisยญsions shouldยญnโt matter.
Named volยญumes still failed:
FATAL: data directory "/var/lib/postgresql/17/data" has wrong ownership
Wait. Named volยญumes are supยญposed to be isoยญlatยญed from host file sysยญtem issues. Theyโre manยญaged entireยญly by Docker. Fresh named volยญume, Docker creยญates it, Docker popยญuยญlates itโand it still shows wrong ownยญerยญship inside the DHI container.
# Fresh named volume:
% docker compose run --rm --entrypoint sh db-pg17 -c "ls -ld /var/lib/postgresql/17/data"
drwxr-xr-x 2 0 0 4096 Jan 10 16:22 /var/lib/postgresql/17/data
DHI PostgreSQLโs entryยญpoint has enviยญronยญmenยญtal assumpยญtions that Colimaโs VM doesยญnโt satยญisยญfy. The imageโs secuยญriยญty hardยญenยญing includes stricter ownยญerยญship valยญiยญdaยญtion. That valยญiยญdaยญtion doesยญnโt account for Colimaโs volยญume handling.
The pragmatic trade-off
So I had to make a decision:
- Debug DHI + Colima comยญpatยญiยญbilยญiยญty (unknown time investยญment, might be unsolvยญable), or
- Switch to the stanยญdard
postgres:17-alpineimage (known workยญing, immeยญdiยญate resolution)
Production sysยญtem. Already 1.5 hours into debugยญging. Swap the image:
# Before (DHI):
image: dhi.io/postgres:17-alpine3.22
volumes:
- postgres17-data:/var/lib/postgresql/17/data
# After (Standard):
image: postgres:17-alpine
volumes:
- postgres17-data:/var/lib/postgresql/data
PostgreSQL iniยญtialยญized sucยญcessยญfulยญly. Data perยญsistยญed across restarts. Services came up healthy.
The trade-โoff:
- โ Gained: Colima comยญpatยญiยญbilยญiยญty, reliยญable data perยญsisยญtence, onward progress
- โ Lost (temยญporarยญiยญly): DHI secuยญriยญty hardeningโdocumented for future investigation
Docker Hardened Images offer secuยญriยญty feaยญtures through stricter defaults and entryยญpoint valยญiยญdaยญtion. Those same strict requireยญments reduce the comยญpatยญiยญbilยญiยญty surยญface. When you introยญduce a difยญferยญent virยญtuยญalยญizaยญtion enviยญronยญment (Colimaโs VirtioFS instead of Docker Desktopโs VM), the hardยญenยญing becomes brittleness.
This isnโt DHIโs faultโitโs the expectยญed conยญseยญquence of defense-โin-โdepth. But if youโre migratยญing from Docker Desktop to Colima, test your image comยญpatยญiยญbilยญiยญty in isoยญlaยญtion first. This is cruยญcial if you are using Docker Hardened Images. Carry out these tests before migraยญtion day.
The Outcome
Migration comยญpletยญed at 11:30 AM. Zero data loss. All serยญvices healthy. Automation restored. RAM reclaimed (Docker Desktopโs overยญhead vs. Colimaโs negยญliยญgiยญble footprint).
The real outยญcome was discoveringโsystematically, through eliminationโthat DHI PostgreSQL and Colima are incomยญpatยญiยญble withยญout furยญther invesยญtiยญgaยญtion. Iโve docยญuยญmentยญed this as a known issue. Future work: test DHI with difยญferยญent volยญume strateยญgies, check whether newยญer DHI verยญsions resolve the issue, evalยญuยญate whether the secuยญriยญty delta matยญters for a single-โuser instance.
For now, Iโm runยญning stanยญdard postgres:17-alpine. The migraยญtion is sucยญcessยญful. The secuยญriยญty regresยญsion is docยญuยญmentยญed and schedยญuled for future invesยญtiยญgaยญtion. Forward progress beats perfectionism.
Key Takeaways
Backups are your safeยญty netโuse them. I restored the dataยญbase once durยญing this migraยญtion. That restore took 30 secยญonds because Iโd verยญiยญfied the backยญup existยญed and was recent.
Systematic debugยญging beats panยญic every time. Bind mounts failed โ tried named volยญumes โ still failed โ isoยญlatยญed to image-โspecific behavยญior. That proยญgresยญsion ruled out host file sysยญtem issues and pointยญed directยญly at image compatibility.
Pragmatic trade-โoffs beat perยญfecยญtionยญism. I couldโve spent hours debugยญging DHI comยญpatยญiยญbilยญiยญty. Instead, I docยญuยญmentยญed the incomยญpatยญiยญbilยญiยญty, switched to stanยญdard images, and moved on. The secuยญriยญty regresยญsion is tracked. The proยญducยญtion sysยญtem is running.
Document failยญures honยญestยญly; theyโre learnยญing opporยญtuยญniยญties. This post exists because the migraยญtion didยญnโt go smoothยญly. The DHI + Colima incomยญpatยญiยญbilยญiยญty is now docยญuยญmentยญed for anyยญone else hitยญting the same issue. Thatโs more valuยญable than a โโhereโs how I moved from X to Yโ sucยญcess story.
| Migration duraยญtion | 2.5 hours actuยญal vs. 1.5 hours planned |
| Issues encounยญtered | 7 critยญiยญcal |
| Data loss | 0 bytes |
| Services | All healthy |
| Memory reclaimed | ~500 MB |
| Novel disยญcovยญerยญies | 1 (DHI + Colima incompatibility) |
| Trade-โoffs documented | 1 (secuยญriยญty hardยญenยญing vs. compatibility |
Running proยญducยญtion infraยญstrucยญture on an 8 GB Mac mini teachยญes you to valยญue both resources and reliยญaยญbilยญiยญty. Colima delivยญers on the resources. This migraยญtion delivยญered on the reliยญaยญbilยญiยญtyโฆ eventually.






