Self-Hosting / Automation

The Self-Hosted App Update Gap That OS-Level Patching Doesn't Cover

Unattended OS-level security updates feel like they cover "keeping things patched" as a category. They don't cover the actual self-hosted applications running on top of that OS — and one of those apps falling one version behind is exactly what silently broke a real, in-daily-use client one day, with no warning from anything monitoring the server itself.

What actually happened

A self-hosted password manager's browser extensions (Firefox and Chrome) silently stopped populating vault data one day — no error dialog, just an empty-feeling extension — while the web vault and mobile app on the exact same account kept working perfectly fine. The browser extensions auto-update themselves through their respective browser stores, entirely outside anyone's control, and had moved to a client version requiring a newer minimum server version than what was actually running. The fix release that closed that gap had shipped the day before the extensions updated — the self-hosted server was exactly one version behind, and nothing had ever updated it, because nothing was watching for that.

The gap this exposed

Unattended OS-level security updates and self-hosted application updates are two completely separate categories, and having one solved doesn't imply anything about the other. Every container's underlying Linux packages were covered by a real, working unattended-upgrades policy. Not one of the actual self-hosted applications running inside those containers — the software people were actually using every day — had any update automation at all. They only ever got updated manually, on no particular schedule, which is exactly the condition that lets a compatibility-breaking version gap like this go unnoticed until a client somewhere silently breaks.

This is easy to miss precisely because "we have automated patching" sounds like a solved problem in general terms — it's worth explicitly asking whether that coverage means the OS, the applications, or both, since the answer is often "just the OS" without anyone having decided that on purpose.

The fix: backup-gated auto-updates, staggered, per service

Rather than reaching for a generic "auto-update everything" cron job, the pattern that's held up well:

A real backup immediately before every update attempt — not a substitute for testing, but a guaranteed rollback path if an update itself introduces a regression, which is a real risk you're explicitly trading for closing the staleness gap.
One systemd service/timer pair per application, each running its own update-then-verify script, rather than one big script trying to handle every app's different update mechanism.
Staggered timers across the early-morning hours instead of everything firing at once — this spreads out the backup storage load and, more importantly, limits the blast radius if one specific update goes wrong: you're debugging one service's timer window, not untangling six simultaneous updates that happened to run in the same few minutes.

Worth deciding explicitly, not by default

Once this exists for one application, it's worth explicitly listing every other self-hosted app in your environment and deciding, one by one, whether it belongs in the same automation — rather than assuming "we have auto-updates now" covers anything beyond whatever specifically triggered building it. The whole point of this gap is that it's invisible until something breaks; closing it for one app doesn't close it for the others sitting right next to it, running the same kind of unwatched, manually-updated (or never-updated) software stack.