Self-Hosting / Databases

Never Write Directly to an App-Encrypted Database Column

Fixing a self-hosted app's broken credential by updating it straight in the database feels like a perfectly reasonable shortcut — until the column you just wrote to turns out to be encrypted at the application layer, not just at rest. The write succeeds. The breakage shows up somewhere else entirely, later, looking nothing like what actually caused it.

The setup

A self-hosted app had a stale integration token stored in its own Postgres database. The real fix (a fresh, valid token, confirmed working via a direct API call) was correct. Writing it into the app's database via a plain UPDATE ... SET token_value = '<new value>' is where it went wrong — that column turned out to be encrypted at the application layer (a common pattern: SQLAlchemy-Utils' EncryptedType and equivalents in other frameworks), not just protected by disk-level encryption. The app always expects to decrypt whatever's sitting in that column using its own installation-specific key. A raw plaintext value written directly in breaks that expectation completely — but silently, at write time. Nothing errors on the UPDATE itself.

Where the actual failure shows up

The failure doesn't happen on write — it happens on the next read, and it can look like a completely unrelated feature broke or lost data. In this case, the app's own backend threw a decryption error every time it tried to read that row back. In the UI, this manifested as an entire configuration section going empty — genuinely alarming, indistinguishable at a glance from the underlying data having been deleted. It hadn't been. The row was still there; it had just become permanently unreadable by the one thing that knew how to decrypt it.

The generalizable lesson: "the page shows nothing" is not proof that data was lost. Before assuming a deletion or a data-loss event, check the application's own backend logs for a decrypt or deserialization error on read. A silently-broken decrypt looks identical to missing data from the outside, but the fix (and the actual cause) are completely different.

There's usually no fixing it after the fact

Without the app's own encryption key — typically generated once per installation and not something you can reasonably recover after the fact (checked environment variables, config files, and install scripts; none of them tend to expose it, by design) — the corrupted value generally can't be repaired in place. Two real options once this has happened:

Delete the corrupted row (confirm first that any foreign key relationships handle the deletion gracefully — a nullable/ON DELETE SET NULL relationship makes this safe) and re-enter the entire affected configuration fresh through the app's own UI, not SQL.
Better, if you have it: restore from a backup taken before the mistake. This brings back the intact, correctly-encrypted row in one shot — only the field you actually meant to fix needs re-entering afterward, through the app itself.

A compounding gotcha worth watching for after a restore

A self-hosted app's own error message about licensing, auth, or a missing feature is not proof of its stated cause. After restoring from backup in a real case like this, the app's admin UI started reporting a "no license" error — completely unrelated to anything above. The real cause: the restored disk was full, which crashed the app's own database process, which the app's frontend then surfaced as a licensing failure because that happened to be the error path it hit first. Freeing real disk space (in this case, old dangling container images left over from prior version pulls, safe to prune since they weren't referenced by anything running) fixed it — nothing about licensing was ever actually wrong.

Before trusting what a self-hosted app claims is wrong with itself — especially anything licensing- or auth-shaped — check basic resource health (disk space, whether its own database process is actually up) first. Plenty of apps surface a generic or misleading error message for what's really just a resource exhaustion problem underneath.

The general rule

Before writing directly to any column via raw SQL on a self-hosted app's database, check whether that specific field is encrypted at the application layer — this is common for API tokens, passwords, and other credential-shaped fields specifically. A quick way to check: look for framework-specific markers (EncryptedType and similar are common in Python/Ruby ORMs) in the app's source or in error tracebacks if something's already gone wrong. If a field is app-encrypted, go through the application's own UI or API to change it, every time — raw SQL stays safe for plain, non-encrypted columns on the same table, but treat anything credential-shaped as guilty until proven otherwise.