Putting a Local LLM to Work on Your Fleet: Notify Only, and Never Let History Pose as Fact

A monitoring system that emails you every problem eventually trains you to ignore email. A local model running on hardware you already own can read those problems, group them, explain them in plain language and send one message a day. That is a real improvement. The moment you let it act on what it read, you have built something else entirely.

Draw the line at notify-only, in writing

Decide this before you write any code, because it is the decision every later shortcut will push against. The pipeline reads state and produces text. It does not reboot, restart, patch or remediate. The last line of the digest can say "this host needs a reboot approved", and a human does the approving.

This is not superstition about AI. It is the same reason you do not give a cron job root on every host: an automated actor that is wrong at 3am is much more expensive than one that is merely unread at 3am. A model that hallucinates a recommendation costs you nothing if the only thing it can do is print it.

Run it where the GPU already is

Put the pipeline on the always-on machine that already serves the model, not on your workstation. A daily job on a desktop is a daily job that silently stops running the first time you shut the desktop down, and nothing tells you.

The failure worth designing around: history stated as current fact

Give the model your own documentation as context and it gets much better at recognising patterns. It also gains a large corpus of statements that used to be true.

In one of the first real runs, the digest correctly identified a live CRIT about memory allocation on the hypervisor. Then it explained the cause by stating the host had 64GB of RAM. That was accurate when the wiki page was written, and wrong by the time it was retrieved, because the machine had been upgraded to 128GB weeks earlier. One true finding, one confidently false fact, in the same paragraph.

The fix is in the prompt, and it needs to be explicit about which source wins:

CURRENT FINDINGS is the only source for specific numbers, versions and states.
REFERENCE MATERIAL is for judging "is this a known pattern here" only.
Never state a fact from REFERENCE MATERIAL that CURRENT FINDINGS does not confirm.

Two sources with different authority, and you have to say which is which. Otherwise the model treats a three-month-old incident write-up as equal evidence to a reading taken thirty seconds ago.

Deduplicate, or the digest becomes the noise

A daily job that reports every currently-failing thing reports the same things every day forever. Keep a small state file and treat categories differently:

• Things needing action stay in every digest while still active. A pending reboot should keep asking.
• Findings are reported the first time only. A vulnerability you already know about does not need restating daily.
• Drop cleared items from state so a recurrence weeks later counts as new. This matters more than it sounds. Something that comes back is a different and more interesting event than something that never left.
• Send nothing when nothing is new. An empty day should produce an empty inbox, which is what makes the non-empty days worth opening.

Batch by section, not by finding

The obvious design sends one prompt per problem. The first run has a hundred-item backlog and takes forever.

Send one prompt per category instead, with the findings listed inside it. Fewer, larger calls, and the model gets to see related problems together, which is exactly when it produces something more useful than a reworded alert.

Give it read-only credentials, and prove they are

The pipeline needs a clone of your documentation and an API key for your monitoring. Both should be scoped so that a mistake in the script cannot write anything.

For a git clone, a deploy key restricted server-side is the strong version, because the restriction lives on the server rather than in the key's own metadata:

# in the git server's authorized_keys, on the key's line
command="git-upload-pack /opt/wiki-repo.git",no-port-forwarding,no-pty ssh-ed25519 AAAA...

Then actually attempt a push with that key and confirm it fails. Read-only that has never been tested is read-only by assumption.

Where a local model is and is not the right tool

Worth being honest about, because the answer is not "more VRAM".

Local models are good at the bounded, low-stakes work: summarising, drafting, reformatting, explaining a known error, answering a lookup. That is genuinely most of what a digest needs, and it costs you nothing per call.

They are meaningfully weaker at agentic judgment, meaning recognising a destructive command, handling an ambiguous instruction, or coordinating a change across several hosts safely. That gap is about training and alignment rather than parameter count, so a bigger card lets you run a bigger model without closing the part that actually matters here. Which is another way of arriving at notify-only.

Check the wrapper still exists after any migration

A small one that cost real time. A helper script in ~/.local/bin did not survive a move to a new machine, so the whole delegation mechanism was quietly unavailable for weeks. Nothing errored, because nothing called it. If a piece of your automation is a script on a PATH, it belongs in the repository with everything else, and its absence should be something a check notices.

When this isn't your problem

• You have few enough alerts to read them. Then a digest adds a layer and a failure mode for no benefit. Fix the alert volume instead.
• You have no GPU. Small models run on CPU, slowly. For a once-daily job that is often acceptable, and you should measure it before ruling it out.
• You want it to fix things. That is a different project with a different risk profile, and the fixes worth automating are usually deterministic enough not to need a model at all. See a guest watchdog that restarts stopped VMs.