termalLocal models are having a moment. Ollama on a workstation, an open-weight model behind your own endpoint, a self-hosted coding agent — you run them for good reasons: privacy, cost, no rate limits, works on a plane. And once a model is already handling your prompts, the obvious next step is to point it at your infrastructure: let it tail a log, restart a service, run the migration.
Here’s the trap that rides along with it: “it’s local, so it’s safe” is not true. Running the model on your own hardware keeps your prompts off someone else’s servers. It does nothing about what happens when that model gets a shell.
Why “local” doesn’t make server access safe
Two things stay exactly as dangerous whether the model runs in a datacenter or on the box under your desk.
The key is still a bearer credential. Drop id_ed25519 into the agent’s environment so it can run ssh itself, and whoever — or whatever — holds those bytes is you, on every host that trusts the key. Running the model locally doesn’t change that; it just moves the copy to a machine you happen to own. And you still can’t take it back: once the key has passed through the model’s context — an env var, a mounted file, a tool call, a transcript — you can no longer prove it didn’t end up somewhere it shouldn’t. The only honest answer to “did the model see my key?” is to rotate it on every host.
A local model still gets confused. Prompt injection doesn’t need a cloud model. A poisoned log line, a malicious filename, a booby-trapped tool result, an over-eager plan — a local model acts on all of them just as readily as a hosted one. If anything the risk is worse, because people trust local setups more, wire them up with fewer guardrails, and hand them broader access, precisely because “it’s on my machine.” That’s the wrong instinct: the model’s judgement isn’t better because the weights are local.
So the goal is the same as for any agent — let the model do the work without ever holding the credential, and make every action scoped, visible and reversible.
Three ways to do it badly
Paste the raw key. The direct route, and the worst: no scope, no expiry, and the irreversibility above. The key that deploys to staging can usually also reach the production database two hops over.
Mint a long-lived token. A deploy token or “automation” credential that expires never. It feels tidier than a key, but it’s the same bearer problem with extra steps. If revoking something means remembering it exists, it’ll outlive the experiment that created it.
Give it its own root login and walk away. The right instinct — a separate identity — done the wrong way. Drop a public key into authorized_keys for a bot user and you’ve built access with no audit trail: nothing distinguishes the model’s commands from anyone else’s, and when something breaks at 2 a.m. you’re reconstructing its session from bash history and vibes.
The common thread: the model holds a standing credential, and observability is an afterthought.
The pattern that works: a custodian, not a copy
Flip it. The model should never hold the credential at all.
- The model asks; a custodian signs. A broker holds the keys (or mints short-lived certificates) and authenticates on the model’s behalf. Compromise the model’s context and you get the ability to request actions through the broker — not the ability to impersonate you from anywhere.
- Scope is explicit. The model reaches only the hosts you’ve listed. A new host is a new decision, not a default.
- Commands are policed. Per host, decide whether the model gets a full shell, an allowlist of commands, or nothing — so “restart nginx” is allowed on the web tier and “anything at all” never is.
- A human can watch — live. Not just in the post-mortem, and with the record distinguishing “the model did this” from “I did this.”
- Revocation is a toggle, not a rotation. Because nothing was ever shared, turning access off costs nothing.
None of this is exotic — it’s roughly how certificate-based SSH already works on serious infra teams. The catch has always been that wiring it up yourself is a project, so people skip to one of the bad options above.
Wiring it to a local model
MCP — the Model Context Protocol — is the clean way to do this, and it’s client-agnostic: any MCP-capable client can drive it, so it doesn’t matter whether your model is Claude, an open-weight model behind Ollama, or something you host yourself.
- Run your local model through an MCP-capable client (a coding agent or chat client that speaks MCP).
- Point it at an MCP server that fronts SSH — one that authenticates for the model rather than handing it a key.
- Turn on only the hosts you want, and set each host’s command policy (full / allowlist / blocked).
- Keep the session in view and let it write to an audit log, so every command the model runs is attributable and replayable.
The model gets tools — open a session, run a command, read or write a file — and the custodian does the authenticating. Your key never enters the model’s environment, on your machine or anywhere else.
How Termalin does it
Termalin is an SSH client with a built-in MCP server, so this pattern is the default rather than a project.
- On your machine, register the bundled local server with your agent (
claude mcp add termalin -- <path>/termalin-mcp, or the equivalent for any MCP client). The model reaches only the hosts you enable — agent access is off by default — and authentication goes through Termalin’s key custodian: you unlock your keys once, Termalin signs on the model’s behalf, and no key file is ever there for the model to read. - Per host, you set the policy — full access, an allowlist of commands, or blocked — so a local model that gets confused can’t run something you never authorized on that box.
- You watch it happen. Agent sessions run as live terminal tabs; the watch grid mirrors them, and the tiles a model is driving glow. Every command is marked in the session recording — output only, never your keystrokes — and written to an audit log with the device and IP it came from.
- No app running? Point the model at Termalin’s hosted MCP endpoint with an API key that’s scoped to specific servers, carries an expiry, is command-policed, and authenticates each run with a short-lived certificate — so even a leaked key can’t reach an unlisted host or run outside its allowed commands.
Either way, the property you wanted holds: your local model can operate your servers, and it has never seen a key.
Start with one boring host
Don’t begin with production. Enroll a low-stakes box — a staging server, a toy VPS — set it to an allowlist, and give the model a real chore: tail the log until the error shows, fix the config, restart the service. Keep the grid open while it works. What you learn in the first hour — how it behaves, where it hesitates, what it does with ambiguity — tells you whether the second host gets enrolled, and with how much rope.
That’s the quiet payoff of the custodian model: you expand one host at a time, because no step you take is one you can’t take back — no matter where the model is running.
Termalin is a free, cross-platform SSH client with a built-in MCP server, a key custodian and per-host agent policy — download it, or read how it handles keys safely.