Insecure Defaults, Part 2: Excessive Agency

My AI tagged a production release on its own read of five words. Here’s the deterministic cage I built to cover the footgun.

“Let’s land #118.”

That’s all I said. Five words about finishing a feature. My agent read them, decided the work was done, computed the next version tag, and pushed it triggering the production deploy pipeline. Mid-feature. Unverified.

I never asked it to ship.

What the heck?

It wasn’t a one-off

Once I started watching for it, the pattern was everywhere. The agent tries to push directly to main; branch protection bounces it; the agent shrugs and opens a PR like it planned that all along. Then it volunteers for more: “I’ll watch the PR and merge it once checks pass.” Helpful little thing.

Uh… no.

Was any of this malicious? No. Was it genuinely useful? Sometimes. Was it authorized? Not even a little.

The OWASP Top 10 for LLM Applications has a name for this: Excessive Agency. I have a less formal one: an eager intern with your keys. The agent isn’t scheming, it’s completion-seeking. It was trained to finish jobs, and “finished” in its head includes shipped. Out of the box, an agent’s permission to act is bounded by its tool access, not its judgment. If the credential in its hands can push a tag, then the only thing between you and an unreviewed Friday deploy is a sentence in a prompt asking it to be careful.

And guess what? The more capable the model, the more likely this behavior is to occur. Fable thinks meat puppet in the middle is definitely not useful. Sonnet is more likely to ask permission.

Silly humans… sit back, I got this.

And lest you start thinking the fix is better prompts, it’s not. You need hard guardrails to keep the AI minions in check or they’ll go wild the second the find a loophole in your instructions.

What I built instead

I started by asking it what “UAT” stands for. Correctly it identified that as “User Acceptance Testing.” I then asked, so if pushing to UAT is for the user to accept what was built, how does that fit with you automatically tagging a release for PROD?

Derp. Yea… it figured out that was probably not a secure SDLC. On to the fix.

1. Server-side protections: necessary, and not enough. Branch protection is what bounced those pushes to main, cool, that’s table stakes. But the tagging incident revealed something the docs don’t put on a poster: PR review protection does not gate tag pushes. And GitHub’s deployment-approval machinery assumes a second human exists, it won’t let you approve your own deploy, which for a solo founder means it won’t let anyone approve anything. Server-side controls end at the branch. The shipping lane was wide open.

2. A deterministic gate in front of the shell. So I wrote a hook that runs before every command the agent executes. Plain code, not a prompt, not a vibe, not a probabilistic maybe. It blocks the whole ship surface: tag pushes however they’re wrapped, release creation by CLI or API, workflow dispatches by name or by numeric id, deployment self-approvals. The first version tried allowlisting the safe invokers and lost; wrap a blocked command in timeout, sudo, expect, xargs, or a plain newline and a name-keyed matcher waves it through. The wrapper zoo is unbounded. The rewrite flips it: commands get split into chunks, and any chunk that touches release machinery passes only if it starts with a known read-only verb. Everything else blocks. Fail-closed and if the gate itself hits a bug, it degrades to a dumb substring block rather than failing open. A broken gate that over-blocks beats a broken gate that fires the footgun.

3. Human-only tools for the human step. Release management is kind of important; preflight checks, computing the version tag, watching the pipeline, smoke tests. The machinations are exactly what agents are for, so I automated it… into ceremony scripts that refuse to proceed unless an approval sentinel exists: a file only I create, valid for thirty minutes, single use, consumed and archived so there’s an attestation trail. The agent can’t forge it, the same gate blocks the agent’s file-editing tools from ever writing into that directory. The robot does the chores. The human does the “yes.”

The part you can take with you

You don’t need my stack to steal the pattern:

1. Separate “can act” from “may ship.” Agents get the repo; the ship surface (tags, releases, dispatches, deploy approvals) is deny-by-default.
2. Put approvals in artifacts agents can’t forge. Files with TTLs and single use, checked by code. Never instructions in a prompt.
3. Automate the toil, keep the yes. Human-only tools should make the safe path the easy path, or the humans will route around them too.

An agent’s job ends at the PR.

“Ship it” is for you to do, not your AI intern. Keeping it that way will save you headaches later.