Inbox Firewall

I was looking for open-ended research problems to hand to Council, to see how well two agents explore when nobody knows the answer.

Council is a shared workspace where Claude Code and Codex work the same repo with one shared context and hand work to each other. Back then that meant Opus 5 and Codex 5.6 Sol.

While digging for problems I found a video on how capable small models have become. The one that stuck was Cactus’s Needle, about 26 million parameters. I gave the agents one brief, find it a real job, and watched them research and argue.

They agreed on an inbox firewall. People now let agents run their inboxes, and an inbox is text written by strangers. One prompt injection and the agent archives, forwards or replies for the attacker. So put a model in front that’s too small to be clever but smart enough to tell a threat from an email. They built the proof of concept and the harness to measure it.

The model never acts. It emits one line of a closed DSL: IGNORE, MARK_URGENT, ARCHIVE, DRAFT_REPLY or ESCALATE. Every argument is an enum, so off-vocabulary output is rejected by a hand-written parser, never eval. Each action has a tier: ignore just runs, archive needs high confidence, a draft is never sent.

1,028 labelled messages, splits disjoint, the hardest and most hostile held back for the final test.

• 88.1%action accuracy, regex and keywords only
• 11unsafe executions

The first surprise was the rules baseline: no learning, 88.1% right, three quarters handled automatically. The second was the 11 messages it acted on that should have gone to a human. Six prompt injections, five phishing, and every one a perfectly valid action, so the validator had nothing to reject.

Types catch malformed output. They can’t catch a well-formed bad decision.

• 0.245 → 0calibration error, held-out clean data
• 11 → 11unsafe executions, unmoved

The obvious fix was to calibrate confidence and gate on it. It worked perfectly and changed nothing. On clean data a confident router is nearly always right, so calibration maps almost every score to ~1.0 and the gate waves confident injections straight through.

Calibration fits the distribution you have. An attacker brings a different one.

• 11 → 7unsafe executions
• 6 → 2of them prompt injections

What moved the number was treating the email body as untrusted input and flagging text that addresses the model rather than the reader. Injections fell from 6 to 2 for a small loss in coverage. What’s left is phishing, which is built to read like a normal email.

Needle itself is still on the bench. Its fine-tuner wants JAX and more than my 2 GB GPU, so the exporter is built and the run is deferred. The conclusion doesn’t depend on it.

Confidence is a coverage instrument, not a safety boundary.

The model decides how much you can automate. Safety lives outside it, in layers that don’t care how sure it feels: typed outputs, action tiers, content-trust checks, and an escalate path for anything they can’t prove safe.