There is a pattern in agent design that looks like free safety. The agent writes a reply. Before it goes out, a second model reads it and asks: is this accurate, on-voice, safe? If not, it rewrites it. A whole category of guardrail products now sells some version of this, running a judge model inline on every response.
We built one into the first version of our new text engine. Three days later we deleted it, along with about 1,100 net lines of code, and we now treat “add a model after the reply” as a design smell. This is why.
What we were worried about
alfred_ acts on your email and calendar by text. Some of those actions wait for your yes: an email to someone outside your accounts is drafted, shown to you, and sent only when you agree.
The failure we feared was a reply like “Done, I sent it to Dana” when the email was still waiting for your yes. That is worse than a slow reply or an awkward one. It tells you something happened that did not.
So the first version of the engine had a voice step. A small, fast model checked each reply. If it found a problem, a larger model rewrote the reply.
What it actually did
We ran it for three days of internal testing, on our own accounts, before any customer was on the new engine.
- Across 184 replies, the checker ran 146 times.
- It sent 96 of those replies (52%) to a full rewrite.
- Each rewrite meant up to two extra model calls on the path between your text and our answer.
Half of all replies being rewritten is not a safety net. It is a second author. And the second author had its own habits. It trimmed replies to a sentence budget, which once cut a finished weather answer down to a fragment. It smoothed wording that was already fine. Every rewrite was another chance to introduce the exact kind of error the step existed to catch, with nothing after it to catch that.
The real cause was upstream
When we looked at the replies the checker was right about, they had something in common. The model had not invented anything. It had been told inconsistent things.
A staged email (drafted, shown to you, waiting for your yes) reached the model under several different labels depending on which part of the system described it. One of them was “sending.” Our instructions said never claim an email is sent until it is. The data said “sending.” The model split the difference, sometimes badly.
A checker downstream cannot fix that. It can only argue with the model after the fact, at the cost of time and a second chance to be wrong.
What we do instead
One label per state, written by code. A staged email now reaches the model as exactly one thing: waiting for the user’s yes, executed false, NOT SENT. A background job that has started is described as started, not done. These labels come from the ledger of what actually ran, so they cannot drift from reality.
Code writes the facts in the reply. Where a reply needs to state an outcome (the receipt for a sent email, a list footer, a confirmation card), code writes that line from the ledger. The model writes the conversation around it.
Code-written lines still sound human. A fixed sentence repeated every time reads like a robot. Each code-written line has a small pool of natural wordings, and we pick one by hashing the turn’s id. No randomness, no model call, and the same turn always renders the same way, which keeps tests stable.
Measure, do not rewrite. We still lint replies for the patterns we care about and record the rate. If a pattern shows up, the fix goes into the data, the instructions or the code that writes facts. It does not go into a model that edits the reply.
What it bought us
Removing the voice step deleted both extra model calls from the reply path. Replies come back sooner, they read the way the model wrote them, and the status words in them come from the same ledger that decides what actually ran.
The rule we keep now is short: no model checks or rewrites a reply after it is written. If a reply is wrong, something before it was wrong. Find that and fix it there.
When a checker does make sense
This is not an argument against using models to evaluate other models. We do that constantly, offline: grading transcripts, comparing prompt changes, scoring a batch of turns before a change ships. That is a measurement. It runs off the user’s path and changes nothing the user sees.
The problem is putting the checker on the live path as an editor. There it adds latency to every reply, rewrites good answers along with bad ones, and hides the real cause of the bad ones behind a patch that seems to work.
And for an agent, a checker on the reply is in the wrong place anyway. By the time the reply exists, the email it describes may already have gone out. The place to stop a wrong action is before it runs, in code that decides what executes. We wrote about that gate in What Counts as “Yes” When Your Agent Lives in Text Messages.