There's a New alfred_ in Town
How we rebuilt the engine behind it

On October 6, one of our heaviest users sent our assistant 87 texts in under two and a half hours. It held or refused 23 of the email actions it proposed. Twice, it acted on the wrong email. Both times the user had said yes.

The yes was real. It had simply attached to the wrong thing: an older action the assistant had been holding, not the one on the screen.

That session did not expose a weak model or a bad prompt. It exposed an architecture in which the model held authority it should not have had. This post is about the engine we built to take that authority away, which we call V8.

An agent that kept getting patched

The engine before V8 was a good engine that kept getting patched: one agent loop over about 170 tools, with a cached prompt. Each incident added a guard. By October the guards were fighting each other.

Tracing that session gave us four causes. None of them was a bug in the usual sense.

• Two judges of “yes”, and no contract between them. The prompt told the model to read approval generously, and a long server word list decided too. “Yes skip” counted as yes. “Delete” counted as nothing.
• A yes attached to “some pending card”, not to one card.
• Six stacked refusal checks, one per past incident, with overlapping conditions. In one sample, 42 of 321 email delete and archive calls ended as refused confirms.
• Preferences written down but not enforced. Of 118 standing instructions we checked, code honoured 52. Of the 36 that existed only as prompt text, it honoured none.

We had first tried to make the old engine faster. A cheap first pass in front of it answered 13 to 21% of turns and added 2 to 4 seconds to everything else. Speed on a patched engine does not fix a design problem.

Every one of these was a place where the model had been given authority that belonged in code.

Judgment, authority and truth

In most agents, the model decides what to do and then does it. V8 splits those apart into three planes.

• Judgment belongs to the model. It reads, plans, writes and proposes any tool call it likes. It cannot execute a call or report a result it was not handed.
• Authority belongs to code: one pure gate, a consent judge that can only return one of five labels, a floor of actions that ask, and a taint bit for text a stranger wrote.
• Truth belongs to a ledger: an intent row before every effect and a receipt after it. The model learns what happened only from receipts.

Around those planes, the runtime is deliberately plain. Each text becomes one job row. A worker takes a chat lease, so a chat does not run two turns at once, and loads the whole context in one database call. The model loop runs for at most 8 model calls or 100 seconds, and every call inside it goes through the gate.

Consent is a classification, not a word list

The hard part of an assistant that acts is not writing the email. It is knowing that “ok send it to her” means send, and that “yes skip” does not.

We judge consent with a small typed-decision model. It reads three things: your newest text, up to four of your earlier texts, and code-built labels of the open offers. It does not see assistant text, tool results or email bodies, so nothing a stranger wrote can argue it into a yes. What comes back is a type, not prose:

// What the judge returns. Not prose: a label and a probability.
type ConsentLabel = "confirm" | "decline" | "modify" | "unrelated" | "unclear";
interface Judgment { label: ConsentLabel; p: number }

The thresholds came from data. In calibration, every confirm at or above 0.9 was right: 70 of 70, over 247 cases. The one unsafe confirm that remained sat around 0.72. So code acts on a confirm at 0.9, and at 0.8 when the reply answers the single offer our last message asked about. In production that judge takes about 130 ms at the median and costs about $0.00005 a call.

When the judge is not sure, code does not guess. The reply goes back to the agent, which reads it and can either ask or call a confirm_offer tool. That tool is held to the same bind checks below, and it cannot confirm an offer in the run that staged it. A model-supplied confirmed: true is refused outright.

We got here by trying the obvious thing first. A small general-purpose model as the judge confirmed “wait no” and “keep it” on a delete, and “thanks” on a send: 18 false confirms on hand-labelled sets, all in testing. A typed model with a calibrated threshold turns consent from a sentence in a prompt into a number you can audit.

Binding a yes to a stored offer

Judging a yes is half the problem. The October 6 failures were binding failures: a real yes attached to the wrong action. In V8, a card that asks you is a row, and a yes consumes it with a conditional update.

-- A yes consumes a stored offer, or nothing at all.
update offers
   set consumed_at = now()
 where id         = :offer_id
   and owner_id   = :user_id
   and expires_at > now()
   and args_hash  = :stored_args_hash  -- what you saw is what runs
   and staged_run <> :this_run         -- no confirming in the staging run
returning args;

What runs is the stored arguments, not arguments the model regenerated afterwards. We learned that one the hard way: three “send” texts once staged three offers for one email, because the confirm hashed freshly generated arguments. Every test had used “yes”. Now we test with “send”, because real approvals look like “ok send it to her”, and that sentence also names a target. With several offers open, code picks: a bare yes answers the newest one, or the card your reply is threaded onto.

One gate for every effect

Everything ends in one function. decide(op, ctx) is pure: no I/O and no model, so it can be tested exhaustively, and there is one place to read to know what can happen.

export function decide(op: Op, ctx: RunContext): Verdict {
  // A "confirmed" flag from the model is refused.
  if ("confirmed" in op.args) return refuse("first_call_confirm");
  // No intent row, no effect.
  if (!ctx.ledger.canWrite(op)) return refuse(LEDGER_DOWN);
  if (op.kind === "read") return allow("ok");
  if (ctx.yes?.boundTo(op)) return allow("judged_confirm");
  // After a stranger's text, anything that leaves your account asks.
  if (ctx.tainted && sinkOf(op) !== "local") return ask("tainted_outward");
  // The floor: outside sends, removes and unknown tools ask.
  if (isOutsideSend(op)) return ask("external_send");
  if (isRemove(op) || !known(op)) return ask(FLOOR);
  // Reversible, and your own data.
  return allow("ok");
}

The real function runs more checks than that, always in the same order, and every verdict carries a reason from a closed list, so every row in the ledger says exactly why.

Outside sends, removes and unknown tools ask, unless you have told alfred_ that a specific person is fine to send to directly. A reversible change to your own data, such as a task, a note, a label or an archive, runs on your word, with the provider’s history as the undo. And because there is one gate, code that refuses or re-asks on top of its verdict is a design violation, and we review for it.

One input to the gate deserves a sentence. Email, invites, files and web pages are text a stranger could have written, so the run carries a taint bit that latches the first time it reads any of them. After that, outward and standing writes ask. An instruction hidden in an email can be read and quoted, but it cannot become a send on its own.

The ledger is the source of truth

Every gate decision writes to the ledger: ids, hashes, status, counts, timings and cost, not content. Every effect writes an intent row before it runs. A duplicate key returns the earlier result instead of acting twice, and if the ledger cannot be written, the operation is refused. When we ask whether something happened, we read the ledger, not the reply.

The model learns what happened the same way. It sees receipts that code writes from the ledger, not raw provider output:

{ "tool": "reply_to_email",
  "status": "waiting_for_user_yes",
  "executed": false,
  "note": "NOT SENT" }

A queued job is “started”, not “done”, because the model once wrote “Done” for a job that then failed. Status lines, short confirmations and proactive texts are written by code too, with the wording picked from a small pool by hashing the run id: varied, natural, identical on replay, and free of model calls. And no model checks or rewrites a reply after it is written. We tried that once.

Down to the last bubble

The same discipline holds at delivery. A reply is often several message bubbles, and a worker can die between any two of them.

// Every bubble is journaled before the first one goes out.
for (const [i, text] of reply.bubbles.entries())
  await journal.pending(`${inbound.id}:${i}`, text);

for (const row of await journal.unsent()) {
  // A worker that has been replaced sends nothing.
  if (!(await claim.stillMine())) return;
  // A duplicate key returns the earlier result instead of sending twice.
  await channel.send(row.key, row.text);
}

A reaper notices a dead worker after 30 seconds of silence, and the next worker resumes from the journal, resending only what is provably unsent, with no model call.

A context the engine owns

What the model reads is as deliberate as what it may do. V8 owns its own tool text, under byte limits that CI enforces. A small core of tools is always loaded, and everything else is deferred and looked up when a turn needs it. The rules prompt has a byte ceiling in CI too, and only a handful of prose rules may still be waiting for an enforcer, because a rule that matters should be code that acts.

What changes every turn lives in one uncached block of ranked sections under a 12,000-character cap. When a turn has more to say than fits, the lowest rank goes first. “Waiting for your OK” is not evicted, because an offer the model cannot see is an offer it can bind wrongly.

The layers are ordered by how often they change (tools, the shared core, you, this turn, the messages), so the provider’s prefix cache stays warm across everyone for free.

Texts, web chat and routines run the same engine

An assistant that acts for you has more ways in than a chat box: texts, web chat, reminders and briefings, document edits, routines that run on a schedule. V8 runs them through one gate and gives each only the shape it needs.

Capabilities differ by service, so V8 fails closed: only iMessage gets reactions, typing and threads. RCS bubbles are joined into one message, because RCS delivered the second bubble seconds after the first. A text that arrives mid-turn is folded in at two checkpoints, and only when nothing is open or staged. Document edits leave the chat loop entirely: a planner emits a closed set of operations, and fixed code applies and verifies them.

Proving it before anyone relied on it

Replacing the engine under a live assistant has one failure that matters more than the rest: a send nobody approved. How we shipped V8 was built around that.

We audited the old engine first. Five read-only crawls of it found 166 items, 133 distinct behaviours, from the cross-device history window to entity references. We treated them as requirements rather than a to-do list. They folded into five systems (a tool contract, a context registry, enforcers, the turn runtime and a ledger projection), and the go-live bar became one number: zero high-impact gaps.

We made go-live test decisions, not infrastructure. Our first go-live check passed six of six: tables existed, functions were deployed, the flag was off. Then we ran the deployed decision code over the verbs that send and remove, and 52 of 53 were allowed. Nobody had been routed yet, so no one was exposed. Every safety check now asserts a verdict: this operation, this context, this decision.

We tested the real path. Our first bench ran real turns with the real model against a write overlay, and it found three go-live blockers that fake-model tests had missed. The harness that replaced it pushes messages into the real SMS seam on a fenced tenant with one injected clock, and grades from the ledger first. Model-judged checks flipped about 31% of the time between identical runs and code-scored checks about 4%, so only the code-scored part counts as firm.

We moved traffic with the kill switch written first. The rollout dial moved only on an explicit call, with kill metrics agreed in advance: any send without a yes, fallback above 8%, silent turns above 2%, p90 above 25 seconds. Memory went before traffic, so nobody arrived at the new engine as a stranger.

Where it landed

Production figures cover text turns from October 13 to November 3, 2026, excluding internal and test accounts.

What we learned

Code decides what runs. Every hard lesson of the rebuild was about authority, not intelligence.

Consent is judged, not matched. Word lists missed “yees” and “fine” and read “yes skip” as yes. A typed judge with a calibrated threshold, bound to stored arguments, is the version we trust.

Do not put a model after the reply. Our first voice step ran a check on every reply and rewrote 52% of them, for three days of internal testing, and fixed nothing. The cause was in the data: four labels for one staged send. We fixed the labels and deleted both calls, about 1,100 net lines.

Cut text on code points. A helper in the old engine could cut a string between the two halves of an emoji, and a strict provider parser rejects the whole request. All truncation now goes through one safe helper.

Green is not effect. Two tool-description edits went green and changed nothing, because V8 reads its own tool text. Deployed means the deploy log names the function; fixed means the production rows changed.

Measure the trap before the number. A latency stamp read negative on every turn because it was written before the turn started, and 36 of 38 “user reactions” were our own tapbacks echoed back. A zero can be the instrument.

If separating judgment from authority sounds like the right way to build an agent people can trust with their inbox, we’re hiring.