A voice-guided wellbeing session runs in the browser. An AI guide asks a question, the person answers out loud, and the app reads that answer to pick the next step in a scripted flow, for example yes, no, a named topic, or unclear. The team wanted a classify step (a point where speech must map to a closed set of outcomes) to run on a smaller candidate model, a fast choice model that returns one outcome plus a confidence score, instead of the general chat language model that currently owns those reads. I was preparing the rollout. Offline scores alone did not show behaviour per step and per language, and awaiting the candidate on live traffic would have put each turn at risk.

Shadow mode means the candidate runs on real turns beside the current reader, logs what it would have chosen, and never changes what the person hears. A fallback happens when confidence sits below the step threshold and the live system would hand the turn back to the language model, so a high fallback rate erases the latency win. For crisis language, an existing tripwire keyword check still routes sessions to crisis handling first.

Leave Await the shadow on, enable Candidate fails sometimes, and press Run 10 turns: reply times climb and a failed turn can appear. Switch to Fire and forget and run again. Replies stay near the language model timing and failures show only in the log. Press Publish with mode on and read which step and language the gate refuses, and why.

Parallel start, language model still owns the outcome

An environment switch selects off, shadow, or on. In shadow, the language model still decides what the person hears. The candidate call starts before the language model read, in parallel, and its result becomes log fields only: agreed, below threshold, confidence, duration, outcome, with failures recorded as fail:<reason> instead of throwing into the turn.

The decision journal is no longer awaited inside the turn handler. It waits on the shadow promise itself, catches errors into fail:error, and schedules the write while the handler returns as soon as the reply is ready. The client waits for the reply, not for its diagnosis. The write is not guaranteed to land; the turn simply never waits on it or fails because of it.

const shadow = mode === 'shadow' ? candidate.decide(args) : undefined
const result = await currentModel.classify(args)
journal(turn, result, shadow)
return reply(result)
 
function journal(turn, result, shadow?: Promise<ShadowResult>) {
  void (async () => {
    let fields
    try {
      fields = shadow && toShadowLog(await shadow, result)
    } catch {
      fields = { agreed: false, outcome: 'fail:error' }
    }
    await saveDecision({ ...turn, ...fields })
  })().catch(() => {})
}

Per-step gates before mode on

Publishing a flow with on refuses any step-language pair that misses the bar: at least 20 shadow turns, agreement at least 85 percent, fallback rate at most 30 percent. The refusal names the reason, for example need 20 shadow turns (have 7) or agreement 0.810 < 0.85. With off or shadow, authors can set thresholds freely. An admin view lists agreement, disagreements with both outcomes and confidence, and the median latency the candidate would have saved. The thresholds came from an earlier offline evaluation; I am not citing numbers from live shadow yet because this work did not ship agreement results.

Agreement with the language model measures sameness, not ground truth. That matters for safety: a candidate that disagrees with the tripwire is not automatically wrong, and swapping the tripwire for the candidate could drop recall.

Crisis signal stays tripwire-first

For crisis language the tripwire stays first and keeps routing. The candidate answers three yes-or-no questions in one call. Code combines them so escalation happens only when the answer is about the person right now and about wanting to die or harm, or nearly so. In shadow the candidate is never awaited and never routes. The next step is a labelled English and Dutch run measuring tripwire alone, candidate alone, and their union on hard negatives. That labelled run has not happened yet. The candidate may only add recall on top of the tripwire.

When this does not apply

Skip shadow rollout when outcomes are not a closed set, when traffic is too sparse to reach the minimum sample (labelled offline cases are the better tool), or when the candidate would perform writes or side effects. A shadow must never write user data or send anything outward.

The reusable rule: run a replacement model beside the live one where it cannot delay or fail the turn, judge it per decision point against a minimum sample before code lets it go live, and for safety signals let it only add to what the existing check catches.

Happy coding!

Sander