Degrade over crash: running a voice pipeline behind ~120 feature flags

A web page that errors shows a stack trace. A voice call that errors goes silent in someone's ear. That asymmetry sounds small, but it drives the entire architecture of our voice engine. On the web you can afford to fail loudly and recover; on a live call, the person on the other end has no console to read and no button to retry. When something breaks mid-call, the only acceptable behavior is to keep the conversation alive.

Degrade over crash

So the engine is built to downgrade gracefully instead of dropping the call. It runs behind roughly 120 feature flags, and the organizing rule behind all of them is degrade-over-crash. Every risky capability — a smarter turn model, a new processor, an experimental transport — can fall back to a simpler, known-good path without ending the conversation. Each of those flags also defaults off, so a new capability proves itself before it is trusted with a real call, and a misbehaving one can be switched out without a redeploy.

This is the same fail-closed instinct that governs the rest of the factory, applied to runtime rather than to shipping. When the system is uncertain, it does the safe, boring thing: it degrades to the path most likely to keep working.

The flags also give a new capability a way to earn its place. It ships dark, off by default, and gets exercised deliberately before it is trusted with a live conversation; if it misbehaves, it can be switched back out without waiting for a redeploy. A risky idea and its safe fallback ride in the same build, and the flag decides — per session, in real time — which one a given call actually uses. That is what makes it reasonable to keep iterating on the pipeline while it is carrying real calls.

Sticky failover

Speech synthesis is the clearest example. If our primary text-to-speech provider degrades mid-call, the pipeline fails over to a second provider — and then stays there for the rest of the call. That stickiness is the interesting part. The naive design flips back to the primary the instant it recovers, but switching the voice a caller is hearing back and forth mid-sentence is more disorienting than a single, one-time change. So we accept a slightly worse steady state to avoid a jarring one. It is a small decision that only reveals itself when you run real calls with real people, rather than clean demos.

Making the first second and the slow second feel human

Two moments betray a voice system more than any other: the very first second of a call, and the second when it has to go and think. The engine treats both as engineering problems rather than accidents. The opening greeting is synthesized once when the service starts, not freshly on every new connection, so the first thing a caller hears arrives without the pause of generating it in the moment. And when a tool call is going to take a while, the caller hears a short, natural Korean filler — the conversational equivalent of 'let me check that for you' — instead of dead air, so a working pause sounds like thinking rather than a dropped line.

Watchdogs around the happy path

Most of the reliability work is not in the happy path at all — it is in the ring of watchdogs around it. Silence detection notices when a caller has gone quiet and offers a gentle nudge instead of dead air. Voicemail is detected and handled rather than talked to. When a tool call runs long, the caller hears a natural acknowledgment instead of an unnerving gap, so the pause feels like thinking rather than a dropped line. Per-call latency is instrumented and posted to analytics, so a slow second somewhere in the stack becomes a data point we can act on rather than a mystery.

The same instinct governs how the engine connects at all. It speaks over more than one transport: browser and mobile connections are live today, while telephony, messaging, and avatar paths are built into the architecture and held behind keys until they are ready. The connection layer, like everything else, is designed to grow a new capability without a rebuild — and to fall back to a known-good path rather than fail when a newer one is not available.

Around all of that sits a suite of custom processors that handle the specifics of the domain: redacting personal information before it is ever logged, applying output guardrails, scoring risk, normalizing Korean text and pronunciation, and shaping prosody so the delivery sounds like care rather than a call center. Each one is a small, testable unit, and each can be toggled independently when something needs to change.

Reliability is a systems property

The lesson underneath all of this is that reliability in voice AI is not a property of any single model. The recognizer, the language model, and the synthesizer are all excellent, and none of them decides whether a bad second becomes a bad call. That decision lives in the pipeline — in how it detects trouble, what it falls back to, and how it holds a conversation together when a component wobbles. Models supply capability; the system supplies dependability. We build for the second one, because it is the one a family actually notices.

And it is the harder one to fake. A polished demo needs a model that sounds good for thirty seconds; a caregiver's phone call needs a system that stays composed for the ten thousandth minute, including the minute a provider hiccups. The watchdogs, the sticky failover, and the flags are unglamorous precisely because their whole job is to make sure nothing memorable happens when something goes wrong.