When Call Systems Pass Checks But Fail Customers

Imagine calling a government hotline, navigating the menu in your language, pressing the right button, and then hearing an AI voice answer in a language you don’t speak. Worse still, the system thinks it worked.

That’s basically what happened at the Washington State Department of Licensing, where callers who pressed 2 for Spanish heard an AI voice speaking English in a heavy Spanish accent. The error persisted for months, drawing widespread public attention because a resident posted a recording to TikTok that racked up over two million views and counting.

Whoops. This kind of failure is easy to miss on a dashboard. Uptime looks fine, call completion is normal, QA signed off. For the caller on the line, the system failed pretty spectacularly.

The layer nobody checks together

A multilingual voice system is essentially a stack. Language detection and selection at the top, speech recognition underneath, text-to-speech playback at the bottom of the pancake. Each layer has its own config, its own vendor, and sometimes its own team.

In my experience, organizations in most cases test those layers independently. The menu gets QA’d, recognition benchmarked, and the voice output reviewed. They all pass. Then they get woven together in production and the seams start to show.

This is where the anecdote stops being one. In a June 2026 benchmark of code-switched speech, ServiceNow’s AI research team found that OpenAI’s Whisper Large V3 Turbo model, given bilingual audio with no explicit language parameter, defaults to translating into English instead of transcribing.

That’s the wrong language out in one of the most widely deployed speech models available in the world. The Washington failure has a mechanism, and that mechanism delivers.

Metrics improve but experience gets worse

Here is the part I find that teams miss. Every contact center metric is calculated from data produced by the failing system, and for this class of failure the arithmetic runs backwards.

Containment rises when a caller who can’t understand the system hangs up instead of asking for an agent. Handle time falls because confused callers leave early in frustration. Post-call surveys go out in the language the system already got wrong, answered only by people who stayed to the end.

So a failing language shows up in the quarterly review as an outperformer, and a good number never gets investigated.

This is the problem I see with how most organizations test multilingual AI voice. They test the parts but rarely the whole journey. The shift I advocate for is simple in principle and harder in practice. Test the full voice journey as a customer experiences it, and not as an engineer configured it. Two totally different perspectives.

This all means four things.

Count the matrix: Language, regional variants, entry path and task type multiply fast. Ten languages, two variants, three entry points and eight task types equal 480 routes. Count how many anyone has actually listened to. The difference is your exposure.

Score comprehension, not accuracy: Word error rate tells you how well a model transcribed, not whether a person got what they needed. Have a native speaker of the variant attempt the real task, then record whether they finished and whether they wanted to switch to English. People switch long before they complain. Businesses lose $75 billion due to poor customer service, a staggering figure.

Pin the language contract and assert it: Pass explicit language parameters through recognition, understanding and synthesis. Don’t rely on automatic detection, then run the outgoing audio through a language identifier and confirm it matches what the caller chose. That check alone would have caught the Washington Spanish accent case on day one. In fact, in hours or minutes.

Watch for divergence: Track abandonment, repeat calls, escalation and completion per language against the English baseline. Absolute numbers can hide the problem, and divergence exposes it. Do this continuously, because models get updated on a vendor’s schedule rather than yours, and any update can break a language that worked last week.

The business case

Deploy voice AI in multilingual contexts and you are usually serving communities already underserved. The U.S. Commission on Civil Rights approved a report on language access this year noting that 26 million Americans are limited in English proficiency and often cannot reach essential services in a language they understand. Expectations are moving from availability toward effectiveness. A hotline that technically offers Spanish service but responds largely in English is arguably worse than one that never offered it. It creates an expectation, but then fails quietly.

The organizations getting this right are treating the voice experience like a product. There’s clear ownership, with a target for how fast a language-level failure gets detected. Detection within days is a reasonable ambition, instead of weeks, or even worse, never knowing.

This is largely not a model quality problem. The same benchmark found leading systems handle code-switched speech with small penalties against their monolingual baselines. The failures live in the seams: the parameter nobody set, the update nobody flagged, the variant that nobody tested, and above all the absence of anyone actually listening in. Those are integration and observability problems, fixable without waiting on anyone else’s roadmap.

The Washington case is small but precise. A system can pass every checkpoint and still fail the person it was built to serve, in this case very publicly. If you have not listened to your voice system the way your customers do, it’s worth making that call to check. Otherwise, word will get out, on TikTok a few months later, in front of millions of people.