τ-Voice and the October 2026 public voice boards are still public exams, so the only eval that answers whether your agent works is a held-out set of your calls and your kitchens – not a leaderboard login.
Artificial Analysis’s Speech-to-Speech Index, snapshot around 18 October, is the board product teams are screenshotting. Grok Voice 3.0 leads the composite at 86.9. GPT-Live-1 sits at 83.2. Gemini 3.8 Live Extended Thinking sits at 82.6. The same table then reports the agentic slice: τ-Voice, simulated support calls with accents, background noise and interruptions. GPT-Live-1 is 74.5 percent. Grok is 71.6 percent. Gemini is 68.6 percent. The composite winner is not the winner of the call that sounds like a customer. That gap has already been a launch-week story. The story that lasts past the screenshot is blunter. τ-Voice is still a public exam. Your tickets are not on it. Your kitchen is not on it. A model can study the public set the way models studied SWE-bench. If you ship on a login, you are scoring a product you are not serving.
The market pitch for Live agents has been latency and “it feels like a call.” The boards now give that pitch a number. Full Duplex Bench for turn-taking. Speech Agent Arena for preference. Task success for tidy completions. τ-Voice for the noisy call. All of those are better than a booth word-error rate. None of them is a lender’s Monday, a retailer’s queue, or a kettle in a UK kitchen with a child talking over the account number. Public exams measure overlap with the exam. Private eval measures the job.
What the public exams actually are
The Speech-to-Speech Index is a blend. Speech reasoning on a Big Bench Audio slice. Agentic performance on τ-Voice. Preference votes in a speech arena. Task-success rate. Time to first audio sits next to the score because speed travels. Grok is strong on the pieces that reward clean intelligence and a finished task. Speech reasoning in the high nineties. Task success 94.8 percent. First audio under a second. Those are real properties of a speech-to-speech model. They are also the properties a quiet headset in a quiet room will flatter.
τ-Voice is the other object on the same page, and it is why serious buyers squint. Accents. Background noise. Interruptions. A simulated support call, not a reasoning puzzle read into a close mic. That is closer to the product enterprise teams keep describing. It is still simulated. It is still shared. It is still the same hundred-or-so conditions every lab can overfit if the items leak, and every vendor can quote if the customer has not frozen anything of their own. “Comparable” on τ-Voice at 40 percent lower latency is a launch sentence. It is not a release gate for your IVR.
Full Duplex Bench is a third object. Pause, barge-in, turn-taking, often in cleaner conditions than a kitchen. A stack can look responsive on that bench and still talk over a customer who hesitates, or stay silent while a kettle reads as a turn. Speech Agent Arena is preference: which voice would a rater rather hear. Preference is not task success. A lovely voice that books the city the caller abandoned is a preference win and a product fail. We already wrote that private eval is becoming the product when SWE-bench started leaking. Voice is the same contamination story with a microphone.
Configuration labels make the public table even less like your call. GPT-Live-1 on Medium. Gemini on Extended Thinking High. Grok without an equivalent effort tag in the launch table. Rival models at different effort levels are not a like-for-like comparison, and they are certainly not your latency budget, your tool schema, or your locale. Shopping from the composite is how you buy a headset demo.
Imagine the call none of those boards contains. A customer in a British kitchen, phone on the counter, radio on, child asking for a drink. They start an account number, stop, start again, talk over the agent, switch from a postcode to a phone number mid-sentence, then ask whether the card has lounge access. The agent has to hear the repair, stop talking, keep the tool call aligned with the corrected slot, and not invent a polite confirmation over the top. τ-Voice is a sketch of that call. It is not that call. A booth WER is not even a sketch. Your tickets, held out, with speakers who are not in the fine-tune, are the only exam that includes your tools, your rooms, and your accents on purpose.
Public boards get studied
OpenAI already said the coding benchmark was contaminated. Tasks from public repositories. Frontier models reproducing the exact patch. Anthropic showing BrowseComp answers sitting on the open web. Voice will not be the exception because the files have a waveform. If τ-Voice items, Full Duplex clips, or arena prompts are crawlable, fine-tuned against, or sitting in a vendor’s “eval pack,” the graph will go up and the kitchen will not move. A public exam that becomes training data is no longer an exam. It is a memorisation contest with extra latency work.
Do not copy the public clips and call them your release gate. Copy the events. Real human audio, not TTS standing in for disfluency. Overlap and interruption labelled, not treated as noise. Self-correction held out so the fine-tune cannot memorise the Boston trick. Multi-step tools that look like your APIs, not only a mock flight search. Speakers held out of training. Rooms that look like production: phone in a kitchen, headset in a shop, far-field on a counter. Locale as a real axis, not a language name. British and Irish English in a UK kitchen is not US English in a parked car.
Scoring on τ-Voice and then fine-tuning on anything adjacent is how you recreate eval leaking into training with a microphone. Freeze a set. Keep the people out. Re-run it when you claim the model moved. If you cannot answer “are eval speakers excluded from training?” in one sentence, assume no. Random clip splits from a call-centre dump are how a model memorises a household of agents and still fails the next customer.
Rights sit in the same brief. Consent for a recogniser is not consent for a Live agent, and it is not consent for TTS from the same session. Call-centre quality monitoring is not a commercial training licence. If you pull “in-the-wild” archive because it already contains overlap, read the purpose line before you celebrate the acoustics. The room may still be wrong. The speakers may never have agreed to this product. A public bench licence is not your speakers’ licence.
Gold before you shop the board. Write one sentence that includes the room and the job. “Handle interruptions and self-corrections on a phone in a UK kitchen, television on, British and Irish English, then complete a two-step booking, speakers held out.” Then say whether you are collecting barge-in events, conversational turns, or TTS, separately. If a catalogue set already matches locale and device, listen with the actual stack you will ship, not with a booth WER and not with the AA login. Custom when the wake-word, the kitchen, or the tool chain is yours.
Your tickets are the only private eval
A familiar demo week will keep happening until the exam changes. The agent sounds human on a headset in a quiet office. Grok is first on the composite. GPT-Live wins τ-Voice. Gemini wins a preference slice. Friday looks like a vendor shortlist. Monday is a kettle, a repair, a second person, and the agent talking over the customer or booking the city they abandoned. Nothing in the public files was your ticket. The latency work was real. The eval was the wrong object.
Build the private set as if the public boards did not exist. Real calls or real kitchens, consented for this product and this use. The events you claim: overlap, repair, second speaker, far-field, code-switch. The tools you actually invoke, with the arguments that fail in production. Easy, medium, hard by how much conflict sits in the request. Speakers and households held out. A listener on the items automation will “clean up” because they sounded messy. Those messy items are the only ones that predict Monday.
Duration and device belong in the spec so QA has a reject for “too clean to be the job” instead of an argument in week three. If you collected in a booth because it is easier to score, label the files as booth. Then do not use them as the proof that Live-class behaviour works in a kitchen. Read speech, a trigger phrase, a conversational turn, and a barge-in are four jobs. Mixing them in one zip called “speech” is how a composite index looks healthy and the agent talks over the customer.
Do not let τ-Voice become the internal name for “we did noisy audio.” It is a shared support-call sketch with a published score. Useful as a sanity check, the way SWE-bench Pro is a sanity check after SWE-bench Verified leaked. Not useful as the sentence you take to a risk committee. The committee wants to know whether this model, on these tools, in this room, with these speakers held out, completed the booking. Only your files contain that sentence.
Public transcription leaderboards will not catch it either. They have the same contamination problem as the coding boards, with a microphone. Preference arenas will not catch a wrong tool call. Full-duplex benches will not catch a locale you never recorded. τ-Voice will not catch your IVR. Collect the events you claim to handle. Hold them out. Listen with the stack you will ship. If you cannot write that without opening the public board, you are not ready to claim Live-class behaviour. You are ready to claim that the model can sit a public exam.
The models can listen and speak at once. The boards can rank them. Your files still cannot, unless you recorded the overlap on purpose and kept it out of the fine-tune. That is the collection. A leaderboard is a different product with a better-looking score. Your calls are the eval. τ-Voice is a sketch you should not memorise.
People4.ai collects consented speech for training and evaluation. Send the tickets and the kitchen, not a leaderboard login. hello@people4.ai · Contact
FAQ
Can we use τ-Voice as our release gate and collect a private set later?
You can try. You will be shipping against a public sketch and then discovering your tools, rooms and speakers. Freeze a private set that looks like your calls first. Use τ-Voice as a sanity check, not as the exam.
If we lead the Speech-to-Speech Index, does that mean the agent works in our kitchen?
No. The composite averages unlike exams. A reasoning win and a preference win will pull the number up while τ-Voice, and your actual tickets, still fail. Score the job. Quote the board in a footnote.
Is a call-centre archive a private eval if we hold out some files?
Only with a licence that covers this training use, this product and these speakers, and only if the hold-out is by person, not by random clip. Quality monitoring is not commercial Live-agent training. The room may still be a booth.
Related reading
Private eval is becoming the product · OpenAI just said the coding benchmark was contaminated