Blog
Speech-to-speech AI, a primer
What changed in 2024, what the words mean, and why a new class of models treats speech as a first-class language rather than a pipeline of text conversions.
Articles
Foundation before vertical
Kyutai: the twelve-person Paris nonprofit turning open releases into shared vocabulary
Cartesia: why AWS put a non-transformer voice AI on its own shelf
Hume AI: the smile inside a sentence, and the nine days that clarified voice AI’s exit shape
ElevenLabs: why a TTS company is priced at $11B
Decagon — the $4.5B bet on speed
Abridge — the eaves and the house, built by a cardiac surgeon who was still writing notes
Meta FAIR Speech: six years, nine papers, and the field's default citations
Mozilla Common Voice — why a CC0 read-speech corpus became the voice-AI industry's yardstick for consent
LDC at 34: the academic consortium that quietly trains 2026's full-duplex stack
NII, Nagoya, and J-Moshi: the morning academia shipped a Japanese listen-while-speaking AI
Alibaba DAMO and the Qwen Audio team: the most-downloaded open audio lab closed only its flagship
Artificial Analysis: how two Australians became the AI industry's independent scoreboard
Mapping the benchmark landscape
Why STS needs new benchmarks
The STS model landscape
Consent, licensing & the opt-in economy
The data ceiling
From pipeline to integrated
The full-duplex threshold
- 01Speech-to-speech AI, a primerWhat changed in 2024, what the words mean, and why a new class of models treats speech as a first-class language rather than a pipeline of text conversions.published
- 02The full-duplex thresholdA number, a biology fact, and a small cluster of systems. What the full-duplex threshold actually is, what it takes to cross it, and what conversations above it unlock.published
- 03From pipeline to integrated“Integrated” sounds like one architecture. It is at least four. A field guide to the 2026 full-duplex STS landscape — four families under one label, their latency math, their data bets, and their license exposure.published
- 04The data ceilingFull-duplex conversational recordings at internet scale do not exist. The two escape hatches engineers reach for first — better separation AI and bigger YouTube scrapes — do not escape. Full-duplex STS still leans on a 2004 telephone corpus for its post-training recipe.published
- 05Foundation before verticalFull-duplex STS sits between the GPT-2 and GPT-3 moments. Asking “which vertical wins first?” in 2026 is a category error — the constraint is whether the foundation the verticals will sit on exists yet. A thesis essay on the foundation threshold, the 30×–150× data gap, and six plausible routes to 100,000+ hours of two-channel dialogue.published
- 06Mapping the benchmark landscapeToo many speech-to-speech benchmarks, each covering a different slice. The map, as of April 2026 — arena versus fixed test set, four capability axes, a coverage heatmap, and a Japanese gap.published
- 07Why STS needs new benchmarksThe STS field inherited evaluation machinery from ASR, TTS, and text-LLM paradigms. None of them measured a live, two-channel, socially-timed conversation. The argument for a rebuild, plus a concrete picture of who could run it.published
- 08The STS model landscapeThirty-plus speech-to-speech models, four architectural families, and a licensing pattern that is starting to split inside each lab. A field guide to the April 2026 map, legible enough to place newly announced models in one or two paragraphs.published
- 09Consent, licensing & the opt-in economyThe consent and licensing stack for conversational voice data in April 2026 is three layers deep: a fixed biometric-privacy floor, a seven-platform patchwork middle, and a transparency ceiling partially in force and partially in draft. An opt-in voice-data economy requires all three to survive together.published
- 10What comes after STSThe open questions the first nine dispatches leave behind — multimodal, on-device, and the next evaluation moat.Q3 2026soon
- v01Kyutai: the twelve-person Paris nonprofit turning open releases into shared vocabularyResearch velocity converted into reputational capital. A twelve-person Paris nonprofit ships weights every ten to twelve weeks, rewriting the vocabulary the open voice-AI field thinks in.published
- v02Sesame AI: the billion-dollar bet on voices you want to keep talking toTBAsoon
- v03Cartesia: why AWS put a non-transformer voice AI on its own shelfThe only voice-AI company commercially competing without a transformer. $191M cumulative, 62% blind-test preference, Sonic-3 on AWS SageMaker JumpStart — earned on a state-space model backbone by the people who wrote the SSM papers.published
- v04Hume AI: the smile inside a sentence, and the nine days that clarified voice AI’s exit shapeHume bet that emotion lives inside prosody. In January 2026, Google DeepMind brought on the founder and left the company standing. A new exit shape for voice AI — not buyout, not wind-down, but a graduation ceremony.published
- v05ElevenLabs: why a TTS company is priced at $11BIn February 2026, a two-person London startup closed a $500M Series D at $11B — a TTS company at the top of the voice-AI valuation stack. The structure behind that fact: founders, hypothesis, product, customers, counterargument.published
- v06Decagon — the $4.5B bet on speedIn January 2026, Decagon closed a $4.5B Series D and five weeks later an employee tender settled at the same mark — while the CEO was on the record that speed is a weapon but not a moat. A profile of the founders, thesis, product, customers, and counterargument that sit inside the price tag.published
- v07Abridge — the eaves and the house, built by a cardiac surgeon who was still writing notesAbridge used its time under Epic's eaves to build a four-layer house underneath. On February 5, 2026, the eaves went free. What happens next depends on whether the house still stands on its own.published
- v08CMU Language Technologies Institute: why commercial labs use a university's evaluation harness to measure their own modelsTBAsoon
- v09Meta FAIR Speech: six years, nine papers, and the field's default citationsBetween June 2020 and October 2024, Meta FAIR Speech shipped nine audio foundation models — Wav2Vec2, HuBERT, dGSLM, MMS, Seamless, Spirit-LM. By 2026, open full-duplex research thinks in the vocabulary Meta left behind. The release cadence, the talent diaspora to Kyutai and Gradium, and why this lab's floor outlasts its own release calendar.published
- v10Mozilla Common Voice — why a CC0 read-speech corpus became the voice-AI industry's yardstick for consent31,841 hours of audio. 286 languages. 800,000 contributors. All CC0. The story of how, over nine years, each person who hit the record button in a browser and handed their voice over as a public good made Common Voice the consent-first yardstick every next voice-data project is measured against.published
- v11LDC at 34: the academic consortium that quietly trains 2026's full-duplex stackProfile of the Linguistic Data Consortium (LDC) and Fisher English Training Speech — why a 1992 Penn-hosted academic consortium still supplies the documented-consent conversational audio that every major open full-duplex STS model in 2026 is fine-tuned on.published
- v12NII, Nagoya, and J-Moshi: the morning academia shipped a Japanese listen-while-speaking AIOn February 25, 2026, NII published LLM-jp-Moshi-v1 under Apache 2.0 — the first commercially usable Japanese full-duplex STS. With no commercial frontier voice-AI lab in the country, this profile traces how NII, Nagoya, and a U-Tokyo data lab formed the tripod that shipped a release 25 years of academic infrastructure made possible.published
- v13Alibaba DAMO and the Qwen Audio team: the most-downloaded open audio lab closed only its flagshipA Chinese big-tech lab populated the middle lane between fully-open and fully-closed. By April 2026 Qwen is the largest open audio family on Hugging Face (1B+ downloads, 200k+ derivatives), yet Qwen3.5-Omni ships API-only. Open-base closed-frontier: the first concrete signal from a Chinese lab of a third option between Meta/Mistral and OpenAI/Anthropic.published
- v14Artificial Analysis: how two Australians became the AI industry's independent scoreboardA profile of Artificial Analysis, the benchmarking company OpenAI now cites in its own launch materials. Three design choices — transparency, reproducibility, neutrality — built the neutral scoreboard of AI infrastructure.published
- v15DeepgramSTT at scale, then the orchestration-layer pivot. How a component vendor becomes an agent platform.Q2 2027soon
- v16NVIDIA AudioNeMo, Parakeet, and PersonaPlex — the GPU vendor that accidentally became a full-duplex open-weights publisher.Q3 2027soon
- v17Character.AIConsumer companion at 20M MAU and the presence-KPI thesis. Where the emotion-product market actually lives.Q3 2027soon