Voice agents are 5x slower than human conversation. Moss AI thinks it found the fix.

The Post-LLM Talent Migration Has a New Ground Zero

OpenAI cut the GPT-4o realtime API input price by 60% in December 2024 and the output price by 87.5%. AWS launched Nova Sonic for real-time voice in Bedrock the same quarter. When the cost of raw intelligence drops that fast, the value doesn't live in the model anymore. It lives in the retrieval, inference, and orchestration layer that wraps the model and makes a conversation feel continuous. Olivia Moore and Anish Acharya at Andreessen Horowitz put it directly in their 2025 voice agent update: "We are just now transitioning from the infrastructure to application layer of AI voice."

Y Combinator's F25 batch has 22% of its companies building voice AI, up from 13% in W24, according to Cartesia's analysis of cohort data. That three-batch acceleration signals that the talent frontier has moved. The hard, valuable engineering problem is no longer training a model — it's making an agent respond to a human in under 500 milliseconds without sounding like a phone tree.

The transition is rewriting what companies hire for. The 2023–2024 recruiting wave chased prompt engineers, RLHF fine-tuners, and model evaluators (people who made LLMs behave). The 2025–2026 wave targets engineers who can build sub-10ms semantic search, manage real-time context across a 10-minute conversation, and ship a streaming inference pipeline that doesn't choke at 1,000 concurrent calls. Sierra Ventures' voice AI market map of 150+ companies, published August 2025, identifies "voice infrastructure" and "enterprise voice middleware" as the fastest-growing categories, not model providers.

The consumer side is pulling too. VML's Future Shopper Report found 49% of global consumers owned a smart assistant in 2024, and 23% regularly use voice-activated assistants to make purchases. The interface through which those consumers interact with AI is increasingly spoken, not typed. The gap between "it works in a demo" and "it works in a noisy kitchen at 7 p.m." is a retrieval and inference problem, not a model problem.

That's why Moss AI's YC F25 slot matters beyond its own product. The sub-10ms retrieval layer it's building (semantic search without vector databases) is exactly the infrastructure play that the cohort is optimizing for. Moss isn't selling a better model. It's selling the thing that makes a voice agent feel like the model doesn't exist. And Y Combinator, for the first time, has a batch where that pitch fits the majority thesis.

Inside Moss AI's Sub-10ms Architecture

Moss AI's core claim is simple: semantic search in under 10 milliseconds without a vector database. The benchmark numbers on its site — P50 at 3.1 ms, P99 at 5.4 ms, tested on 100,000 documents with embedding inference included — put it roughly 100x faster than Pinecone (P50: 432.6 ms) and Qdrant (P50: 597.6 ms) on the same workload. The benchmark script is open-source on GitHub, and the test ran on a MacBook Pro M4 Pro with 24 GB of RAM.

The architecture that produces those numbers departs from the retrieval layer most AI teams have built around since 2023. Moss runs a three-part system. Moss Cloud handles ingestion, embedding, and document storage. The index (documents and their vectors packaged as a single artifact) lives on Moss Cloud and gets pulled over HTTPS. The runtime embeds directly in the application, holds the index in memory, and serves queries locally. Once an index is loaded, queries never leave the process. That is the entire latency story: no network hop on the hot path, no cluster to tune, no HNSW parameters to fiddle with.

The SDK ships in Python, TypeScript, Swift, Elixir, Go, and C, with a WebAssembly build for browser-only deployments. Framework integrations cover LangChain, DSPy, LlamaIndex, Pipecat, LiveKit, Vapi, ElevenLabs, CrewAI, Haystack, Mastra, Pydantic AI, AutoGen, and the Vercel AI SDK. The GitHub repo shows 427 stars, 50 forks, and active development as of June 2026, with recent commits adding DynamoDB connectors and a Go SDK.

This is not a vector database replacement in the traditional sense. Moss does not manage clusters or expose a query API over HTTP. It is a search runtime: you load an index into your process and call a function. That distinction matters for the kind of engineer who builds on top of it.

The engineering profile Moss demands is closer to systems programming than to LLM application development. The team's open-source repository includes Rust/PyO3 bindings, a C SDK (libmoss), and a Go SDK built on a CGO wrapper over that C library. The Python SDK is a thin wrapper over a native core. This is not a team that fine-tunes models or writes prompt templates. It is a team that compiles to WebAssembly, manages in-memory index state, and thinks about cache locality.

That has hiring implications. The roles Moss is likely recruiting for — real-time systems engineers, search infrastructure builders, ML inference specialists, and SDK engineers who can write idiomatic bindings across six languages — are a sharp departure from the prompt-engineering and fine-tuning profiles that dominated AI job boards in 2023 and 2024. The company's own documentation and GitHub structure suggest the engineering org is split between core runtime work (Rust, C, Go) and SDK/framework integration (Python, TypeScript, Swift). The voice-agent examples, including a Pipecat integration and a LiveKit deployment on Vercel, require engineers who understand real-time audio pipelines, not just REST APIs.

For engineers evaluating where to place their time, the signal is clear: the retrieval and inference layer is where the hard systems problems in AI are concentrating, and companies like Moss are hiring for the skills that layer demands.

The Production Voice-Agent Gap

The numbers expose a brutal gap. Human conversation runs on a 200–300 millisecond response window (research across multiple studies confirms this is neurologically hardwired across languages and cultures). Simple reaction times average around 220 milliseconds; recognition reaction times, 384 milliseconds. Yet production voice AI agents deliver median latency of 1,400 to 1,700 milliseconds, according to an analysis of over 4 million voice agent calls in production published by Hamming AI. That is roughly five times slower than the human benchmark.

At the 90th percentile, response times hit 3,300 to 3,800 milliseconds. This is a range where talk-overs and user frustration become the norm. At the 99th percentile, calls stretch to 8,400 or even 15,300 milliseconds. Complete breakdown. Users don't complain about "latency." They report agents that feel slow, interrupt at the wrong time, or don't understand when they're done talking. Research shows 68 percent of customers abandon calls when systems feel sluggish.

The component math explains why. A typical cascading pipeline (STT, then LLM, then TTS, plus network and processing overhead) stacks up fast. Industry benchmarks put the typical latency budget at roughly 200ms for speech-to-text, 500ms for LLM inference, 150ms for text-to-speech, 50ms for network transport, and 100ms for orchestration and processing. That totals around 1,000 milliseconds on a good day. Retell AI's own benchmark data shows its platform averaging 620 milliseconds end-to-end, the fastest among major providers, while Google Dialogflow CX lands at 920 milliseconds, Twilio Voice at 1,040 milliseconds, and PolyAI at 780 milliseconds. OpenAI's Realtime API sits around 1,003 milliseconds. The open-source Voice AI Leaderboard maintained by Dasha.ai shows similar spreads: Dasha at 870 milliseconds, OpenAI at 1,003 milliseconds, Telnyx at 1,053 milliseconds, with ElevenLabs and VAPI stretching past 2,200 milliseconds.

The business cost of these delays compounds at scale. For a contact center handling 10,000 calls a day, even a 10 percent abandonment spike driven by latency translates to thousands of failed interactions daily, including calls that never resolve, customers who switch to human agents, and revenue that evaporates in the gap between silence and response. Retell AI's research links high latency directly to reduced trust, conversation overlap, abandoned interactions, and lower conversion rates in sales contexts.

This is the gap Moss AI is targeting with its sub-10ms semantic search layer. The company's architecture aims to collapse the retrieval bottleneck that sits inside the larger latency stack. This is the step where the system fetches context, checks memory, or resolves intent before the LLM ever starts generating. Shaving milliseconds off that internal retrieval loop doesn't sound dramatic until you realize the LLM typically accounts for 70 percent of total latency, and the retrieval layer feeds directly into inference quality. A faster retrieval layer means the model gets better context faster, token arrives sooner, which means the whole pipeline tightens.

The voice AI agent market is projected to grow from roughly 2.5 billion dollars in 2025 to over 35 billion dollars by 2033, according to Grand View Research. That growth depends on solving exactly this problem: making voice agents feel less like telephone menus and more like actual conversations. The components exist: Deepgram delivers 150ms ASR, ElevenLabs hits 75ms TTS, Groq serves LLMs at 200ms. The remaining bottleneck is the orchestration and retrieval layer that ties them together. That is where the next wave of engineering talent, and funding, is converging.

YC F25's Bet on Moss Signals a Broader Cohort-Level Pivot

Y Combinator's F25 batch, which kicked off in January 2025, placed Moss AI among a cohort that reflects a sharp shift in what the accelerator, and the early-stage market behind it, considers a fundable AI startup. The era of pitching a proprietary foundation model as a moat is fading. What replaced it is a class of companies building the retrieval, inference, and interface layers that make those models actually usable in real products.

Moss AI fits squarely in that mold. Its sub-10ms semantic search architecture targets the gap between what a language model can reason about and how fast a voice agent needs to respond. This is a problem that didn't exist at commercial scale two years ago. YC's decision to back Moss signals that the accelerator sees the same talent migration playing out across the industry: the highest-value engineering problems have moved downstream from training to serving.

The broader F25 batch composition backs this up anecdotally. YC CEO Garry Tan has publicly described the cohort as heavy on infrastructure and tooling plays rather than consumer-facing AI wrappers, a reversal from the W22 and S23 batches where "wrap GPT in a UI" was a viable path to demo day. Founders in the current cohort are, by YC's own characterization, more technical on average, and more of them are building for developers or for systems that sit between a model and an end user.

This tracks with what hiring data across the frontier AI sector shows. The demand profile has shifted from researchers who can train or fine-tune models to engineers who can make those models fast, cheap, and reliable in production. Companies like Anthropic are still hiring aggressively for research roles, but the fastest-growing category of roles at AI-native companies is inference, retrieval, and real-time systems. That's exactly the layer Moss is building.

For job seekers and hiring managers reading the signal, the implication is straightforward: YC's batch composition is a leading indicator of where early-stage capital and talent are converging. The accelerator doesn't bet on companies it thinks will plateau. Moss AI's presence in F25 says the interface and retrieval layer, the stuff that makes AI feel like a product and not a demo, is where the next wave of companies will be built and staffed.

What Moss AI's Hiring Profile Reveals About Post-LLM Engineering Demand

Moss AI's open roles read like a blueprint for what the post-LLM stack actually needs. The company's careers page lists positions squarely in real-time semantic search for conversational AI. This is a profile that would have been niche two years ago and is now a signal of where frontier AI hiring is heading.

The roles break into three clusters. First, real-time systems engineers who can keep inference latency under 10 milliseconds at scale, a constraint that rules out most off-the-shelf retrieval pipelines and demands people who understand memory management, kernel-level optimization, and the ugly physics of network hops. Second, search infrastructure builders who can design semantic retrieval without leaning on vector databases, which Moss has explicitly sidestepped as a bottleneck. That's a narrow skill set. It draws from the same talent pool that built search at companies like Elasticsearch and Vespa, not the fine-tuning crowd. Third, ML inference specialists who can serve models in production with the reliability voice agents require, where a 200-millisecond glitch breaks the illusion of conversation and a user hangs up.

This is a sharp departure from the hiring profiles that dominated 2023 and 2024. Back then, the hottest titles were prompt engineer, LLM fine-tuning specialist, and AI safety researcher, all roles oriented around coaxing better behavior out of existing models. Those skills still matter, but the bottleneck has moved downstream. The model is no longer the hard part. Getting it to respond in under 10 milliseconds, with accurate retrieval, in a production voice pipeline: that's the hard part now.

The salary data backs up the shift. AI engineer compensation in 2025 is hitting around $206,000 on average, according to multiple market surveys, with inference and infrastructure roles commanding premiums because the supply of engineers who can work at that layer is thin. Anthropic's own recent listings show that even foundation-model companies are paying up for talent that sits closer to deployment than to research.

Moss's hiring profile is a small sample, but it tracks with a broader pattern: the center of gravity in AI engineering is moving from training to inference, from prompting to retrieval, from "can the model do this" to "can the system do this in under 10 milliseconds." Builders who can answer that second question are the ones getting hired first.

Who Else Is Building the Retrieval Layer

Moss AI enters a retrieval and inference market that is fragmenting into three distinct camps: infrastructure giants optimizing for speed, agent-context startups optimizing for accuracy, and vertical voice companies optimizing for conversation quality. None of them have solved the sub-10ms semantic retrieval problem Moss is targeting, but each occupies a different slice of the stack that voice agents depend on.

The inference-speed camp is the most crowded. Groq's Language Processing Unit targets record-breaking tokens-per-second for LLM inference. Cerebras Systems builds wafer-scale engines; the WSE-3 packs 4 trillion transistors and 900,000 AI cores on a single chip to eliminate memory bandwidth bottlenecks. Fireworks AI positions itself as the fastest inference platform for generative AI, with compound AI orchestration that chains multiple models, retrieval, and tools. Together AI raised over $400 million to optimize open-source model serving with custom kernels. These companies attack the compute side of the latency problem. They make the model respond faster. They do not address the retrieval step that happens before the model ever runs: the lookup that determines what the agent knows about the caller, the account, the last three things said.

That retrieval gap is where a second cluster operates. Glean builds enterprise search across 100-plus SaaS applications with permission-aware knowledge graphs. a16z's "Your Data Agents Need Context" thesis, published March 2026, explicitly named context and retrieval as the missing layer for functional agents. Foundation Capital's December 2025 "context graphs" memo framed retrieval infrastructure as AI's trillion-dollar opportunity. These are the companies that understand the retrieval problem but attack it at the enterprise knowledge-management level, indexing documents, connecting Slack and Confluence, and building permission graphs. They are not optimized for the sub-10ms round-trip a live phone call demands.

The voice-specific camp is where Moss's direct competitors live. Retell AI offers conversational AI for phone calls with a low-code platform and bring-your-own-carrier model. Sierra AI, founded by Bret Taylor and Clay Bavor, raised over $300 million to build enterprise conversational AI with deep CRM and order management integration. AssemblyAI provides the speech-to-text and transcription layer underneath. HubSpot Ventures mapped vertical voice agents as a distinct category in December 2025. Dawn Capital's April 2026 "Beyond the uncanny" post explicitly called out the move toward lifelike voice AI. These companies own the voice channel but rely on conventional retrieval stacks (vector databases, embedding pipelines, RAG) that introduce latency Moss claims to eliminate.

The investor attention data backs up the timing. CB Insights published a dedicated AI agent tech stack map in August 2025 and followed with an agentic commerce map in June 2026. Bessemer Venture Partners released a voice AI roadmap in November 2025. a16z's voice agent update in January 2025 and its June 2026 "AI Assistants in iMessage" map show the firm tracking voice as a platform shift. Redpoint's InfraRed Report 2026 and Radical Ventures' "Rise of NeoLabs" in June 2026 both flagged inference and retrieval infrastructure as a top investment theme. The market maps compiled on GitHub's Awesome AI Market Maps repository show voice AI, AI infrastructure, and AI agents as three of the most-mapped categories across 500-plus entries from 2024 through 2026.

Moss's bet is that the retrieval layer is the wrong place to compromise. Every voice agent architecture in production today chains speech-to-text, retrieval, LLM reasoning, and text-to-speech in sequence. The retrieval step, finding the right context, the right account data, the right procedural knowledge, is the one that most implementations still handle with vector databases that add 50 to 200 milliseconds per query. Moss's sub-10ms claim, if it holds at production scale, removes that bottleneck entirely. The companies that matter most as competitors are not the inference-speed players but the voice-platform companies, including Retell, Sierra, and the vertical voice agents, because they are the ones feeling the retrieval latency problem most acutely and the least equipped to solve it at the infrastructure level.

Working in AI? Zero G Talent tracks the openings: browse AI jobs, openings at Databricks and Anthropic, and the people building the field.