Zep’s Graphiti Hits 94.7% Accuracy at 155 ms Latency

Graphiti Goes Open Source

Zep AI open-sourced Graphiti in January 2025, releasing a temporal context graph engine that tracks how facts change, when they change, and where they came from, and it already powers the company's commercial Context Lake platform. The framework landed on GitHub with a paper (Zep: A Temporal Knowledge Graph Architecture for Agent Memory, arXiv:2501.13956) and a PyPI package, graphiti-core, now at version 0.29.3. Adoption signals compounded fast: 29.6k GitHub stars, 35-plus contributors, and a growing developer base.

Enterprises deploying AI agents hit a wall when those agents need to remember. Traditional retrieval-augmented generation freezes knowledge at ingestion time. LLMs synthesize answers non-deterministically, destroying the paper trail compliance teams require. Graphiti replaces static knowledge graphs with a temporal model: every fact (an entity, a relationship, a preference) carries a validity window: when it became true, when it was superseded, when the system learned it, and when it learned it was false. Point-in-time queries follow from the data model. Retrieval fuses vector similarity, full-text search, and graph traversal in a single ranked call, no LLM-in-the-loop reranking required. The engine runs on Neo4j, FalkorDB, or Amazon Neptune and plugs into OpenAI, Azure OpenAI, Google Gemini, and Anthropic. A Model Context Protocol server connects it directly to Claude, Cursor, and other MCP-compatible clients.

The split is deliberate. Graphiti gives developers a flexible OSS core: bring your own graph database, build your own tooling. Zep sells the turnkey Context Lake: managed infrastructure, dashboard with graph visualization, debug logs, API logs, SDKs for Python, TypeScript, and Go, plus enterprise features and deployment in the customer's cloud. "Choose Zep if you want a turnkey, enterprise-grade platform with security, performance, and support baked in," the documentation says. "Choose Graphiti if you want a flexible OSS core and you're comfortable building and operating the surrounding system."

Early enterprise deployments run on that commercial layer. Zep delivers sub-200-millisecond retrieval, SOC 2 Type II and HIPAA certification, and validation from S&P Global Market Intelligence. Customers feed it chat logs, voice transcripts, email, and structured business data, not just conversation history. The platform governs each subject's context graph at scale, handling provenance that file-based memory cannot.

Benchmarks back the claims. On LOCOMO, Graphiti hits 94.7 percent accuracy at 155 milliseconds latency with a 5,760-token context. On LongMemEval, 90.2 percent accuracy at 162 milliseconds with 4,408 tokens. The paper reports that Zep outperforms MemGPT on the Deep Memory Retrieval benchmark and exceeds it on evaluations the authors say better reflect real-world enterprise workloads. The roadmap stays aggressive: custom graph schemas, enhanced retrieval configurability, a hardened MCP server, and expanded test coverage are all in active development. The engine that powers Zep's commercial moat is now the same engine any team can download, extend, and run locally, and the enterprise memory race just got a new baseline.

Speed That Keeps Agents Interactive

Traditional RAG pipelines often take one to three seconds just for retrieval. For interactive agents — voice assistants, coding copilots, customer-facing chatbots — that delay breaks the illusion of intelligence. Users perceive anything above 200 milliseconds as "loading." Zep's Graphiti engine targets that threshold directly, and the numbers hold up under load.

Retrieval stays under 200 ms regardless of graph size or count. The architecture achieves this through pre-computed relationship indices, optimized graph traversals, and LLM-optimized context blocks that eliminate token waste. Typical multi-hop RAG with re-ranking exceeds two to three seconds; Zep's average latency sits at 187 ms with a 0.41 percent error rate.

Speed translates to accuracy. On LongMemEval using GPT-4o, Zep scores 63.8 percent versus Mem0's 49.0 percent, a 15-point gap. On LOCOMO, Zep reports a corrected 75.14 percent after arguing initial benchmark configurations used suboptimal settings. The arXiv paper shows Zep achieving up to 18.5 percent accuracy improvement while cutting response latency 90 percent compared to baseline implementations. Full-context GPT-4o-mini runs at 55.4 percent accuracy in 31.3 seconds; Zep with GPT-4o-mini hits 63.8 percent in 3.2 seconds. Full-context GPT-4o scores 60.2 percent in 28.9 seconds; Zep with GPT-4o reaches 71.2 percent in 2.58 seconds.

The latency gap is striking. LangMem's higher latencies stem from its on-demand extraction pipeline, which does more work at query time (agentmarketcap.ai analysis, April 2026)

The engine's relationship_depth parameter lets teams tune the tradeoff. Depth-1 retrieves directly related facts at roughly 50 ms. Depth-3 enables complex multi-hop reasoning while staying sub-200 ms thanks to graph pre-computation. Most production agents settle at depth-2.

Three search functions operate in concert: cosine semantic similarity, Okapi BM25 full-text, and breadth-first graph traversal. Reranking layers include Reciprocal Rank Fusion, Maximal Marginal Relevance, a graph-based episode-mentions reranker, a node-distance reranker, and cross-encoders. The bi-temporal model tracks four timestamps — created and expired in transactional time, valid and invalid in valid time — so agents never treat outdated facts as current. That temporal discipline matters in regulated settings: HIPAA-compliant clinical agents need precise patient-state tracking, and financial-advice agents require audit trails for regulatory review.

Reliability shows up in multi-agent scenarios too. When agent swarms collaborate, context sharing can become a broadcast storm. Zep's optimized context blocks synchronize state across agents with sub-200 ms updates, keeping distributed systems coherent without overwhelming network or LLM resources. Production agents implementing multiple memory types — episodic, semantic, procedural — complete multi-session tasks more reliably than single-context agents, even when the latter have longer context windows.

The tradeoffs are real. Graph construction is thorough but expensive; immediate post-ingestion retrieval can fail, with correct answers surfacing hours later after background processing completes. At 600,000-plus tokens per conversation for complex use cases, the knowledge graphs are not lightweight. Single-session assistant questions actually dip in performance — down 17.7 percent for GPT-4o and 9.06 percent for GPT-4o-mini — suggesting the temporal graph adds overhead that only pays off over longer horizons.

LangMem posts p50 search latency around 18 seconds and p95 near 60 seconds, ruling it out for interactive use. Mem0 manages p50 at 148 ms and p95 at 200 ms but lacks Zep's temporal reasoning and relationship awareness. Zep's position is distinct: managed-service simplicity with open-source transparency, plus capabilities that would require massive engineering investment to replicate. Teams choose it when agent reliability and latency directly impact business outcomes.

Why Enterprises Are Buying Now

The AI memory market is no longer a niche concern, it has become the central bottleneck in enterprise AI deployment. according to Mordor Intelligence, the 2025 market was $21.7 billion; Mordor Intelligence found the 2026 market at $27.9 billion; and Mordor Intelligence's figures put the 2031 forecast at $98.3 billion. The broader semiconductor market grows faster: World Semiconductor Trade Statistics' data shows 25 percent year-over-year growth in 2026 to roughly $975 billion, with the memory segment expanding at 30 percent. Investment banks estimate the 2026 memory market alone could exceed $440 billion. BofA reported the HBM slice hitting $54.6 billion in 2026, up 58 percent from the prior year, while Goldman Sachs forecasts HBM demand for custom ASIC-based AI chips will surge 82 percent and claim one-third of the market.

Three forces converge to push enterprises toward agent memory solutions. First, hyperscale buyers move from staged pilots to committed deployment cycles across cloud infrastructure. AMD and Meta Platforms announced a 6-gigawatt AMD GPU infrastructure partnership in February 2026, with first-phase shipments beginning in the second half of the year. NVIDIA's Vera Rubin platform ramped into full production in May 2026, delivering 10x agent throughput at scale and integrating HBM4 across its accelerator and CMX context memory storage architecture. Each rollout of that scale pulls in HBM, server DRAM, storage memory, and networking memory simultaneously, growth no longer confines to a single memory type.

Second, the workload mix shifts. AI Training and Model Development accounted for 45.5 percent of revenue in 2025, but AI Inference is projected to grow at the fastest pace (29.6 percent CAGR through 2031) as services move from periodic training events to constant user-facing workloads. Edge AI Systems expand at 29.6 percent CAGR, widening demand beyond centralized data centers into automotive, robotics, and industrial settings. Penguin Solutions stated in March 2026 that inference workloads are 30 percent compute-driven and 70 percent memory-driven, underscoring why memory bandwidth and accessible capacity have become the main design variables in deployment planning.

Third, traditional RAG pipelines hit hard limits. Users expect sub-second responses, yet such pipelines still take that long just for retrieval. Persistent KV cache growth in long-context agentic workloads drives memory content inflation across AI servers and racks. NVIDIA's March 2026 launch of BlueField-4 STX with CMX context memory storage and Penguin Solutions' production-ready CXL-based KV cache server with up to 11TB of CXL memory both signal that memory is now designed into inference infrastructure as a dedicated layer rather than a secondary component.

Enterprises respond. S&P Global Market Intelligence positions Zep as a de facto partner in the enterprise agent stack, citing enterprise focus plus bi-temporal context graphs that integrate raw episodic and derived semantic memory while tracking how facts evolve over time. Teams from startups to Fortune 500s — including Nasdaq-100 companies and Samsung — deploy it for agent memory, Graph RAG for dynamic data, and context retrieval and assembly.

The procurement dynamic has fundamentally shifted. Buyers no longer size memory as a support component adjustable late in the purchase cycle. Memory now shapes system cost, architecture choice, and procurement timing together. That favors suppliers offering dependable qualification, stable output, and roadmap continuity, and it creates a wider revenue base across HBM, server DRAM, LPDDR, and expansion memory inside the AI memory stack. For Zep, the timing aligns: its context graph approach addresses the exact layer — persistent, temporally-aware agent memory — that the market's infrastructure build-out now demands.

Rivals Respond: Pinecone and Weaviate Add Hybrid Search

Pinecone and Weaviate are not waiting for Zep's context graphs to rewrite the agent memory playbook. Both vector database incumbents shipped feature sets in the last quarter that directly target the hybrid retrieval and enterprise deployment gaps Graphiti exposes.

Pinecone's most visible move is Nexus, now in public preview. The company frames it as a unified control plane for its serverless and dedicated tiers, but the practical effect is a single API surface that lets teams mix semantic, keyword, and full-text search in one retrieval call, with built-in reranking to sort results. That hybrid workflow mirrors what Graphiti achieves through temporal graph traversal, except Pinecone delivers it without requiring developers to model entities, relationships, or time explicitly. The tradeoff: Pinecone's indexes stay stateless. They retrieve vectors fast (16 ms p50 on 10 million dense records, 8 ms p50 on sparse) but they do not track how a fact changed between Tuesday and Thursday.

Full-text search landed natively in Pinecone this cycle. Tokenization, stemming, and phrase matching now run inside the index itself, eliminating the Elasticsearch sidecar that many RAG stacks bolted on. Hybrid search combines that lexical signal with dense and sparse vectors in a single query, then applies Pinecone's reranker. The company publishes 12 ms p50 latency with metadata filters applied, and 31 ms p50 at 1 billion vectors. Those numbers beat Zep's sub-200 ms claim on raw retrieval, though Zep's benchmark includes graph assembly and provenance resolution, work Pinecone leaves to the application layer.

Weaviate took a different tack. Its 2026 hybrid search release positions the platform as an "enterprise semantic memory" layer, not just a vector store. The open-source core now ships with BM25 plus dense vector fusion out of the box, and the managed cloud tier adds multi-tenancy, role-based access, and SOC 2 coverage. Weaviate's Go runtime and self-hosted option give data-sovereignty teams a path Pinecone's closed-source SaaS cannot. The tradeoff: operational burden. Weaviate users run their own clusters or pay for managed; Pinecone users never touch a node.

Both vendors added bring-your-own-cloud deployment this year. Pinecone lets enterprises spin up a private region inside their own VPC, keeping data inside their security perimeter while Pinecone manages the control plane. Weaviate's BYOC model has existed longer and supports AWS, GCP, and Azure. For regulated industries, that parity matters more than raw latency.

Pinecone's hiring reinforces the enterprise push. The company lists four salaried roles on Zero G Talent with a median band of $208k, including a Senior/Staff Software Engineer for "Experience" at $190k–$270k and an Enterprise Account Executive at $300k–$350k. Weaviate does not publish roles on the board, but its 2026 hybrid search marketing targets the same buyer: platform teams building agentic workflows that need audit trails, not just similarity scores.

The vector DB layer commoditizes fast. Sub-20 ms retrieval at billion-vector scale is table stakes. Differentiation has shifted to what sits on top: graph logic, temporal reasoning, and the governance wrapper that lets a bank or hospital say "yes" to production. Zep's Graphiti open-source play forces Pinecone and Weaviate to decide whether they build graph primitives themselves or partner. So far, both are building.

Funding, Talent, and the LangChain Dependency

Zep AI's financing history shows conflicting signals across data sources, a common pattern for early-stage infrastructure companies that raise via convertible instruments before a priced round. Public records show a $500K round in April 2024 from Y Combinator; other sources track additional convertible notes. Neither source captures a Series A. The company's public positioning ("Agent memory, at enterprise scale") and its S&P Global Market Intelligence validation suggest it operates on a lean capital base relative to vector-database peers that have raised nine-figure sums.

Talent movement into Zep AI from Scale AI or Dropbox is not documented in primary sources. A research digest flags a "Zep AI hires Scale AI Dropbox roles" query, but the linked content resolves to Zep Inc. (the cleaning-products manufacturer acquired by Truelink Capital) not the AI agent memory platform at getzep.com. That Truelink acquisition brought a "seasoned consumer and industrial products CEO with a nearly three-decade track record" to the cleaning business in April 2026, a detail with no bearing on the AI startup. Scale AI added 12 roles in the past seven days, including a VP of Research ($453K–$567K), Director of Engineering for Physical AI ($302K–$378K), and multiple Tech Lead Manager positions ($252K–$362K). Those hires signal Scale AI's own scaling, not outbound flow to Zep. Without a verifiable announcement or LinkedIn migration pattern, any claim of key-person recruitment from Scale or Dropbox would be speculative.

The ecosystem move with clearer paperwork is LangChain's ongoing evolution toward LangGraph as the definitive agent runtime. For Zep, this matters because LangChain's memory abstraction layer is a primary integration surface: developers plug Zep's Graphiti-backed memory into LangChain agents. When LangChain rewires its memory interface, every downstream provider must adapt or risk breakage. LangChain's recent releases have signaled an enterprise RAG focus (structured retrieval, access control, and audit trails) aligning with Zep's compliance posture. But framework churn forces a choice: maintain compatibility shims or rewrite to the new LangGraph-native primitives. Zep's open-source Graphiti repository, at 29.6k GitHub stars and 35+ contributors, has the contributor bandwidth to track LangChain's pace, yet each framework churn cycle consumes engineering cycles that could go to core retrieval latency, currently 155 ms on LoCoMO, 162 ms on LongMemEval.

The funding gap, the absent talent pipeline evidence, and the framework dependency risk form a coherent picture: Zep AI is a technically differentiated, capital-efficient player whose enterprise traction outpaces its balance sheet. Its next inflection will likely come from either a priced round that lets it hire ahead of LangChain's migration calendar, or a strategic partnership that embeds Graphiti deeper into the agent runtime stack so framework churn becomes someone else's problem.

The Baseline Has Shifted

When Zep did so, it handed the market a temporal context graph that tracks facts across time, provenance, and validity, and dared competitors to match it. Pinecone and Weaviate responded with hybrid search. Enterprises responded with procurement cycles that now treat memory as architecture, not afterthought. The open-source core that began as Zep's engine is now the baseline any agent memory platform must clear: sub-200ms retrieval, bi-temporal reasoning, and provenance that survives audit. The commercial layer adds the governance wrapper. The race is no longer about who retrieves fastest. It's about who keeps the record straight when the rules change.

Working in frontier tech? Zero G Talent tracks the openings: see every open Scale AI role, browse frontier tech jobs, openings at Pinecone, and the people building the field.