Anthropic's 339 Open Roles Mask the Engineer Auditors Now Require

A Prompt Score Doesn’t Capture a Running Agent

NVIDIA's developer blog framed this in May 2026 with a blunt premise: when an AI system plans a task, calls external tools, and adapts to failures inside a live environment, grading a single prompt response proves little. The test surface has moved from a static reply to a running process, shifting the lens to the trajectory—the end-to-end sequence of reasoning, tool calls, and environment observations. Anthropic published a parallel engineering note in January 2026 explaining that earlier LLMs relied on single-turn, non-agentic evals, while multi-turn tests became standard as agents left the demo stage.

You can describe single-turn evaluation in three moves: a prompt goes in, a response comes out, grading logic scores it. That suited models that answered questions or drafted text. An agent evaluation tests the behavior of a system operating end-to-end—planning, calling tools, handling uncertainty, and completing real workflows in a dynamic environment. We no longer measure the final answer but the path the agent takes through a stateful world. This shift has turned agent evaluation into a standalone engineering field, with labs and vendors shipping practical frameworks and institutions like Brookings opening governance debates over how to standardize measurement.

Anthropic's engineering team said the traits that make agents useful also make them hard to test. The system modifies state across many turns and adapts as it goes. Mistakes propagate and compound. A model-level check alone misses how those issues appear in execution.

Anthropic's engineering note states that the capabilities that make agents useful also make them difficult to evaluate.

NVIDIA's blog gave a split example: two agents can return the same answer while behaving nothing alike. One uses three precise tool calls; another thrashes through dozens of irrelevant steps. Teams shift from measuring knowledge to measuring outcomes. Technical tracking follows: they score whether the agent finished the job, called the right tools, and avoided needless loops.

Foundation model benchmarks like MMLU and HumanEval measure reasoning in isolation. NVIDIA's May post said a high MMLU score is a prerequisite, not a guarantee of a reliable agent. Most production agents live or die on tool use—APIs, databases, search—not on phrasing. Testing a model or a tool by itself builds false confidence when the combined workflow stays brittle.

Anthropic's January guidance says agent evaluations typically combine three grader types: code-based, model-based, and human. A transcript—also called a trace or trajectory—records a trial completely: outputs, tool calls, reasoning, intermediate results. The outcome is the final state in the environment. Capability evals ask what the agent does well; regression evals ask whether it still handles tasks it used to.

Uncertainty handling widens the gap further. The system's tool calls, intermediate state, memory, and multi-agent handoffs all become points where failures emerge. Lakera red-teamed NVIDIA's NeMo Agent Toolkit in April 2026 and found failures rarely live in a single component. Risk propagates across steps, so evaluation must treat the agent as a system rather than a weights file.

Teams that build evals early convert failures into test cases. Anthropic's post said evals make behavioral changes visible before users hit them, and the value compounds across the agent lifecycle. Once evals exist, teams get baselines and regression tests free: they track latency, token use, cost per task, and error rates on a static task bank. Teams without evals sink into reactive loops—patch one break, spawn another, unable to tell signal from noise.

AI agent evaluation remains a new and fast-changing field. As agents take longer tasks and coordinate in multi-agent systems, the techniques will keep adapting. The labs now draw a clear line: test the running system, not the parameters.

NVIDIA and Anthropic Build Test Harnesses

A NVIDIA Developer video featuring a Prime Intellect engineer in 2026 noted that agents now burn tokens by the million on runs far longer than the five-minute cap common in 2024, yet old single-turn grades don't catch broken tool calls. NVIDIA's May 19, 2026 developer post laid out five practical tips for evaluating AI agents as production systems, one of the first vendor moves to package agent testing as a repeatable engineering step. The tips start with design. Rather than retrofit observability, NVIDIA wrote, treat evaluation as part of agent design. That flips the usual order: you build the test harness before the agent ships, not after a production outage forces it.

Second, evaluation moves into dynamic environments. NVIDIA pushes tests beyond static prompts to GAIA for real-world assistance, SWE-bench for resolving GitHub issues, and WebArena for web-based task execution. Those benchmarks force the agent to act inside a changing state, not just emit text.

Third, track hard numbers. The post specifies Task Success Rate (TSR) to measure intent resolution, Tool Call Accuracy to ensure precision in function calling, and Trajectory Efficiency to spot redundant steps. A July 2026 Prime Intellect talk noted that just because a model accepts a million-token context doesn't mean it can reason across the whole span, producing "context rot" in long agents. Trajectory Efficiency directly flags the wasted steps that symptom creates.

Fourth, don't rebuild your stack. NVIDIA NeMo Agent Toolkit plugs into existing agent frameworks and adds evaluation, optimization, and observability without a full rewrite. Teams running Cursor-style agents or internal tool bots can attach metrics without freezing feature work.

A July 1 NVIDIA reinforcement learning post added a fifth tip that rounds out the set: start with evaluation before training. Run the current model on a held-out task set, inspect failures, profile the verifier or reward function before updating weights. The same post described a seven-part RL loop—policy model, task, action, environment, verifier, rollouts, policy update—that makes the verifier the backbone of measurement. The lab's Nemotron 3 Super model used this loop across 21 NeMo Gym verifiers and 37 datasets, generating about 1.2 million environment rollouts—roughly the population of a mid-size city.

The Four Pillars framework, circulated by MachineLearningMastery, names the categories these metrics serve: task success, tool quality, reasoning coherence, and cost efficiency. NVIDIA's tracked metrics map to three of the four directly. The alignment looks like this:

NVIDIA extended the framework beyond metrics into supply-chain checks. The company's July 9, 2026 GitHub skills repo ships official verified skills for agents, each with a detached OMS signature. A successful verification confirms the skill contents have not been modified since NVIDIA signed them. That closes a gap where an agent loads a corrupted instruction set and passes every behavior test yet still misbehaves.

The agent flywheel NVIDIA describes makes production failures into evals, evals into environments, environments into rewards, rewards into model improvements. For a team shipping agents this week, the move is concrete: pick a pillar set, wire TSR and Tool Call Accuracy into your logs on day one, and treat every failed run as a new test case. The scores those logs produce are exactly what regulators have started to question.

Can Regulators Trust Vendor Benchmarks?

On October 14, 2025, more than 40 experts from government, academia, civil society, and industry gathered in person and online at the Brookings Institution in Washington, D.C., for a workshop that Carnegie Mellon University, the Brookings Institution, and the University of California, Berkeley jointly hosted. Their stated trigger: we cannot govern what we cannot measure. The meeting was the first in a series aimed at closing the gap between controlled benchmark scores and the behavior agentic systems show in real deployment.

Brookings published the brief How can we best evaluate agentic AI? arguing that agentic AI evolves faster than the evaluation mechanisms teams built to assess it. The policy reaction centers on a hard truth: meaningful governance requires evaluation frameworks that reach past model-centric benchmarks to cover system behavior, effects on workers and communities, and risks to the public. Policy thinkers say the current measurement stack lags and propose expert-driven governance.

Definitions block the first step. Brookings notes no consensus on what precisely defines "agentic AI." The institution notes that practitioners describe agentic systems as models that perceive context, set and update goals, plan, and act through tools or environments, yet no single definition wins universal acceptance. That vacuum makes it harder to ground measurement frameworks and the policy decisions that hang on them. Brookings suggests treating agency as a spectrum, not a binary switch; this yields more useful analysis than forcing one definition.

Existing evaluation practice fails the agentic test on multiple fronts. Agentic systems operate through sustained interaction with environments and users, so contained benchmarks cannot fully characterize their behavior. They act stochastically, shifting across runs, environments, and time horizons. A single accuracy score on a static benchmark misses this variability. General-purpose benchmarks ignore domain-specific performance. Training data contamination undermines generalizability. Competitive evaluation regimes may push teams to overfit to benchmark tasks. Accuracy-focused metrics hide cost-performance tradeoffs. Evaluation itself burns resources.

The measurement gap hits deployers directly. Organizations struggle to ship systems whose behavior they cannot reliably predict. Without reliable evaluation, they cannot assess risks around security, liability, misuse, and systemic harms. Brookings says today's agents fly like the first wobbly aircraft: impressive demonstrations under controlled conditions, but far from the reliability needed for sustained real-world use in higher-risk domains. Agentic systems that ace model-level safety benchmarks often break under red teaming or adversarial testing.

Regulators have noticed. The Brookings findings signal that regulators in the EU, U.S., and Singapore now move to require documented pre-deployment evaluation of high-risk AI systems; benchmark-only evidence won't suffice for agentic deployments. That risks non-compliance for organizations that lean solely on vendor scores. The paper aligns with NIST's AI Risk Management Framework, the EU AI Act's emerging conformity standards, and Singapore's IMDA agentic AI governance framework from 2026. As high-risk use-case registrations pile up under the EU AI Act, regulators may soon enforce against deployments lacking behavioral tests.

The brief outlines four paths forward. First, experts lack a shared definition of an AI agent. Second, measurement shows structural gaps for agentic AI. Third, research must target effective evaluation and governance. Fourth, independent safety assessments must gain credibility.

Brookings, CMU, and UC Berkeley committed over 2026 to a multistakeholder workshop series. They plan a Carnegie Mellon convening in spring 2026 and a UC Berkeley workshop in fall 2026. Target outcomes include a research roadmap for agentic AI measurement, an evidence-informed policy and governance framework, and a collaborative community spanning academia, industry, government, and civil society. The research calls for frameworks that prioritize predictive validity: test results must forecast real-world system behavior, not just in-lab performance.

Brookings says deployed agents already run for hours and will likely run for weeks within a few years. Systematic methods to evaluate long-term reliability, drift, and failure over those horizons remain unexplored. Research is also needed on individual agent behavior inside networks of agents and the emergent properties of multi-agent systems. They prioritize pulling deployers, legal scholars, and policymakers into later workshops.

The governance reaction is clear: vendor benchmarks alone will not satisfy auditors or regulators. The next concrete step is the spring 2026 CMU convening, where the research roadmap will either gain teeth or reveal how wide the expert divide remains.

The New Hiring Market for Agent Evaluators

Galileo.ai's December 7, 2025 career guide says most agent evaluation engineers didn't start in this role because it barely existed two years ago. The function is standing up inside software and IT organizations as a production discipline, not a research sideline.

The shift matters for how you staff and structure teams shipping autonomous workflows. This section maps the constituencies now owning agentic evaluation and draws a hard line between this work and traditional deterministic QA.

Galileo.ai's guide defines the AI agent evaluation engineer as someone who assesses performance, safety, and reliability through adversarial testing and continuous production monitoring. The role sits at the intersection of adversarial security testing, ML engineering, and production systems reliability. A LinkedIn post from Software Testing Studio calls the role the next step for QA pros who design, build, and maintain systems that test and monitor agent behavior at scale. Titles range from AI Evaluation Engineer to Responsible AI Engineer.

The feeder pools are concrete. QA engineers can move in, but they must swap pass/fail assertions for statistical grading, according to the Software Testing Studio post. Galileo.ai notes that ML engineers with production experience make the leap successfully, security engineers bring a red-team mindset that transfers almost directly, and research engineers translate academic evaluation work into live systems. DevOpsSchool's role blueprint lists common entry paths: backend or full-stack software engineers with LLM feature work, applied ML engineers, RAG-focused data engineers, MLOps or platform engineers with LLMOps focus, and search relevance engineers. The MLOps constituency is central because traditional monitoring wasn't built for systems that make autonomous decisions across multi-step workflows.

Pay reflects scarcity. The table below folds the salary data into one view.

Zero G Talent's first-party board data shows Anthropic carried 339 open roles and logged 30 new ones in the past 7 days, a density that signals how hard frontier labs pull on adjacent engineering talent.

Hiring bars are specific. JobsByCulture lists five skills that get you in: strong Python, hands-on use of an eval framework such as Inspect, Promptfoo, Braintrust, Arize, or LangSmith, statistical literacy for A/B and significance testing, and judgment on tradeoffs. DevOpsSchool adds that interviews should probe production-grade software engineering, agentic design patterns, tool integration safety, RAG quality, evaluation discipline, security awareness, and cross-functional communication.

This discipline is not traditional software QA. Galileo.ai states the difference plainly: traditional QA engineers validate deterministic software, while agent evaluation engineers examine systems that dynamically select tools, execute multi-step reasoning chains, and make decisions with real-world consequences. The LinkedIn post frames agents as non-deterministic, context-aware, probabilistic, and continuously evolving, which makes expected-versus-actual testing insufficient. The same prompt can produce different responses across model versions.

The work itself spans seven interconnected responsibilities Galileo.ai assigns to the role: building adversarial tests and red-team protocols, implementing eval frameworks that measure task completion and tool accuracy, monitoring production for drift, analyzing failures across multi-turn traces, laying down guardrails, collaborating with business teams, and catching policy violations where outputs are technically correct but unsafe. An AI evals engineer owns the systems that decide whether an agent ships, per JobsByCulture.

McKinsey describes an agentic team as a smaller multidisciplinary human group that supervises AI workflows across the value chain. That fits: evaluation engineers don't operate alone. They sit beside AI agent engineers who design, build, and run the agents, and beside MLOps staff who keep the runtime stable. DevOpsSchool predicts stronger runtime control in coming years, moving from free-form loops to constrained execution with typed plans and validated tool calls.

The role is expected to grow rapidly as agents become core business systems. DevOpsSchool's mid-term view (2–5 years) foresees standardized platforms, heavier governance requirements, and demand for measurable business outcomes instead of AI novelty. QA engineers who once wrote binary test cases now write graded rubrics and watch live drift; teams that treat agent eval as a checkbox will ship hallucinations to customers.

Working in AI? Zero G Talent tracks the openings: browse AI jobs, openings at Anthropic, and the people building the field.