Agent Evaluation Engineer Jobs in New York

Hands-on experience building, evaluating, or improving LLM-based applications or agents, including tool use and multi-step workflows.

As an Agent Engineer, you will work directly with customers who have already purchased Profound’s Agents product and need help deploying it effectively to

AIS spans multiple workstreams — agent evaluation and oversight, orchestration and tool-use infrastructure, model and systems optimization, and applied research

…more relevant areas, such as agent harnesses, tool use, context engineering, model evaluation, browser automation,

Design and engineer the core agentic logic governing agent behavior, applying engineering principles to optimize quality, reliability, and correctness

Role Overview As a Software Engineer, Agents, you'll build the systems that make our AI agents indispensable to legal professionals

We're seeking a Senior Software Engineer to build the AI agents patients actually talk to.

Experience evaluating LLM agents, multi-agent systems, synthetic populations, recommender systems, probabilistic models, simulations, or decision-support

We’re looking for research engineers who can build reliable data and evaluation systems, question what a metric actually tells us, and work closely with researchers

…all in one place, alongside agents that carry the load with them.

…company gets work done — agents and automations people trust.

…We’re looking for applied AI engineers to help bring Codex agents from impressive demos to dependable tools.

criteria, while protecting evaluators' ability to publish their own conclusions Scope evaluations with researchers and engineers, turning evaluators' questions