Remote Llm Evaluation Jobs

…AI quality, and build the evaluation practices

Experience with synthetic data, LLM evaluations, automated raters, or preference data is a plus.

Familiarity with LLM observability, evaluation frameworks, cost optimization, and usage analytics.

…in Python and experience with LLM APIs; able to build and ship evaluation tooling independently.

…ones the team uses to evaluate everything that comes after.

…delivering measurable impact Hands-on experience evaluating AI products, with demonstrated ability to translate evaluation findings into meaningful

…infrastructure engineer to own model evaluation at Runway, end to end.

…and iterate on classifiers that evaluate

Openbenchmarks is the evaluation infrastructure for agents: domain-specific, reproducible evaluations that help agents pick tools with confidence.

You build the LLM agents that turn model outputs into language a person can act on, and the evaluation harnesses that keep them honest, because a fabricated number

…ones the team uses to evaluate everything that comes after.

…with the latest research in LLMs, RAG, and related fields.

…the customer's ERP, and the evaluation

…the customer's ERP, and the evaluation

…and curation, experiment design, and evaluation.