Job Description
This is an evals role. You will spend most of your time deciding whether an AI system actually did the job, then building the datasets, graders, and checks that make that call hold up. We are a small YC team. You work with the founders. If you want a 9-to-5 or a remote-first job, skip this. If you want to own how we measure things, keep reading.
What you will do
- Turn real failures into tasks we can score
- Build datasets, gold labels, graders, and human review loops
- Know when a metric is lying, and say so
- Ship changes to the eval setup
- Find LLM incapabilities in production systems
You will like this if
- You have shipped agents. Python is home.
- You have built evals, graders, or datasets for LLMs or agents
- You have a story about a metric that lied
- You can sit with a lot of traces and find the few that matter
Interview Process
We keep this short and we move in about two weeks.
1. Intro call (20-30 min)
Talk through what you have built. We will ask about one eval, dataset, or agent system you actually shipped. No slides.
2. Work sample (2-3 hours)
A small, real eval problem: look at traces or a task set, say what is broken, and propose a grader or a few new cases. Judgment over polish.
3. Offer
If it is a yes on both sides, we send an offer quickly. You will meet both founders before that happens.
We are looking for people who have built something, can explain why a score is trustworthy,
Optimize Your Resume for This Job
Get a match score and see exactly which keywords you're missing
Job Details
- Category
- Software
- Employment Type
- Full Time
- Location
- Bengaluru, India (Hybrid)
- Posted
- Compensation
- ₹1,200,000 - ₹3,600,000 per year
About Ressl AI
We automate the back office so field crews can focus on the actual work. Trades businesses (HVAC, plumbing, electrical, collision repair, roofing) run on legacy software and "glue" employees: people whose entire job is clicking through systems, copy-pasting between tools, and managing inbound volume across channels. A collision shop has one person generating estimates in CCCOne, another chasing parts vendors, another answering calls and Yelp messages, another doing invoices in QuickBooks. None of that requires a human. We build proactive background agents that run these operations autonomously -- quoting, estimating, procurement, insurance coordination, scheduling, follow-ups -- on top of the software these businesses already own. We started as an AI consulting firm talking to businesses across every industry. After a month, the pattern was obvious - trades and field services had the most "glue work" and the least tech sophistication to solve it themselves. Every shop we walked into had 2–4 people doing work that software should be delivering as an outcome, not just enabling. Our GTM is direct to operators and through PE-backed roll-ups, who are actively looking for margin improvements across their portfolio companies. If this is of interest, please reach out [email protected]
More Roles at Ressl AI
Similar Software Roles
Found this role interesting?