Skip to main content

Software Engineer, Entity Resolution

CrustdataEngineeringEngineering
Pay
$150K–$300K
per year
Work mode
On-site
Full Time
Education
Master's or PhD

San Francisco, CA at a glance

Rent
#2 of 51
$2,680/mo+46% vs US avg
Weather
#17 of 51
295 mild days0 hot · 0 cold
Income tax
#1 of 51
13.3% top rateCalifornia

What you need

  • PhD or research master's in CS, ML, data management or related field
  • Strong Python and comfort with a systems language
  • Thesis or first-author papers on entity resolution or similar topics
  • Want methods running in production on a billion records
  • Care about correctness and measure it

What you'll do

  • Build candidate generation using blocking and similarity search at scale
  • Create matching models with calibrated confidence for disagreeing records
  • Turn pairwise scores into one canonical record per person and company
  • Decide which source to trust for each field with provenance
  • Build labeled sets and metrics for matching with incomplete ground truth

Crustdata builds web scale person and company search for AI agents. We cover 1B+ people, we're profitable, and we got to $13M ARR in under 1.5 years. We're backed by SV Angel, YC and General Catalyst.

The entity resolution team turns what we index into one canonical, evidence-backed record for each person and company, linking records across social networks, job boards, web directories such as school and institution pages, and proof-of-work platforms. AI agents query that data directly.

People and companies have no shared id across these sources. You'll work on the methods that decide which records get compared, which ones match, and how matches become entities, and you'll ship them to production.

What you'll do

  • Candidate generation: blocking and similarity search (LSH, sketching, learned embeddings) that finds the likely matches for each record among billions, with the recall you give up measured.
  • Matching: models that decide whether two records are the same person or company, with calibrated confidence, when names, employers, schools, handles and dates disagree.
  • Clustering: turn pairwise scores into one record per person and company, and handle merges and splits as new data arrives.
  • Truth discovery: decide which source to trust for each field, using reliability, recency and corroboration, with provenance on each value.
  • Evals: labeled sets and metrics for matching and merging when ground truth is incomplete, and lineage that shows which source or stage caused an error.

What we're looking for

  • A PhD or research master's in CS, ML, data management or a related field, finished or finishing in 2026, with a thesis or first-author papers on topics like similarity search and hashing, entity resolution or record linkage, information extraction, knowledge graphs, data integration or large-scale ML.
  • Strong Python and comfort with a systems language. Code of your own that others can read, like a paper's released repo or an open-source project.
  • You want your methods running in production on a billion records.
  • You care about correctness and measure it.

Bonus

  • LSH, MinHash, sketching or approximate nearest neighbor search at scale.
  • Weak supervision, active learning or LLM-based labeling for matching.
  • Spark, Ray or another distributed data framework.
  • Time at a startup or on a small team.

Example projects

  • Generate candidate pairs across 1B+ person records with a recall you can measure and defend.
  • Link one person's profiles across a social network, a job board, a school directory and a proof-of-work platform when no field matches exactly.
  • Build the eval set that estimates our person-matching error rate without labeling everything.
  • Resolve each company to one record across domains, name variants, subsidiaries and acquisitions.

Optimize your resume for this job

Get a match score and the keywords you're missing

Optimize resume

About Crustdata

Crustdata builds a robust and easy-to-use set of APIs that solves the problem of getting accurate, fresh data on companies and people.

Similar roles

Software Engineer, Entity Resolution
$150K–$300K · Crustdata
Apply