Llm Serving Jobs in San Francisco, CA

Requirements • Hands-on LLM fine-tuning and post-training, with strong Python and PyTorch or JAX and practical distributed-training experience.

evaluation systems, including LLM-as-judge, human preference, and task-specific benchmarks Experience serving and optimizing open-weight models

Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.

You're experienced with modern inference systems like TGI, vLLM, TensorRT-LLM, and Optimum, and comfortable creating custom tooling for testing and optimization

It also serves as the underlying training framework for the data quality evaluation pipeline.

• Work with the LLM researcher and harness engineer to bring structured observations and model responses into robots and products.

Experience building model-serving, inference, fine-tuning, or developer-platform infrastructure.

The same platform also serves research workloads, so demand comes from multiple user surfaces, often at the same time.

LLM-as-judge on every event is easy.

Experience serving nonprofits or mission-driven organizations.

Experience developing, deploying, or managing conversational AI agents, LLM-powered applications, or workflow automation products.

Our stack includes Next.js, React, Node.js, OpenSearch, AWS, SQS, Spark, and a range of inference providers and LLMs.

Shipped an LLM-based support or operations agent in production and understand the trust, escalation, and quality-measurement problems that come with it

Understand modern LLM inference, including prefill and decode behavior, batching, KV-cache tradeoffs, and model parallelism.

We build the infrastructure that enables our customers to serve, scale, and optimize frontier models with enterprise-grade reliability and performance.