Distributed Llm Inference Engineer Jobs in San Francisco, CA

About the role As a Distributed LLM Inference Engineer, you will help systems and optimizations that push the boundaries

You're experienced with modern inference systems like TGI, vLLM, TensorRT-LLM, and Optimum, and comfortable creating custom tooling for testing and optimization

…directly with research scientists and engineers to make new visual models fast and efficient at inference time.

…of ML platform, infrastructure, and distributed-systems engineers working alongside the teams that build our ML internals and

Qualifications CUDA + GPU inference optimization vLLM, SGLang, or TensorRT-LLM experience KV caching, paged attention, batching

• Hands-on experience with modern inference engines and optimization stacks (e.g., vLLM, TensorRT-LLM, lmd, or Dynamo).

…a small, fast-growing team of engineers in San Francisco powering Fortune 100 enterprises, YC startups, public investment firms, and growth-stage

As a Software Engineer on Inference, you’ll help define Sierra’s inference architecture across both self-hosted models and third-party inference providers

…Role We're hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our models

…vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and

infrastructure or general LLM serving stacks.

…to a Senior Staff Software Engineer.

…is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing

…closely with safety researchers and inference engineers to bring new classifiers and novel classes of ML defenses to production.

We are looking for high-agency engineers who are excited to work on powering model inference at enterprise scale.