Inference Optimization Engineer Jobs in San Francisco, CA

Evaluate new inference runtimes, compilers, frameworks, optimization techniques, and accelerator hardware.

About the Role Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing

Responsibilities Build inference services with smart batching and caching Optimize kernels, tokenization

About the Role We're hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our models

Optimize inference performance.

Qualifications CUDA + GPU inference optimization vLLM, SGLang, or TensorRT-LLM experience KV caching, paged attention, batching

We're looking for performance engineers who treat correctness as part of performance.

AI models Optimizing model serving infrastructure for high throughput and low latency at scale Developing and integrating advanced inference optimization

data centers and model engine pods.

About the Team We’re hiring a Developer Productivity engineer to support OpenAI’s Inference Runtime teams.

The toolchains, development environments, and CI/CD that let inference engineers build, test, and ship across every accelerator platform.

We build and run the inference engine behind every Perplexity query and deploy dozens of model architectures at scale with tight latency and cost budgets.

Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft.

lives, how caches are placed, how fast the system reacts to demand, and the protocols that keep the control plane and the inference engines in sync Partner

Inference serves hundreds of thousands of customers every day, and the size & span of the inference fleet requires sophisticated routing, scaling, and networking