Distributed Llm Inference Engineer Jobs

About the role As a Distributed LLM Inference Engineer, you will help systems and optimizations that push the boundaries

…THE ROLE We’re seeking an inference systems engineer to build the distributed serving and high-performance networking layer behind

You're experienced with modern inference systems like TGI, vLLM, TensorRT-LLM, and Optimum, and comfortable creating custom tooling for testing and optimization

Role Mission As HAI's LLM Inference Engineer, you will own the serving infrastructure that determines whether our breakthrough healthcare

…a strong blend of infrastructure engineering, production ML systems, LLM/LVM inference, evaluation harnesses, and inference

…of ML platform, infrastructure, and distributed-systems engineers working alongside the teams that build our ML internals and

Qualifications CUDA + GPU inference optimization vLLM, SGLang, or TensorRT-LLM experience KV caching, paged attention, batching

Inference systems are highly performance sensitive distributed systems.

Familiarity with LLM inference optimization, batching strategies, and multi-accelerator deployments is highly encouraged but not strictly necessary.

As a Software Engineer on Inference, you’ll help define Sierra’s inference architecture across both self-hosted models and third-party inference providers

…vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and

We are looking for high-agency engineers who are excited to work on powering model inference at enterprise scale.

platforms Benchmark, fine-tune, and accelerate inference engines (e.g., SGLang, vLLM, TensorRT-LLM) Develop custom

Shape the core inference backbone that powers Together AI’s frontier models.

…routing, or traffic management systems LLM inference optimization, batching