The Founding ML Platforms Engineer at Cumulus Labs
Cumulus Labs is hiring one founding engineer to erase the need for dedicated ML platform teams at every company that ships production AI. The startup launched in 2025 as a YC W26 batch company, keeps two employees, and works from a San Francisco office. A LinkedIn post shows the open role: founding ML platforms engineer.
Cumulus turns raw GPU capacity into fast, cheap production AI. The hire will own the system that schedules workloads, splits GPU memory among tenants, and keeps a mixed multi-cloud fleet busy. That layer lets engineering teams ship AI without hiring a dedicated ML platform team.
The product wraps eight subsystems — gateway, router, cache, observability, evaluation, fine-tuning, custom hosting, and the Ion inference engine — into one platform. The orchestration piece, called Paladin, schedules training and inference across mixed clouds, allocates GPUs in fractions, and moves running jobs between chips with no downtime. Above it sit Talos, which builds and runs agents, and Ion, a C++ inference engine with custom CUDA kernels for NVIDIA Grace chips (GH200, GB200, GB300). The new engineer will extend Paladin itself, not a side tool. Beyond those layers, the OS handles on-prem GPU fleet management, bin-packing, and Kubernetes orchestration, per Respan.
The mandate is broad. The hire will write the GPU scheduler, build multi-tenant quotas and isolation, meter usage, and stand up a tenant-facing inference gateway. They will watch fleet metrics, logs, and traces, and debug from scheduling logic down to GPU memory and network wires. Cumulus expects real architecture calls, not ticket-queue tweaks. "You’ll own real infrastructure from day one, not tickets in a backlog," the LinkedIn posting said.
The founder wants someone who can walk into any part of a production system and improve it. Today the stack uses Go, Kubernetes, and Terraform, but the company cares more about how a candidate thinks than which languages they know.
This role proves the claim that separate ML platform teams are obsolete. A team shipping production AI normally stitches those fragmented functions from different vendors — a patchwork that breaks easily and costs more. Paladin folds those duties into one managed system. The engineer who takes the job writes the scheduler, gateway, and migration code that lets a customer skip building those pieces in-house.
Customers already run inference and agents on Cumulus’s platform, backed by YC and other investors. The founding engineer will maintain live infrastructure, not a prototype. The orchestration code must keep a multi-cloud GPU fleet at high utilization while moving jobs live and serving a tenant gateway.
The hire flips the staffing model. Instead of each customer stitching together five point solutions, Cumulus packs that work into one founding role inside its own walls. The customer buys the platform; the vendor employs the orchestration expert. The person filling this job writes the code that collapses five vendors into one.
How Does Ion Beat vLLM on Grace Hardware?
Cumulus Labs' Ion inference engine serves 30 to 50 percent more tokens per second than vLLM and SGLang on NVIDIA Grace and Blackwell chips, a figure the company posts on its own glossary page. That range anchors the founding ML platforms hire described above. The engineer inherits a mandate to keep those numbers real on production fleets.
The platform bet is simple: compress those fragmented orchestration duties into one operating system so teams ship AI without building such a team. None of that consolidation pays off if the lowest layer bleeds money. Ion is that base layer. Cumulus states on its site that Ion powers the custom hosting subsystem and the throughput claim that lets the platform price tokens competitively. It makes the higher layers viable.
Ion is a hardware-native C++ runtime with proprietary attention kernels. It targets NVIDIA’s GH200 Grace Hopper and extends to Blackwell. The core trick is IonAttention. Where vLLM’s PagedAttention manages memory across many attention requests, IonAttention overlaps prefill and decode through phantom-tile scheduling, pushes eager KV writeback through Hopper’s Tensor Memory Accelerator, and keeps a bounded working set that drains to LPDDR over NVLink-C2C. Those moves exploit specific silicon: NVLink-C2C coherency between CPU and GPU memory, Hopper’s TMA, and Blackwell’s larger HBM.
On the same GH200, IonAttention delivers that throughput uplift for the production workloads Cumulus serves, measured against stock vLLM and SGLang. "Our Ion engine beats stock vLLM and SGLang on the same chip. More requests per second, lower cost per token," Cumulus Labs said on its homepage.
The company’s YC profile repeats the figure and ties it to top-tier inference economics for customers.
Independent checks add weight. StartupHub.ai reported that Ion’s coherent CUDA graphs, eager KV writeback, and phantom-tile scheduling deliver 7,167 tokens per second on a single chip with 12.5-second cold starts, benchmarks the site calls outperformers of every major competitor. Agent-Wars.com gave a model-specific data point: 7,167 tokens per second on Qwen2.5-7B on one GH200, about double what leading inference providers deliver, with per-second billing and no cold starts. The prudent conclusion: the engine hits its marks on Grace silicon, though all numbers come from friendly sources.
The numbers carry hard limits. Ion is not a drop-in replacement for vLLM everywhere. Cumulus’ glossary notes that for batch size one on tiny models, warp specialization overhead isn’t worth it. On non-Hopper hardware, IonAttention won’t build. An April 2026 benchmark roundup from IoT Digital Twin PLM found vLLM wins on raw throughput and ease of use, TGI dominates cloud-native deployments, and SGLang fits structured output at scale. Ion’s edge is narrow but deep: it lives on Grace and Blackwell, and the hired engineer must protect that lane.
The throughput claim funds the unified platform’s pitch. Cumulus says it saves customers 50 to 70 percent by charging for physical resources used, and the faster inference with ultra-low cold starts removes the debugging tax that fragmented stacks impose. The Cumulus Router selects Ion when the workload fits and dispatches elsewhere when it doesn’t. That selective trust depends on the engine hitting its marks on every Grace and Blackwell node the company runs.
Looking forward, an April 2026 preview from IoT Digital Twin PLM points to Q3 2026 benchmarks on NVIDIA Blackwell B200, first single-node then multi-node NVL72. The founding ML platforms engineer will need to reproduce that token-per-second gain on that newer silicon, or the consolidation story loses its sharpest evidence. The role is about shipping a kernel that beats vLLM on the chips Cumulus actually owns.
Why Fragmented Inference Stacks Are Being Replaced
Clusters sit idle up to six in ten times while AI teams burn weeks managing failures and predicting compute needs, a LinkedIn post from Cumulus shows. That waste is not a hardware glitch. It is the tax of running production inference across a patchwork of point solutions.
The fragmented approach forces engineering organizations to stand up specialized orchestration headcount just to keep the lights on. YC’s company page for Cumulus calls this pattern "brittle, expensive, and a common reason enterprises fail with AI." A market shift is now visible: teams want production AI without that specialized headcount, and that demand pulls infrastructure hires like the founding platforms engineer posting Cumulus placed on LinkedIn.
Each added vendor is a blind spot. Cumulus’s homepage said, "Production is brittle. Five vendors. Five blind spots." The failure mode is concrete: one outage took a product down for six hours when teams wrote failover paths but never tested them, as the company’s site illustrates with a three-provider setup across reasoning, summarization, and classification. The site reduces the argument to a line: "5 → 1 → Cumulus."
Infrastructure hell compounds the cost. Engineers spend weeks configuring Kubernetes, debugging out-of-memory errors, and managing failovers instead of building models, YC’s page said. Google built Kubernetes for stateless pods that reschedule on hardware death, not for stateful GPU inference jobs that hate cold starts, a point made in a Kubernetes community video. The video noted the project drew inspiration from Google’s Borg, a system for stateless services, and that maintainers only started addressing ML training and serving about three years prior. Point-solution projects like KServe (just serving) and LND (networking layer) each solve one layer, leaving teams to wire them. That wiring is the internal platform team’s full-time job.
The open source crowd bolted on a Gateway API inference extension just to get a router pod inside the cluster. Fragmentation worsens at multi-agent scale. A 2025–2026 analysis on website-files.com warned that isolated tools with custom integrations turn fatal at that scale, and three protocols emerged to fix interoperability. Yet protocols still leave teams integrating point solutions.
The money bleed is measurable. Idle GPUs run at only 30 to 40 percent utilization, sometimes as low as four in ten, because scaling is unpredictable. Runway burns two to three times faster than planned as GPU bills spiral. Cold starts lag 10 to 30 seconds and kill user experience. Vendor lock-in makes switching near impossible.
Those numbers explain why engineering leaders now ask a different question. Instead of "which routing vendor do we pair with which observability tool," they want a single API that removes the ML platform team from the org chart. Cumulus’s about page states: "AI teams should not have to assemble seven vendors plus a cloud GPU rental to ship an inference product." The platform’s parent claim is that teams ship AI in production without staffing that function.
Consolidation into one orchestration layer is the countermove. A LinkedIn post from Cumulus described "one orchestration layer, zero vendor lock-in" as its answer. The post also promised scale-to-zero when idle and scale-to-100 GPUs when busy, with jobs charged by physical GPU fraction consumed so teams never pay for idle capacity. That economic hook targets the exact waste lines above.
This market pull directly drives the hiring pattern. When a company advertises a founding platforms engineer role, as Cumulus did in the LinkedIn posting with $150k–$300k pay, it is not building an ML platform team for customers — it is selling the absence of that team to customers. The role exists to make the unified stack real.
The next time a CTO reviews a stack diagram with eight boxes, they will ask why one of those boxes cannot be the whole page.
Cumulus Labs’ Hire Redraws the Infrastructure Job Map
Cumulus sells a single OpenAI-compatible API that fuses gateway, router, semantic cache, observability, evaluation, one-click LoRA fine-tuning, and inference serving. Veer Shah and Suryaa Rajinikanth (ex-TensorDock lead engineer) started the company. That pitch reshapes who gets hired for AI infrastructure and who loses budget. Two groups feel it first: infrastructure engineers who built careers stitching point tools, and the point-tool vendors themselves.
Infra engineers pulled toward the unified stack
A machine learning infrastructure engineer builds the platforms, pipelines, and compute systems that let data science teams deploy models at scale. The job market for that skill set is large, with postings in the tens of thousands and pay at major labs reaching six figures. The table below shows where the demand concentrates:
| Source | ML/AI infra job count | Context |
|---|---|---|
| Indeed.com | 35,744 ML Infrastructure Engineer jobs | live postings |
| Robert Half | 49,200 AI/ML/data science postings | 2025, +163% YoY |
| Zero G Talent (Databricks) | 398 roles | median $250k |
| Zero G Talent (Anthropic) | 343 roles | median $405k |
Those figures signal companies still pay for infra talent, but Cumulus argues the talent should build one platform instead of seven.
Cumulus’s site frames the old way as disconnected point solutions rented from separate shops. Its answer collapses those into one platform. For an infra engineer, the founding hire at Cumulus is a chance to own that collapse rather than maintain it. The product lets engineering teams ship AI in production without an internal platform team, removing the role many infra engineers filled. The same engineer now chooses: build the unified OS, or watch the internal platform team get dissolved.
The unified bet remains unproven — Respan noted benchmarks are self-reported without independent validation — yet the hire alone redirects infra talent and vendor strategy. Aventure's analysis says Cumulus integrates inference infrastructure that teams usually assemble from seven or more vendors into one platform. Infra engineers tired of paging on fragmented alerts can instead write the routing logic once.
Point-solution vendors on the defensive
Cumulus competes with the point vendors named below, plus cloud GPU providers. An April 2026 Stack-Based roundup named Arize Phoenix as best observability layer, BentoML as best inference serving platform, and Orq.ai as best provider-routing layer. Each wins by solving a different production problem well. Cumulus bundles all three functions behind one API. The fragmentation these vendors rely on is exactly what enterprises cite as a reason AI projects fail, per Aventure's analysis of Cumulus.
Respan's March 2026 market map shows the tiny startup fighting well-funded Modal, Replicate, and RunPod. That size gap means point vendors are not extinct. Stack-Based noted mature teams may use those three tools together because they solve different layers. But Cumulus claims the cost savings and throughput gains described earlier through pay-per-compute and its Ion engine. Those numbers, self-reported, still pressure single-layer vendors to justify separate line items.
The hire touches vendors by forcing a choice: integrate or be replaced. A point vendor selling only routing now faces a platform that routes, observes, evaluates, and fine-tunes in the same call, backed by the fleet-management and bin-packing features described above. That hire will reduce a rival dashboard to a config flag.
Working in AI? Zero G Talent tracks the openings: see every open Databricks role, browse AI jobs, openings at Anthropic, and the people building the field.