Tech Lead Manager, Inference

You’ll lead the team that owns Luma’s entire inference serving stack — routing, scheduling, and fleet-wide orchestration across thousands of GPUs, multiple clouds, and hardware vendors — where all of Luma’s compute meets all of its users. This is a hands-on tech-lead-manager role. It’s leadership by shipping: at least half your time stays hands-on in the serving stack, alongside hiring, growing the team, and setting technical direction. It fits someone who’s operated inference fleets at the thousands-of-GPUs scale and genuinely wants to keep building, not move into pure management. If you want a hands-off management seat, this isn’t it. What You’ll Own Spend at least half your time hands-on: architect and build core platform components, own the hardest design decisions, and debug the toughest incidents yourself. Lead, grow, and develop the inference engineering team — hiring, coaching, on-call, incident response, capacity planning, and postmortems. Set the technical roadmap for serving: engines, routing, scheduling, autoscaling, caching, observability, and deployment. Own the platform’s SLOs and economics: latency, availability, GPU utilization, and cost per generation. Partner with research to ship new architectures to production on day zero and integrate serving into online RL and evaluation loops. Build scheduling and queueing that leverages expensive GPU resources against live traffic, cluster availability, and user priority. First 90 Days One way the first 90 could unfold. Days 1–30 — Immerse & Diagnose: Learn the serving stack, the team, and where reliability, latency, or cost hurt most. Days 30–60 — Ship & Validate: Personally ship a meaningful platform improvement while setting the team’s technical bar. Days 60–90 — Scale & Systemize: Set the roadmap, grow the team, and harden SLOs and economics across the fleet. What You Bring 8+ years in large-scale distributed systems or ML infrastructure, with several years building and operating model-serving or inference platforms in production. Experience running inference platforms at the thousands-of-GPUs scale across multiple clusters or clouds, and knowing what breaks there. Technical leadership experience through rapid growth, with a genuine desire to stay at least half hands-on. Deep expertise in LLM and foundation-model serving engines (vLLM, SGLang, TensorRT-LLM), ideally having modified engine internals. Strong command of continuous batching, KV-cache management, quantization, speculative decoding, and parallelism strategies (TP/EP/pipeline). Strong Python and PyTorch, Kubernetes at scale, and experience with queues, scheduling, traffic control, and fleet management. Nice to Have Experience serving diffusion, video, or other multimodal generative models, and with FFmpeg/multimedia processing. Modern networking stacks — RDMA (RoCE, InfiniBand), NVLink — and multi-node serving topologies. Experience across heterogeneous accelerators (NVIDIA, AMD, TPU, Trainium) and the porting and validation that comes with them. Contributions to open-source serving infrastructure (vLLM, SGLang, Ray, Kubernetes ecosystem). Systems-language depth (Rust, C++, CUDA/HIP) for kernel- and runtime-level optimization. About Luma: Luma’s mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer. #J-18808-Ljbffr Luma