Staff Inference Engineer

Responsibilities

• Build and operate production-grade model-serving and inference systems for high-throughput, low-latency AI workloads.
• Optimize inference infrastructure for token throughput, latency, scalability, GPU utilization, reliability, and cost efficiency.
• Improve the scalability and operational maturity of the inference platform as customer demand grows.
• Partner with AI training, GPU performance, orchestration, infrastructure, and operations teams to transition models into production serving.
• Develop monitoring, alerting, and operational practices and resolve performance, reliability, and capacity issues.
• Contribute to architecture decisions and engineering standards for the evolving platform.

Requirements

• Experience building and operating production machine learning inference or model-serving systems at scale.
• Strong understanding of latency, throughput, memory utilization, and cost-efficiency trade-offs when serving large AI models.
• Experience designing reliable distributed systems or production infrastructure.
• Understanding of GPU-backed AI workloads and inference-system scaling challenges.
• Strong engineering fundamentals and ability to independently own complex technical problems.
• Experience with inference-serving frameworks such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or similar technologies is preferred.
• Experience optimizing LLM inference workloads or large-scale AI serving platforms is preferred.
• Experience with API-based AI products, high-volume production services, GPU scheduling, distributed systems, Kubernetes, or cloud infrastructure platforms is preferred.
• Familiarity with quantization, batching, caching, or performance tuning is preferred.
• Experience at a hyperscaler, AI lab, GPU cloud provider, or large-scale ML infrastructure organization is preferred.

Benefits

• Hybrid role in Bellevue, Washington, with approximately three days per week in the office.
• Candidates elsewhere in the U.S. may apply if open to relocation.
• Medical, dental, and vision insurance for U.S.-based employees.
• 401(k) plan with company match and paid holidays.
• Certain roles may receive merit increases, annual bonuses, and long-term incentives.
• U.S. work authorization is required; visa sponsorship is not currently available.