Senior Inference Engineer

Responsibilities

• Build and operate production-grade model-serving and inference systems for high-throughput, low-latency AI workloads.
• Optimize inference infrastructure for token throughput, latency, scalability, GPU utilization, memory efficiency, reliability, and cost.
• Improve the scalability and operational maturity of the inference platform as customer demand grows.
• Partner with AI training, GPU performance, orchestration, platform engineering, and operations teams to transition models into production serving.
• Develop monitoring, alerting, and operational practices for reliable inference services.
• Investigate and resolve performance, reliability, and capacity challenges across inference workloads.
• Contribute to platform architecture decisions and engineering standards.

Requirements

• Experience building and operating production machine learning inference or model-serving systems at scale.
• Strong understanding of latency, throughput, memory utilization, and cost-efficiency trade-offs when serving large AI models.
• Experience designing reliable distributed systems or production infrastructure.
• Understanding of GPU-backed AI workloads and inference-system scaling challenges.
• Strong engineering fundamentals and ability to independently own complex technical problems.
• Experience with one or more inference-serving frameworks such as vLLM, SGLang, TensorRT-LLM, or Triton Inference Server is preferred.
• Experience optimizing LLM inference workloads or large-scale AI serving platforms is preferred.
• Experience operating API-based AI products or high-volume production services is preferred.
• Experience with GPU scheduling, distributed systems, Kubernetes, or cloud infrastructure platforms is preferred.
• Familiarity with quantization, batching, caching, or performance tuning is preferred.
• Experience at a hyperscaler, AI lab, GPU cloud provider, or large-scale ML infrastructure organization is preferred.

Benefits

• Hybrid role in Bellevue, Washington, with approximately three days per week in the office.
• Candidates elsewhere in the United States who are open to relocation are encouraged to apply.
• Medical, dental, and vision insurance for U.S.-based employees.
• 401(k) plan with company match and paid holidays.
• Certain roles may receive merit increases, annual bonuses, and long-term incentives based on individual performance.