Staff + Sr. Software Engineer, Scaling

Responsibilities

• Design, build, and maintain distributed systems serving Claude to millions of users.
• Develop intelligent request routing, load balancing, traffic management, autoscaling, and fleet orchestration across accelerators and cloud providers.
• Build and operate production-grade deployment pipelines for releasing new models reliably.
• Optimize inference performance, compute efficiency, and infrastructure costs across production, research, and experimental workloads.
• Integrate new AI accelerator platforms and support inference for new model architectures.
• Analyze observability data, tune production performance, and manage multi-region and geographic routing deployments.

Requirements

• Significant software engineering experience, particularly with distributed systems.
• Experience with high-performance, large-scale distributed systems is preferred.
• Experience implementing and deploying machine learning systems at scale is preferred.
• Experience with load balancing, request routing, or traffic management systems is preferred.
• Familiarity with LLM inference optimization, batching, and caching strategies is preferred.
• Experience with Kubernetes and cloud infrastructure such as AWS, GCP, or Azure is preferred.
• Proficiency in Python or Rust is preferred.
• Bachelor’s degree or an equivalent combination of education, training, and/or experience in a relevant field.

Benefits

• Hybrid policy requiring staff to work from an office at least 25% of the time.
• Competitive benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office collaboration space.
• Visa sponsorship is offered with reasonable efforts made after an offer, although sponsorship cannot be successfully provided for every role or candidate.