Staff + Sr. Software Engineer, Inference

About the company The company’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute‑agnostic inference deployments. We handle the entire stack from intelligent request routing to fleet‑wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve explosive customer growth, and enabling breakthrough research by providing scientists with high‑performance inference infrastructure. Key responsibilities Build and maintain distributed inference systems. Design request routing, load balancing, and traffic management. Optimize compute utilization and manage elastic scaling. Deploy and integrate new AI accelerator platforms. Build deployment pipelines for new models. Provide infrastructure support for research teams. Tune performance using observability data. Manage multi‑region deployments and geographic routing. Representative projects Designing intelligent routing algorithms that optimize request distribution across thousands of accelerators. Autoscaling our compute fleet to dynamically match supply with demand across production, research, and experimental workloads. Building production‑grade deployment pipelines for releasing new models to millions of users. Integrating new AI accelerator platforms to maintain our hardware‑agnostic competitive advantage. Contributing to new inference features (e.g., structured sampling, prompt caching). Supporting inference for new model architectures. Analyzing observability data to tune performance based on real‑world production workloads. Managing multi‑region deployments and geographic routing for global customers. Qualifications Significant software engineering experience, particularly with distributed systems. Results‑oriented, with a bias toward flexibility and impact. Ability to pick up slack, even if it goes outside job description. Enjoy pair programming. Willingness to learn more about machine learning systems and infrastructure. Thrives in environments where technical excellence directly drives business results and research breakthroughs. Care about societal impacts of work. Preferred experience High‑performance, large‑scale distributed systems. Implementing and deploying machine learning systems at scale. Load balancing, request routing, or traffic management systems. LLM inference optimization, batching, and caching strategies. Kubernetes and cloud infrastructure (AWS, GCP, Azure). Python or Rust. Compensation Annual Salary: $300,000 – $485,000 USD. Logistics Minimum education: Bachelor’s degree or an equivalent combination of education, training, and experience. Required field of study: a field relevant to the role as demonstrated through coursework, training, or professional experience. Minimum years of experience: correlated with internal job level requirements for the position. Location‑based hybrid policy: staff to be in one office at least 25% of the time. Visa sponsorship: we sponsor visas; we will make reasonable efforts to obtain a visa if an offer is made. #J-18808-Ljbffr United States Digital Space LLC