Staff Software Engineer, LLM Cloud Inference
Anthropic
Anthropic, a safety-focused AI company, is seeking backend engineers for the Cloud Inference team to scale Claude across AWS, GCP, Azure, and future CSPs. You will own services from API integration to inference execution and capacity management, working across platforms to keep reliability and cost-effectiveness at massive scale. Ideal candidates have strong distributed systems experience, cloud exposure, and a taste for cross-functional collaboration with product, security, and partner teams. #J-18808-Ljbffr
Vacancy posted 19 hours ago
Similar jobs that could be interesting for youBased on the Staff Software Engineer, LLM Cloud Inference in San Francisco, CA vacancy
- ...group of committed researchers, engineers, policy experts, and... ...systems. About the role The Cloud Inference team scales and optimizes Claude... ...qualifications Have significant software engineering experience, with... ...Are curious about LLM serving; prior inference or...Suggested
EngineersOfAI
San Francisco, CA19 hours ago $190k - $265k
...improve their business. Founded by engineers — and customer-obsessed — we... ...of AI.The Foundation Model Inference team is the backbone of Databricks... ...impact you will have:Build LLM infrastructure powering large-... ...systems, scalable APIs, or cloud-native infrastructureExperience...SuggestedLocal areaWorldwideDataBricks
San Francisco, CA8 days ago$190k - $265k
...improve their business. Founded by engineers — and customer-obsessed — we... ...AI. The Foundation Model Inference team is the backbone of... ...impact you will have: Build LLM infrastructure powering large-... ...distributed systems, scalable APIs, or cloud-native infrastructure ~...SuggestedLocal areaWorldwideDatabricks Inc.
San Francisco, CA4 days ago- ...Databricks seeks highly autonomous backend/infrastructure engineers to power enterprise-scale model inference on the Mosaic AI platform. You will build and optimize LLM infrastructure, serving OpenAI, Anthropic, Gemini and self-hosted models, tackling reliability and...Suggested
Databricks
San Francisco, CA3 days ago $190k - $265k
...improve their business. Founded by engineers — and customer-obsessed — we... ...across real-time and batch inference, powering model inference at enterprise... ...impact you will have: Build LLM infrastructure powering large-... ...systems, scalable APIs, or cloud-native infrastructure Strong...SuggestedLocal areaWorldwideNEON Inc
San Francisco, CA3 days ago$320k
...Staff + Senior Software Engineer, InferenceSan Francisco, CA | New York City, NY | Seattle... ...systems.About the RoleOur Inference team is responsible for... ...hardware running in multiple cloud platforms.Inference systems... ...management systemsFamiliarity with LLM inference optimization,...Work at officeWorldwideVisa sponsorshipFlexible hoursanthropic
San Francisco, CA3 days ago$190.9k - $232.8k
P-1285About This RoleAs a staff software engineer for GenAI inference, you will lead the architecture, development, and optimization of the inference engine... ...cross-team collaboration: with platform engineers, cloud infrastructure, and security/compliance teamsRepresent...Local areaWorldwideDataBricks
San Francisco, CA8 days ago- ...team of researchers, engineers, designers, and more,... ...experienced Back-End Software Engineer who wants to... ...Model Vault is Cohere’s inference platform for deploying... ...workloads. As a Senior/Staff Software Engineer, you... ...model deployments across clouds and hardware platforms...Full timeWork at officeLocal areaRemote workHome office
Cohere
San Francisco, CA20 hours ago $250k - $300k
...data center construction, and cloud services.If you want to do... ...the Role:We are seeking Sr. Staff Software Engineers to serve as the Managed Platform... ...profiles (training and inference) to identify what features and... ...— both rule-based and LLM-based — our platform can offer...Permanent employmentTemporary workShift workCrusoe
San Francisco, CA8 days ago- ...services that enable ML engineers and data scientists... ...products. As a Staff Engineer, you'll... ...infrastructure, model training, LLM fine-tuning, low-latency model inference, large-scale... ...operating great software systems.Who you... ...).Familiarity with cloud services (e.g., AWS...Flexible hours
Stripe
San Francisco, CA5 days ago - ...Cohere is a team of researchers, engineers, designers, and more, who are... ...for Members of Technical Staff to join the Model Serving team... ...GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid servingExperience... ...latency and throughput of inference.Strong understanding or...Full timeWork experience placementWork at officeLocal areaRemote workHome office
Cohere
San Francisco, CA3 days ago - ...Software Engineers at Senior / Staff LevelsSnorkel AI is the frontier AI data lab, helping... ...simulation environments, LLM infrastructure, and distributed... ...workflows, training, inference, or production ML systemsStrong... ...distributed systems and cloud platforms (AWS preferred),...Local areaShift work
Snorkel AI
San Francisco, CA1 day ago - ...Baseten powers mission-critical inference for the world's most dynamic... ...us and help build the platform engineers turn to ship AI products.... ...runtime that powers large-scale LLM inference on Baseten. Our inference... ...routing ~ How Baseten multi-cloud capacity management (MCM)...Flexible hours
Baseten
San Francisco, CA19 hours ago - ...Senior LLM Platform EngineerHandshake is hiring a Senior LLM Platform Engineer to join our Data and ML Platform team... ...or self-hosted inference. You'll also contribute... ...model serving, shared cloud infrastructure, and developer... ...ForStrong production software engineering...Full timeWork at officeFlexible hours
Handshake
San Francisco, CA2 days ago $200k - $250k
...proprietary vertical indexes with LLM-optimized retrieval systems to... ...useful. Our team includes engineers, researchers, product builders... ...passes through one layer: our inference gateway. It allows users to... ...an inference provider or GPU cloud, or contributions to an open-source...Immediate startRemote workWork from homeFlexible hoursMixpeek
San Francisco, CA4 days ago- ...Products, Talent, Spend, IT, Data Cloud, and among product areas. As a Senior Staff Software Engineer, you will own broad, ambiguous,... ...infra, model infrastructure, inference, sandboxing, workflow... ...eval frameworks, harness systems, LLM tooling, model-quality systems,...Work at office3 days per week
Rippling
San Francisco, CA6 days ago - ...Learning Systems Administrator / DevOps Engineer, you will maintain and build the core infrastructure... ...behind both our model training and our inference cloud. You’ll split your time between our... ...Experience deploying and operating LLM serving stacks (e.g., vLLM, SGLang, Ray)...Work at officeRemote workRelocation package
Zyphra
San Francisco, CA3 days ago - ...growing group of committed researchers, engineers, policy experts, and business... ...role Anthropic is looking for a Staff+ Software Engineer to work on the vector search... ...models and the retrieval demands of LLM training and inference Experience using GPUs or other accelerators...
Anthropic
San Francisco, CA4 days ago $200k - $300k
...product–market fit. The Role: We are seeking a hands‑on Staff Software Engineer who thrives in building from 0‑1 and driving execution,... ...(Temporal), streaming pipelines, vector search, and LLM inference paths; balancing latency, quality, and cost. Build for...Full timeSarah Smith Fund
San Francisco, CA4 days ago- ...Software Engineer - Multimodal InferenceOpenAI's Inference team powers the deployment of our most advanced models - including our GPT models, 4o Image Generation... ...familiarity with inference tooling like vLLM, TensorRT-LLM, or custom model parallel systems.Own problems end-...
Openai
San Francisco, CA19 hours ago $248k - $310k
...services and tools including LLM fine-tuning, alignment and... ...Will Make: As a senior staff machine learning engineer, you will be responsible for... ...with product managers, software engineers, data scientists... ...tuning, optimizing models and inference run-time Post-training...Work experience placementRemote workNubeero Limited
San Francisco, CA3 days ago- ...Inference Performance EngineerBaseten powers mission-critical... ...build the platform engineers turn to ship AI... ...the exciting field of LLM inference.You'll get to... ...enhancing the performance of software systems, particularly in... ...multi-region or multi-cloud capacity.Written or optimized...Flexible hours
CapitalG
San Francisco, CA8 hours ago $350k
...want to build it. About the Role We're hiring a Software Engineer to build and scale the distributed production inference systems that serve Inkling, Inkling-Small, and... ...Preferred Qualifications Experience with LLM inference engines such as vLLM, SGLang, or TensorRT...Visa sponsorshipWork visaRelocation packageThinking Machines Lab Inc.
San Francisco, CA3 days ago$227.2k - $324.5k
About the Role:As a Staff Software Engineer on the ML Infrastructure team, you will... ...-class machine learning inference platforms. These platforms power... ...that support Deep Learning, LLM, and Search models. This... ...experience with AWS or an equivalent cloud platformExperience building...Full timeTemporary workLocal areaFlexible hoursTubi TV
San Francisco, CA8 days ago- ...firms. About The Role We are looking for a seasoned Staff DevOps and Platform Engineer to own and evolve the infrastructure that powers... ...Own and improve Canary routing infrastructure for LRA (LLM REST API) via Traefik and GitOps patterns. Maintain and...For contractorsRemote work
Liberate
San Francisco, CA4 days ago $207k - $300k
...Provide oversight and guidance to executive engineers.Partner with cross-functional teams to... ...and architecture; and testing/launching software products.Experience architecting, designing... ...AI agents or large language model (LLM) applicationsPreferred qualifications:Master...Flexible hoursGoogle
San Francisco, CA5 days ago$220k - $290k
...manufacturing.About the Role:We're looking for an experienced Software Engineer to help architect and scale the agentic systems at the core of... ...intensive systems, and knows what it takes to make non-deterministic, LLM-driven systems reliable in production.Key ResponsibilitiesOwn...Local areaWork from homeFlexible hoursNight shiftWeekend workAxion Ray
San Francisco, CA5 days ago$231k - $340k
...started. Role Overview As a Staff Software Engineer on the Model Infrastructure... ...excellence for AI inference. Design and improve Harvey... ...understanding of distributed systems, cloud infrastructure, networking,... ...with AI infrastructure, LLM serving, or machine learning...Harvey
San Francisco, CA4 days ago$188k - $275k
...CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers... ...Of The Team The Inference team is responsible for delivering... ...are looking for an Applied AI Engineer to help us understand, measure... ...decisions. ~ Familiarity with LLM inference systems and tools...Permanent employmentFull timeTemporary workCasual workWork at officeFlexible hoursCoreWeave
San Francisco, CA4 days ago$180k - $220k
...We’re looking for a Senior/Staff Backend Engineer to architect and build large... ...maintainable, and reliable software. Strong backend engineer. Deep... ...Haves Built or contributed to LLM-powered or agent-driven systems — such as real-time inference pipelines, context and...Full timeWork at officeShift workActively AI
San Francisco, CA3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Software Engineer, LLM Cloud Inference. Be the first to apply!
Related searches
- senior devops cloud engineer San Francisco, CA
- senior aws cloud engineer San Francisco, CA
- full stack cloud developer San Francisco, CA
- cloud network engineer San Francisco, CA
- cloud automation engineer San Francisco, CA
- google cloud architect San Francisco, CA
- senior cloud network engineer San Francisco, CA
- principal cloud computing engineer San Francisco, CA
- cloud solutions architect San Francisco, CA
- cloud engineer San Francisco, CA