Staff Software Engineer, Inference
$188k - $275kCoreWeave
CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Learn more at .
What You’ll Do:
Inference Platform Team
The Inference team builds and operates CoreWeave’s Kubernetes-native inference platform, powering low-latency, high-throughput AI workloads at massive scale. The team is responsible for request routing, scheduling, GPU resource management, and system-wide optimizations that drive performance, efficiency, and reliability across real-time inference systems.
About the role:
As a Staff Software Engineer (IC5) on the Inference team, you will act as a technical leader driving architecture, performance, and reliability across multiple services and teams. Your day-to-day will involve leading cross-team design initiatives, optimizing inference performance (latency, throughput, and GPU utilization), and improving system reliability at scale. You will work deeply in distributed systems and Kubernetes-based infrastructure, focusing on areas like scheduling, batching, and memory optimization. This role requires hands-on technical leadership and the ability to influence engineering direction across the organization.
Who You Are:
- 8–12+ years of experience building and operating large-scale distributed systems or cloud platforms
- Proven experience leading cross-team technical initiatives impacting multiple services or organizations
- Strong programming skills in Go, Python, or C++
- Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design
- Strong understanding of distributed systems, networking, and performance optimization
- Experience designing and operating low-latency, high-throughput systems with strict P95/P99 latency requirements
- Hands-on experience with inference systems, including batching or micro-batching strategies, caching, and memory optimization
- Experience improving system performance using metrics-driven approaches (e.g., latency, throughput, utilization)
- Familiarity with mixed precision (BF16, FP8) and streaming inference workloads
Preferred:
- Experience with inference frameworks such as vLLM, Triton, TensorRT-LLM, Ray Serve, or TorchServe
- Experience with GPU systems and performance optimization (CUDA, NCCL, RDMA, NUMA, GPU interconnects)
- Experience leading multi-team or org-level technical initiatives
- Exposure to large-scale AI/ML infrastructure or hyperscale cloud environments
Wondering if you’re a good fit?
We believe in investing in our people, and value candidates who can bring their own diversified experiences to our teams – even if you aren't a 100% skill or experience match. Here are a few qualities we’ve found compatible with our team. If some of this describes you, we’d love to talk.
- You love to design and optimize high-performance distributed systems at scale
- You’re curious about AI inference, GPU systems, and emerging performance techniques
- You’re an expert in building reliable, low-latency infrastructure and driving system-wide improvements
Why CoreWeave?
At CoreWeave, we work hard, have fun, and move fast! We’re in an exciting stage of hyper-growth that you will not want to miss out on. We’re not afraid of a little chaos, and we’re constantly learning. Our team cares deeply about how we build our product and how we work together, which is represented through our core values:
- Be Curious at Your Core
- Act Like an Owner
- Empower Employees
- Deliver Best-in-Class Client Experiences
- Achieve More Together
We support and encourage an entrepreneurial outlook and independent thinking. We foster an environment that encourages collaboration and enables the development of innovative solutions to complex problems. As we get set for takeoff, the organization's growth opportunities are constantly expanding. You will be surrounded by some of the best talent in the industry, who will want to learn from you, too. Come join us!
The base salary range for this role is $188,000 to $275,000. The starting salary will be determined based on job-related knowledge, skills, experience, and market location. We strive for both market alignment and internal equity when determining compensation. In addition to base salary, our total rewards package includes a discretionary bonus, equity awards, and a comprehensive benefits program (all based on eligibility).
What We Offer
In addition to a competitive salary, we offer a variety of benefits to support your needs. The benefits below reflect our US-based offerings for full-time employees; for roles in other locations, benefits vary and are shared during the hiring process. These include:
- Medical, dental, and vision insurance - 100% paid for by CoreWeave
- Company-paid Life Insurance
- Voluntary supplemental life insurance
- Short and long-term disability insurance
- Flexible Spending Account
- Health Savings Account
- Tuition Reimbursement
- Ability to Participate in Employee Stock Purchase Program (ESPP)
- Mental Wellness Benefits through Spring Health
- Family-Forming support provided by Carrot
- Paid Parental Leave
- Flexible, full-service childcare support with Kinside
- 401(k) with a generous employer match
- Flexible PTO
- Catered lunch each day in our office and data center locations
- A casual work environment
- A work culture focused on innovative disruption
California Applicants
Equal Opportunity & Accommodations
CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information.
As part of this commitment and consistent with the Americans with Disabilities Act (ADA) , CoreWeave will ensure that qualified applicants and candidates with disabilities are provided reasonable accommodations for the hiring process, unless such accommodation would cause an undue hardship. If reasonable accommodation is needed, please contact: View email address on coreweave.com .
Export Control Compliance
This position requires access to technology and/or source code subject to U.S. export control laws. Any such access will be provided only in compliance with applicable U.S. export control requirements. As part of the recruitment process, CoreWeave may request limited information reasonably necessary to assess applicable export-control requirements.
$201.3k - $352.3k
...Company Description It all started when engineer Fred Luddy wrote code that... ...innovation group building intelligent software and smart user experiences using existing... ...future. We are seeking a Senior Staff Software Engineer - AI/ML Inference Engineering to lead the...SuggestedFull timeWork experience placementWork at officeImmediate startRemote workFlexible hours$254k - $350k
...CA / Seattle, WA / Remote (United States)Software - Software Systems /Full-time /On-... ...work alongside a team of strong software engineers and act as a force multiplier for our internal... ...of cutting-edge ML Training OR Inference performance optimization techniques to scale...SuggestedFull timeRemote work$209.1k - $282.9k
As a Software Engineer on our AI Inference Runtime team, you will set technical direction for critical components of distributed Inference runtime for running SOTA AI Models.You will lead hands-on work across scheduling, batching, KV-cache management, memory allocation,...SuggestedWork at officeLocal area$209.1k - $282.9k
As an engineer on Arm’s AI Inference Cloud team, you will shape the technical direction and develop highly available, scalable services for running... ..., networking, and workload lifecycle management.Strong software and production engineering skills, including hands-on programming...SuggestedWork at officeLocal areaVisa sponsorshipRelocation package$188k - $275k
...in March 2025. Learn more at . What You’ll Do: As a Staff Software Engineer (IC5) on the AI Workload Orchestration Platform team, you... ...SUNK (Slurm on Kubernetes) and underpins both training and inference workloads across the CoreWeave cloud. This role requires...SuggestedFull timeTemporary workCasual workWork at officeFlexible hours$137.3k - $251.8k
...Our flexible structure gives engineers direct influence over product... ...What You Will Do: Our Staff Engineers are end-to-end owners... ...are seeking a highly motivated software engineer with a passion content... ..., data indexing, building ML inference and re-training frameworks....Full timeTemporary workWorldwideFlexible hours$405k
...group of committed researchers, engineers, policy experts, and business... ...the role Anthropic's Inference organization serves Claude to... ...we add. We're looking for a Staff engineer to be the technical... ...testing, and shipping inference software. This is a senior IC role...Remote jobWork at officeLocal areaVisa sponsorshipFlexible hours- ...group of committed researchers, engineers, policy experts, and business... ...About the role Anthropic's Inference organization serves Claude to... ...we add. We're looking for a Staff Engineer to be a technical... ...of them Have significant software engineering experience, with...Full timeWork at officeVisa sponsorshipFlexible hours
$320k
...group of committed researchers, engineers, policy experts, and business... .... About the role Our Inference team is responsible for... ...qualifications Significant software engineering experience, particularly... ...: Currently, we expect all staff to be in one of our offices...Work at officeLocal areaWorldwideVisa sponsorshipFlexible hours- ...Senior Principal / Staff Engineer — Watchtower Labs In-person only - Bellevue, WA U.S. Citizenship required (DoD Clearance preferable) How to apply Email ****@*****.*** with three bullets that prove you can do this. No long intro. No generic claims...
$320k
...group of committed researchers, engineers, policy experts, and business... ...About the role The Cloud Inference team scales and optimizes... ...required Have significant software engineering experience, with... ...policy: Currently, we expect all staff to be in one of our offices...Work at officeLocal areaVisa sponsorshipFlexible hours$320k
...group of committed researchers, engineers, policy experts, and business... ...About the role The Cloud Inference team scales and optimizes... ...qualifications Have significant software engineering experience, with... ...: Currently, we expect all staff to be in one of our offices...Work at officeLocal areaVisa sponsorshipFlexible hours$236k - $339.2k
...CoWork, and UX features Developer tooling that enable our engineers to move fast and deliver Frontend infrastructure and shared... ...impact is significant. What We’re Looking For Our ideal Staff Software Engineer will have: ~8+ years of software engineering...Full timeFlexible hours- ...office location 4 days per week; Lambda’s designated work from home day is currently Tuesday. About the Role As a Staff Software Engineer for the Compute pillar, you will play a critical role in defining the technical vision for Lambda's next-generation GPU and...Full timeWork at officeLocal areaWork from homeFlexible hours
$206.09k - $302.2k
...build teams that learn from one another. If you’re driven to solve meaningful challenges, we’d love to meet you. Job Title: Staff Software Engineer Location: 10770 NE 8th St., Suite 700, Bellevue, WA 98004 *Telecommuting permitted: work may be performed within...Full timeTemporary workWork at officeRemote workWorldwide$180k - $260k
...success. About the Role We are looking for a Senior / Staff Software Engineer — Data Platform to build and evolve the data and... ...platforms Familiarity with ML training, feature generation, inference, and model-serving workflows Experience with modern frontend...Immediate start- ...patterns, rendering strategies, and progressive loading. Mentor engineers, review critical performance paths, and promote organization-... ..., and caching layers. Requirements ~8+ years of software engineering experience with deep expertise in web performance...Full time
$217.1k - $298.55k
...the durable architecture across all these surfaces, raising the engineering bar of the combined team, and shaping the next generation of... ...insights, and governance. Champion reliable, high-quality software and the operational practices that let a focused team confidently...Worldwide$152k - $204k
...in March 2025. Learn more at . What You’ll Do: Senior engineers are area owners who lead designs, raise engineering standards,... ...orchestration, and hardware teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale. About the role:...Full timeTemporary workCasual workWork at officeFlexible hoursShift work$139k - $204k
...GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a... ...workloads. About the role We are looking for Senior Software Engineer to help us understand, measure, and improve the...Full timeTemporary workCasual workWork at officeFlexible hours$182k - $242k
...) in March 2025. Learn more at . About the Team Senior engineers are area owners who lead designs, raise engineering standards,... ...orchestration, and hardware teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale. Key...Full timeTemporary workCasual workWork at officeFlexible hoursShift work$120k - $160k
...GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a... ...most demanding workloads. About the role As a Junior Software Engineer (IC1/IC2) on the Inference team, you’ll own a single...Full timeTemporary workCasual workWork at officeRemote workFlexible hours- ...: CRWV) in March 2025. Learn more at Senior Deployment Engineer - LLM Inference & Optimization About the role We're looking for a Deployment... .... What you'll bring ~5+ years of experience in software engineering, ML engineering, AI infrastructure, or a related...Full timeTemporary workCasual workWork at officeFlexible hours
$160k - $250k
...better models power more useful products, and what we learn from people using those products shapes what we build next. SOFTWARE ENGINEER, TRAINING/INFERENCE (C++) At SpaceXSI we are building the high-performance inference platform that serves Grok to millions of users...Temporary work$188k - $275k
...clusters, agent building, and inference at scale, we're combining... ...problems at the intersection of software, hardware, and AI, there's never... ...What You'll Do: Performance Engineering at Weights & Biases ensures... ...About the role: As a Staff Engineer on the Performance...Permanent employmentFull timeTemporary workCasual workWork at officeFlexible hours$188k - $275k
...clusters, agent building, and inference at scale, we’re combining... ...problems at the intersection of software, hardware, and AI, there’s never... ...You’ll Do: Performance Engineering at Weights & Biases ensures... ...About the role: As a Staff Engineer on the Performance...Remote jobFull timeTemporary workCasual workWork at officeFlexible hours$221.2k - $241.1k
...and AI infrastructure platform, so our customers can focus on the high-value challenges that are central to their missions. Our engineering teams build highly technical products that fulfill real, important needs in the world. We constantly push the boundaries of data...Local areaWorldwide$143.7k - $194.4k
AWS Neuron is the complete software stack for AWS Inferentia and Trainium — AWS purpose-built... ...learning. Join the Machine Learning Inference Applications team to build the serving technology... ...and efficiently on Neuron chips.As an engineer on this team, you'll work on core...InternshipFlexible hours$168.1k - $227.4k
AWS Neuron is the complete software stack for AWS Inferentia and Trainium... ...learning. This senior software engineering role is part of the Machine Learning Inference Applications team and focuses on... ...other employees, supervisors, and staff; adhere to standards of...Work experience placementInternshipLocal areaFlexible hours$140k - $230k
...Senior Software Engineer (AI Inference & Runtime Platform) AZX | Remote (U.S.) | Full time | Applications close November 12 Compensation: $140,000 - $230,000, plus bonus eligibility and equity You will own the layer where AI work physically happens: the machines,...Full timeRemote workWork visaFlexible hoursDay shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Software Engineer, Inference. Be the first to apply!
- software engineer - cloud services Bellevue, WA
- software technical support engineer Bellevue, WA
- remote software sales Bellevue, WA
- software sales representative Bellevue, WA
- software quality assurance Bellevue, WA
- software qa Bellevue, WA
- entry level software sales Bellevue, WA
- software sales Bellevue, WA
- software sales executive Bellevue, WA
- government software Bellevue, WA





