Software Engineer, Inference
$120k - $160kCoreWeave
CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Learn more at .
About CoreWeave
CoreWeave is an AI cloud built for high-performance GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a powerful alternative to traditional hyperscalers. Infrastructure is our product, and we operate it at a massive scale to power the most demanding workloads.
About the role
As a Junior Software Engineer (IC1/IC2) on the Inference team, you’ll own a single production service end-to-end shipping features, maintaining dashboards/alarms, and improving operational excellence. You’ll collaborate with peers across runtime, scheduling, and networking to move key SLIs.
What you’ll do
- Own one model-serving service end-to-end: design, implementation, tests, deployment, and ops.
- Add and tune observability (dashboards, alerts, traces); automate runbook steps to reduce toil.
- Improve latency/throughput via micro-batching, concurrency tuning, and caching.
- Participate in on-call; lead post-incident action items for your service.
- Review peers’ PRs and contribute design feedback in team reviews.
What we’re looking for
- 0–3 years of software engineering experience (or equivalent) building cloud/distributed services in production.
- Proficiency in Python or Go; familiarity with gRPC/ APIs and containerized deployments.
- Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines.
- Experience with metrics/alerting (Prometheus/Grafana/OpenTelemetry) and structured logging.
- Basic understanding of GPU inference concepts (KV caching, mixed precision, streaming tokens).
Nice to have
- Experience with inference frameworks (Triton, vLLM, TensorRT-LLM, Ray Serve, TorchServe).
- Hands-on performance profiling/benchmarking (e.g., wrk, vegeta, perf, Nsight).
- Contributions to reliability practices: SLI/SLOs or postmortem improvements.
Why CoreWeave
Help shape an industry-defining inference platform that enables teams to deploy generative AI and real-time applications at scale. If optimizing tail latency and delivering reliable model serving excites you, this is the place to build.
Benefits (highlights)
- 100% employer-paid medical, dental, and vision
- 401(k) with a generous match
- Flexible PTO and a remote-friendly culture
- Work with top talent in a culture of curiosity, ownership, and innovation
The base salary range for this role is $120,000.00 - $160,000.00. The starting salary will be determined based on job-related knowledge, skills, experience, and market location. We strive for both market alignment and internal equity when determining compensation. In addition to base salary, our total rewards package includes a discretionary bonus, equity awards, and a comprehensive benefits program (all based on eligibility).
What We Offer
In addition to a competitive salary, we offer a variety of benefits to support your needs. The benefits below reflect our US-based offerings for full-time employees; for roles in other locations, benefits vary and are shared during the hiring process. These include:
- Medical, dental, and vision insurance - 100% paid for by CoreWeave
- Company-paid Life Insurance
- Voluntary supplemental life insurance
- Short and long-term disability insurance
- Flexible Spending Account
- Health Savings Account
- Tuition Reimbursement
- Ability to Participate in Employee Stock Purchase Program (ESPP)
- Mental Wellness Benefits through Spring Health
- Family-Forming support provided by Carrot
- Paid Parental Leave
- Flexible, full-service childcare support with Kinside
- 401(k) with a generous employer match
- Flexible PTO
- Catered lunch each day in our office and data center locations
- A casual work environment
- A work culture focused on innovative disruption
California Applicants
Equal Opportunity & Accommodations
CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information.
As part of this commitment and consistent with the Americans with Disabilities Act (ADA) , CoreWeave will ensure that qualified applicants and candidates with disabilities are provided reasonable accommodations for the hiring process, unless such accommodation would cause an undue hardship. If reasonable accommodation is needed, please contact: View email address on coreweave.com .
Export Control Compliance
This position requires access to technology and/or source code subject to U.S. export control laws. Any such access will be provided only in compliance with applicable U.S. export control requirements. As part of the recruitment process, CoreWeave may request limited information reasonably necessary to assess applicable export-control requirements.
- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...ultra high-speed inference.About the RoleWe're hiring a Software Engineer to help contribute to projects on our Inference Platform...Suggested
$160.36k - $240.54k
...components. Develop observability to track ML model lifecycles from data generation to on-road validation.Maintain an in-house ML inference platform to serve large language models efficiently.Maintain an in-house ML compiler platform to compile, deploy, and validate Nuro...SuggestedImmediate startFlexible hours- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...with ultra-fast decode on the Cerebras Wafer-Scale Engine.We are hiring a Software Engineer to productionize and optimize our GPU serving stack...Suggested
$2,000 per month
...architecture and design of the Sohu host software stack Implement high-performance,... ...handling continuous batching and real time inference Implement inference-time... ...person team in Cupertino, and greatly value engineering skills. We do not have boundaries between...SuggestedWork at officeRelocation package- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...high-speed inference.About the RoleWe’re hiring a Senior Software Engineer for our Inference Cloud team. This team owns the cloud layer...Suggested
$150.4k - $277.6k
Role Number: 200690248Team: Software and ServicesLocation: Cupertino; SeattlePosted: 2026-10-19SummaryApple is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute, the backbone of Siri AI and Apple Intelligence...Remote workWorldwideRelocation$193.3k - $261.5k
...layer is what our team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and... ...will:Build and optimize the low-level data-movement software across accelerators, servers, and heterogeneous...InternshipLocal areaFlexible hours$165.2k - $223.6k
...layer is what our team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and... ...will:Build and optimize the low-level data-movement software that transfers KV cache and activations across...InternshipLocal areaFlexible hours$127.1k - $185k
...Services (AWS) builds the AWS Neuron SDK, the software development kit used to accelerate deep... ...learning accelerators.The Neuron Inference organization is at the forefront of optimizing... ...to the hardware-software boundary, our engineers build systematic infrastructure, develop...InternshipLocal areaFlexible hours$165.2k - $223.6k
...Services (AWS) builds the AWS Neuron SDK, the software development kit used to accelerate deep... ...learning accelerators.The Neuron Inference organization is at the forefront of optimizing... ...to the hardware-software boundary, our engineers build systematic infrastructure, develop...InternshipLocal areaFlexible hours$184k - $287.5k
NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a Senior Software Engineer - AI Inference Performance to advance innovative LLM and VLM inference. You will push workloads toward practical performance limits on NVIDIA GPU-accelerated...Full time- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ....About the TeamThe Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure...
$152k - $241.5k
NVIDIA is seeking talented engineers to join the TensorRT team and advance high-performance AI inference on NVIDIA GPUs. Our team develops software that accelerates state-of-the-art generative AI models for local and enterprise deployments. We solve challenging problems...Full timeLocal area- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ...with ultra-fast decode on the Cerebras Wafer-Scale Engine.We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes...
$193.3k - $261.5k
We develop AWS Neuron, the complete software stack for Trainium, Amazon's custom cloud-scale machine learning accelerators. Join... ...on the Trainium hardware.As a Sr. Software Development Engineer on the Inference Model Enablement team, you will onboard and optimize state...InternshipLocal areaFlexible hours- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ....About the TeamThe Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure...Remote work
$165.2k - $223.6k
...Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep... ...and JAX enabling unparalleled ML inference and training performance.The Inference Enablement... ...the hardware-software boundary, our engineers build systematic infrastructure, innovate...Work experience placementInternshipLocal areaFlexible hours$192k - $278k
...definition and optimization of the hardware/software stack to enable performant training and... ...:Bachelor's degree in Electrical Engineering, Computer Engineering, Computer Science... ...performance optimizations for ML training and inference.Experience with deep learning...Worldwide- ...-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture... ...on a global scale. From chips and software to rack-scale systems, our team is tackling... ....The role: Principal System Software Engineer, AI Inference ExecutionWhat you will do:...3 days per week
$152k - $241.5k
...company”.We are looking for an AI & Deep Learning Compiler Engineer. NVIDIA is hiring software engineers for its Deep Learning & AI Compiler (DLC) team... ...of AI, our DLC has been the backbone of NVIDIA’s inference engine, spanning across data centers, personal devices,...Full timeRemote work- ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU-... ...About the RoleWe’re hiring a senior backend engineer to build and scale the services that... ...experience building and operating production software.Education. Bachelor’s or master’s degree...
$160.36k - $240.54k
...components. Develop observability to track ML model lifecycles from data generation to on-road validation. Maintain an in-house ML inference platform to serve large language models efficiently. Maintain an in-house ML compiler platform to compile, deploy, and validate...Immediate startFlexible hours$160.36k - $240.54k
...components. Develop observability to track ML model lifecycles from data generation to on-road validation. Maintain an in-house ML inference platform to serve large language models efficiently. Maintain an in-house ML compiler platform to compile, deploy, and validate...Immediate startFlexible hours$280k - $350k
About InworldInworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We build first... ...running on capacity reserved for it.As a Staff or Principal Software Engineer, you'll design that platform's core architecture and write...Full timeWork at officeRelocation$135k - $175k
...is actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, INFERENCE (AI DATA ENGINEERING) The application software team is the central nervous system of SpaceX - we create mission...Temporary workRemote workWorldwideWeekend work$120k - $160k
...GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a... ...most demanding workloads. About the role As a Junior Software Engineer (IC1/IC2) on the Inference team, you'll own a single...Full timeTemporary workCasual workWork at officeRemote workFlexible hours- ...to run fast enough to control robots in the real world, and efficiently enough to support large-scale research. As an Inference Infrastructure Engineer Intern, you'll work with our infra and research engineers to make model inference faster and more efficient across the...Internship
$194k - $349k
...capabilities into the product and use AI tools throughout your engineering workflow. Leadership & Impact As a technical leader, you... ...knowledge of LLM fundamentals applied to real systems: inference behavior, hallucination patterns, non-determinism, context management...Permanent employmentWork experience placementWork at officeLocal area$182k - $242k
...) in March 2025. Learn more at . About the Team Senior engineers are area owners who lead designs raise engineering standards and... ...and hardware teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale. Key Responsibilities...Full timeTemporary workCasual workWork at officeFlexible hoursShift work$213k - $263k
...solutions to speed up developer velocity. We’re looking for a software engineer to join the team to build and maintain the critical data and... ...Staff Software Engineer. You will: Develop Waymo's inference platform to make it scalable, high throughput, and low latency...Full timeRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer, Inference. Be the first to apply!
- senior software design engineer Sunnyvale, CA
- software web developer Sunnyvale, CA
- senior software engineer Sunnyvale, CA
- software developer fintech Sunnyvale, CA
- security software engineer Sunnyvale, CA
- software developer night shift Sunnyvale, CA
- new graduate software engineer Sunnyvale, CA
- senior software engineer ruby on rails Sunnyvale, CA
- intel software engineer Sunnyvale, CA
- graduate software developer no experience Sunnyvale, CA



