Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Software Engineer, Inference

$120k - $160k

CoreWeave

CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Learn more at  .

About CoreWeave

CoreWeave is an AI cloud built for high-performance GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a powerful alternative to traditional hyperscalers. Infrastructure is our product, and we operate it at a massive scale to power the most demanding workloads.

About the role

As a Junior Software Engineer (IC1/IC2) on the Inference team, you’ll own a single production service end-to-end shipping features, maintaining dashboards/alarms, and improving operational excellence. You’ll collaborate with peers across runtime, scheduling, and networking to move key SLIs.

What you’ll do

  • Own one model-serving service end-to-end: design, implementation, tests, deployment, and ops.
  • Add and tune observability (dashboards, alerts, traces); automate runbook steps to reduce toil.
  • Improve latency/throughput via micro-batching, concurrency tuning, and caching.
  • Participate in on-call; lead post-incident action items for your service.
  • Review peers’ PRs and contribute design feedback in team reviews.

What we’re looking for

  • 0–3 years of software engineering experience (or equivalent) building cloud/distributed services in production.
  • Proficiency in Python or Go; familiarity with gRPC/ APIs and containerized deployments.
  • Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines.
  • Experience with metrics/alerting (Prometheus/Grafana/OpenTelemetry) and structured logging.
  • Basic understanding of GPU inference concepts (KV caching, mixed precision, streaming tokens).

Nice to have

  • Experience with inference frameworks (Triton, vLLM, TensorRT-LLM, Ray Serve, TorchServe).
  • Hands-on performance profiling/benchmarking (e.g., wrk, vegeta, perf, Nsight).
  • Contributions to reliability practices: SLI/SLOs or postmortem improvements.

Why CoreWeave

Help shape an industry-defining inference platform that enables teams to deploy generative AI and real-time applications at scale. If optimizing tail latency and delivering reliable model serving excites you, this is the place to build.

Benefits (highlights)

  • 100% employer-paid medical, dental, and vision
  • 401(k) with a generous match
  • Flexible PTO and a remote-friendly culture
  • Work with top talent in a culture of curiosity, ownership, and innovation

The base salary range for this role is $120,000.00 - $160,000.00. The starting salary will be determined based on job-related knowledge, skills, experience, and market location. We strive for both market alignment and internal equity when determining compensation. In addition to base salary, our total rewards package includes a discretionary bonus, equity awards, and a comprehensive benefits program (all based on eligibility). 

What We Offer

In addition to a competitive salary, we offer a variety of benefits to support your needs. The benefits below reflect our US-based offerings for full-time employees; for roles in other locations, benefits vary and are shared during the hiring process. These include:

  • Medical, dental, and vision insurance - 100% paid for by CoreWeave
  • Company-paid Life Insurance 
  • Voluntary supplemental life insurance 
  • Short and long-term disability insurance 
  • Flexible Spending Account
  • Health Savings Account
  • Tuition Reimbursement 
  • Ability to Participate in Employee Stock Purchase Program (ESPP)
  • Mental Wellness Benefits through Spring Health 
  • Family-Forming support provided by Carrot
  • Paid Parental Leave 
  • Flexible, full-service childcare support with Kinside
  • 401(k) with a generous employer match
  • Flexible PTO
  • Catered lunch each day in our office and data center locations
  • A casual work environment
  • A work culture focused on innovative disruption

California Applicants

Equal Opportunity & Accommodations

CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information.

As part of this commitment and consistent with the Americans with Disabilities Act (ADA) , CoreWeave will ensure that qualified applicants and candidates with disabilities are provided reasonable accommodations for the hiring process, unless such accommodation would cause an undue hardship. If reasonable accommodation is needed, please contact: View email address on coreweave.com .

Export Control Compliance

This position requires access to technology and/or source code subject to U.S. export control laws. Any such access will be provided only in compliance with applicable U.S. export control requirements. As part of the recruitment process, CoreWeave may request limited information reasonably necessary to assess applicable export-control requirements.

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Software Engineer, Inference in Sunnyvale, CA vacancy
  •  ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud...  ...ultra high-speed inference.About the RoleWe're hiring a Software Engineer to help contribute to projects on our Inference Platform... 
    Suggested

    Cerebras Systems

    Sunnyvale, CA
    4 days ago
  • $160.36k - $240.54k

     ...components. Develop observability to track ML model lifecycles from data generation to on-road validation.Maintain an in-house ML inference platform to serve large language models efficiently.Maintain an in-house ML compiler platform to compile, deploy, and validate Nuro... 
    Suggested
    Immediate start
    Flexible hours

    Nuro

    Mountain View, CA
    3 days ago
  •  ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud...  ...with ultra-fast decode on the Cerebras Wafer-Scale Engine.We are hiring a Software Engineer to productionize and optimize our GPU serving stack... 
    Suggested

    Cerebras Systems

    Sunnyvale, CA
    2 days ago
  • $2,000 per month

     ...architecture and design of the Sohu host software stack Implement high-performance,...  ...handling continuous batching and real time inference Implement inference-time...  ...person team in Cupertino, and greatly value engineering skills. We do not have boundaries between... 
    Suggested
    Work at office
    Relocation package

    Etched

    Cupertino, CA
    2 days ago
  •  ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud...  ...high-speed inference.About the RoleWe’re hiring a Senior Software Engineer for our Inference Cloud team. This team owns the cloud layer... 
    Suggested

    Cerebras Systems

    Sunnyvale, CA
    1 day ago
  • $150.4k - $277.6k

    Role Number: 200690248Team: Software and ServicesLocation: Cupertino; SeattlePosted: 2026-10-19SummaryApple is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute, the backbone of Siri AI and Apple Intelligence... 
    Remote work
    Worldwide
    Relocation

    Apple

    Cupertino, CA
    4 days ago
  • $193.3k - $261.5k

     ...layer is what our team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and...  ...will:Build and optimize the low-level data-movement software across accelerators, servers, and heterogeneous... 
    Internship
    Local area
    Flexible hours

    Amazon

    Cupertino, CA
    4 days ago
  • $165.2k - $223.6k

     ...layer is what our team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and...  ...will:Build and optimize the low-level data-movement software that transfers KV cache and activations across... 
    Internship
    Local area
    Flexible hours

    Amazon

    Cupertino, CA
    2 days ago
  • $127.1k - $185k

     ...Services (AWS) builds the AWS Neuron SDK, the software development kit used to accelerate deep...  ...learning accelerators.The Neuron Inference organization is at the forefront of optimizing...  ...to the hardware-software boundary, our engineers build systematic infrastructure, develop... 
    Internship
    Local area
    Flexible hours

    Amazon

    Cupertino, CA
    16 hours ago
  • $165.2k - $223.6k

     ...Services (AWS) builds the AWS Neuron SDK, the software development kit used to accelerate deep...  ...learning accelerators.The Neuron Inference organization is at the forefront of optimizing...  ...to the hardware-software boundary, our engineers build systematic infrastructure, develop... 
    Internship
    Local area
    Flexible hours

    Amazon

    Cupertino, CA
    1 day ago
  • $184k - $287.5k

    NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a Senior Software Engineer - AI Inference Performance to advance innovative LLM and VLM inference. You will push workloads toward practical performance limits on NVIDIA GPU-accelerated... 
    Full time

    Nvidia

    Santa Clara, CA
    3 days ago
  •  ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale...  ....About the TeamThe Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure... 

    Cerebras Systems

    Sunnyvale, CA
    1 day ago
  • $152k - $241.5k

    NVIDIA is seeking talented engineers to join the TensorRT team and advance high-performance AI inference on NVIDIA GPUs. Our team develops software that accelerates state-of-the-art generative AI models for local and enterprise deployments. We solve challenging problems... 
    Full time
    Local area

    Nvidia

    Santa Clara, CA
    2 days ago
  •  ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale...  ...with ultra-fast decode on the Cerebras Wafer-Scale Engine.We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes... 

    Cerebras Systems

    Sunnyvale, CA
    16 hours ago
  • $193.3k - $261.5k

    We develop AWS Neuron, the complete software stack for Trainium, Amazon's custom cloud-scale machine learning accelerators. Join...  ...on the Trainium hardware.As a Sr. Software Development Engineer on the Inference Model Enablement team, you will onboard and optimize state... 
    Internship
    Local area
    Flexible hours

    Amazon

    Cupertino, CA
    1 day ago
  •  ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale...  ....About the TeamThe Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure... 
    Remote work

    Cerebras Systems

    Sunnyvale, CA
    16 hours ago
  • $165.2k - $223.6k

     ...Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep...  ...and JAX enabling unparalleled ML inference and training performance.The Inference Enablement...  ...the hardware-software boundary, our engineers build systematic infrastructure, innovate... 
    Work experience placement
    Internship
    Local area
    Flexible hours

    Amazon

    Cupertino, CA
    2 days ago
  • $192k - $278k

     ...definition and optimization of the hardware/software stack to enable performant training and...  ...:Bachelor's degree in Electrical Engineering, Computer Engineering, Computer Science...  ...performance optimizations for ML training and inference.Experience with deep learning... 
    Worldwide

    Google

    Sunnyvale, CA
    2 days ago
  •  ...-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture...  ...on a global scale. From chips and software to rack-scale systems, our team is tackling...  ....The role: Principal System Software Engineer, AI Inference ExecutionWhat you will do:... 
    3 days per week

    d-Matrix

    Santa Clara, CA
    1 day ago
  • $152k - $241.5k

     ...company”.We are looking for an AI & Deep Learning Compiler Engineer. NVIDIA is hiring software engineers for its Deep Learning & AI Compiler (DLC) team...  ...of AI, our DLC has been the backbone of NVIDIA’s inference engine, spanning across data centers, personal devices,... 
    Full time
    Remote work

    Nvidia

    Santa Clara, CA
    1 day ago
  •  ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU-...  ...About the RoleWe’re hiring a senior backend engineer to build and scale the services that...  ...experience building and operating production software.Education. Bachelor’s or master’s degree... 

    Cerebras Systems

    Sunnyvale, CA
    3 days ago
  • $160.36k - $240.54k

     ...components. Develop observability to track ML model lifecycles from data generation to on-road validation. Maintain an in-house ML inference platform to serve large language models efficiently. Maintain an in-house ML compiler platform to compile, deploy, and validate... 
    Immediate start
    Flexible hours

    Nuro

    Mountain View, CA
    2 days ago
  • $160.36k - $240.54k

     ...components.  Develop observability to track ML model lifecycles from data generation to on-road validation. Maintain an in-house ML inference platform to serve large language models efficiently. Maintain an in-house ML compiler platform to compile, deploy, and validate... 
    Immediate start
    Flexible hours

    Nuro

    Mountain View, CA
    9 days ago
  • $280k - $350k

    About InworldInworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We build first...  ...running on capacity reserved for it.As a Staff or Principal Software Engineer, you'll design that platform's core architecture and write... 
    Full time
    Work at office
    Relocation

    Inworld AI

    Mountain View, CA
    16 hours ago
  • $135k - $175k

     ...is actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, INFERENCE (AI DATA ENGINEERING) The application software team is the central nervous system of SpaceX - we create mission... 
    Temporary work
    Remote work
    Worldwide
    Weekend work

    SpaceX

    Palo Alto, CA
    16 hours ago
  • $120k - $160k

     ...GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a...  ...most demanding workloads. About the role As a Junior Software Engineer (IC1/IC2) on the Inference team, you'll own a single... 
    Full time
    Temporary work
    Casual work
    Work at office
    Remote work
    Flexible hours

    CoreWeave

    Sunnyvale, CA
    23 days ago
  •  ...to run fast enough to control robots in the real world, and efficiently enough to support large-scale research. As an Inference Infrastructure Engineer Intern, you'll work with our infra and research engineers to make model inference faster and more efficient across the... 
    Internship

    rhoda-ai

    Mountain View, CA
    5 days ago
  • $194k - $349k

     ...capabilities into the product and use AI tools throughout your engineering workflow. Leadership & Impact As a technical leader, you...  ...knowledge of LLM fundamentals applied to real systems: inference behavior, hallucination patterns, non-determinism, context management... 
    Permanent employment
    Work experience placement
    Work at office
    Local area
    Sunnyvale, CA
    6 hours ago
  • $182k - $242k

     ...) in March 2025. Learn more at . About the Team Senior engineers are area owners who lead designs raise engineering standards and...  ...and hardware teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale. Key Responsibilities... 
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours
    Shift work

    CoreWeave

    Sunnyvale, CA
    2 days ago
  • $213k - $263k

     ...solutions to speed up developer velocity. We’re looking for a software engineer to join the team to build and maintain the critical data and...  ...Staff Software Engineer. You will: Develop Waymo's inference platform to make it scalable, high throughput, and low latency... 
    Full time
    Remote work

    Waymo

    Mountain View, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineer, Inference. Be the first to apply!