Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Software Engineer - Inference Performance

Baseten

Inference Performance Engineer Baseten powers mission-critical inference for the world's most dynamic AI companies. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. Join us and help build the platform engineers turn to ship AI products. We're looking for inference performance engineers who want to make the world's most demanding AI workloads run faster and more efficiently. You'll work across the stack, from the inference engine and runtime through scheduling, serving, and routing. Along the way you'll apply techniques like prefill/decode disaggregation, speculative decoding, and KV-cache management. You'll reason from first principles about where time and memory go, find what's holding performance back, and close the gap. Your work directly impacts how fast our customers' models run and how efficiently we serve them. This role is ideal for someone who thrives in a fast-paced startup environment and is eager to make significant contributions to the exciting field of LLM inference. You'll get to work on these types of projects as an Inference Performance engineer: Agentic Kernels in Production How we built the new fastest API for GLM-5.2 Live draft model training for speculative decoding Making Kimi K3 Tokenization 18x faster The Baseten Inference Stack Driving model performance optimization Implement and productionize cutting-edge inference techniques, working deep in runtime internals. That includes quantization, speculative decoding, KV-cache reuse, chunked prefill, LoRA, guided generation for structured outputs, and custom scheduling and routing algorithms. Profile and optimize inference end to end, from kernel launch overhead and memory layout up to request scheduling, prefill/decode disaggregation, and cache-aware routing. Run cross-layer investigations, such as tracing a tail-latency regression from request timing through routing and batching down to a kernel. Turn performance into cost savings. Improve tokens per GPU-hour, raise utilization, and give customers and internal teams clear latency/throughput/cost tradeoffs. Bring up and tune new model architectures on new hardware quickly, often in the same week they're released. Build benchmarking frameworks that measure real-world performance across model architectures, batch sizes, sequence lengths, and hardware configurations. Contribute upstream to open-source inference engines (vLLM, SGLang, TensorRT-LLM), and partner closely with model, infrastructure, and customer-facing teams to ship wins. Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or related field. Experience with one or more general-purpose programming languages, such as Python or C++. Familiarity with LLM optimization techniques (e.g., quantization, speculative decoding, continuous batching). Strong familiarity with ML libraries, especially PyTorch, TensorRT, or TensorRT-LLM. Demonstrated interest and experience in LLMs. Deep understanding of GPU architecture. Proficiency in enhancing the performance of software systems, particularly in the context of large language models (LLMs) Contributed to vLLM, SGLang, TensorRT-LLM, or another inference engine. Worked on large-scale distributed serving: autoscaling, load balancing, multi-region or multi-cloud capacity. Written or optimized GPU kernels (CUDA, Triton, CUTLASS, or similar) Worked on quantization (FP8/FP4, AWQ, GPTQ) or speculative decoding in production. Deep understanding of software engineering principles and a proven track record of developing and deploying AI/ML inference solutions. Competitive compensation, including meaningful equity (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) Paid parental leave Fertility and family-building stipend through Carrot (U.S. only) Company-facilitated 401(k) Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable). Baseten

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Software Engineer - Inference Performance in Seattle, WA vacancy
  • $143.7k - $194.4k

    AWS Neuron is the complete software stack for AWS Inferentia and...  .... Join the Machine Learning Inference Applications team to build the...  ...on Neuron chips.As an engineer on this team, you'll work on...  ...SGLang, optimizing model serving performance on Neuron and broadening the... 
    Performance
    Internship
    Flexible hours

    Amazon

    Seattle, WA
    4 days ago
  • $168.1k - $227.4k

    AWS Neuron is the complete software stack for AWS Inferentia and Trainium, AWS...  ...machine learning. This senior software engineering role is part of the Machine Learning Inference Applications team and focuses on delivering high-performance model inference solutions for... 
    Performance
    Work experience placement
    Internship
    Local area
    Flexible hours

    Amazon

    Seattle, WA
    5 days ago
  •  ...Member Baseten powers mission-critical inference for the world's most dynamic AI companies...  .... Join us and help build the platform engineers turn to ship AI products. Voice is...  ...with Forward Deployed Engineers, Model Performance Engineers, and sister engineering teams... 
    Performance
    Flexible hours

    Baseten

    Seattle, WA
    2 days ago
  • $182k - $242k

    Senior Engineer CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers...  ...combines superior infrastructure performance with deep technical expertise to accelerate...  ...teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale... 
    Performance
    Temporary work
    Casual work
    Work at office
    Flexible hours
    Shift work

    GenAI.Jobs

    Bellevue, WA
    2 days ago
  • $120k - $160k

     ...CoreWeave combines superior infrastructure performance with deep technical expertise to...  .../ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost...  ...workloads. About the role As a Junior Software Engineer (IC1/IC2) on the Inference... 
    Performance
    Full time
    Temporary work
    Casual work
    Work at office
    Remote work
    Flexible hours

    CoreWeave

    Bellevue, WA
    2 days ago
  • $150.4k - $277.6k

    Role Number: 200690248Team: Software and ServicesLocation: Cupertino; SeattlePosted...  ...is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute...  ...network protocols and write high performance code. You'll also build scaling... 
    Performance
    Remote work
    Worldwide
    Relocation

    Apple

    Seattle, WA
    2 days ago
  •  ...anchors on two systems. The first is our inference control plane — open-weight models and...  ...before you go home — building the fork engine, the guest agent, and the multi-substrate...  ...reliable, metered production endpoints. ~ Performance discipline in distributed systems: you... 
    Performance
    Full time
    Remote work
    Work visa
    Flexible hours
    Day shift

    careers.azx.io

    Seattle, WA
    8 days ago
  • $262.7k - $355.4k

    As a Principal Software Engineer on our AI Inference Runtime team, you will set technical direction for critical components of distributed Inference...  ...workload execution, kernel development and optimization, and performance benchmarking and analysis. Your work will directly... 
    Performance
    Work at office
    Local area

    ARM

    Seattle, WA
    2 days ago
  • $254k - $350k

     ...WA / Remote (United States)Software - Software Systems /Full-time...  ...you excited to drive our ML Performance Optimization initiatives and...  ...alongside a team of strong software engineers and act as a force...  ...cutting-edge ML Training OR Inference performance optimization techniques... 
    Performance
    Full time
    Remote work

    Zoox

    Seattle, WA
    4 days ago
  • $152k - $204k

     ...CoreWeave combines superior infrastructure performance with deep technical expertise to...  ...more at  . What You’ll Do: Senior engineers are area owners who lead designs, raise...  ...teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale... 
    Performance
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours
    Shift work

    CoreWeave

    Bellevue, WA
    4 days ago
  •  ...group of committed researchers, engineers, policy experts, and...  ...About the role Anthropic's Inference organization serves Claude to...  ...inference serving stack, whose performance, correctness, and...  ...of them Have significant software engineering experience, with... 
    Performance
    Full time
    Work at office
    Visa sponsorship
    Flexible hours

    Anthropic

    Seattle, WA
    16 days ago
  • $160k - $250k

     ...products, and what we learn from people using those products shapes what we build next. SOFTWARE ENGINEER, TRAINING/INFERENCE (C++) At SpaceXSI we are building the high-performance inference platform that serves Grok to millions of users every day. As a Software... 
    Performance
    Temporary work

    xAI

    Seattle, WA
    12 days ago
  • $139k - $204k

     ...CoreWeave combines superior infrastructure performance with deep technical expertise to...  .../ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost...  ...role We are looking for Senior Software Engineer to help us understand, measure... 
    Performance
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    Bellevue, WA
    4 days ago
  • $209.1k - $282.9k

    As an engineer on Arm’s AI Inference Cloud team, you will shape the technical direction and develop highly...  ..., and product teams to enhance the performance and usability of Arm’s AI platform....  ...lifecycle management.Strong software and production engineering skills, including... 
    Performance
    Work at office
    Local area
    Visa sponsorship
    Relocation package

    ARM

    Seattle, WA
    2 days ago
  • $175k - $308.5k

     ...SummaryWe are the Foundation Model Inference team within Cloud OS and AI...  ...to build the most highly performant, secure and private...  ...inference efficiency, hardware/software codesign, systems architecture...  ...technical direction for the engineers around you.This role sits within... 
    Performance
    Worldwide
    Relocation

    Apple

    Seattle, WA
    2 days ago
  • $175k - $308.5k

    Role Number: 200692468Team: Software and ServicesLocation: SeattleWeekly...  ...are looking for senior engineer to build tooling, automation...  ...that strengthen our AI inference platform. This role will focus...  ...on developing sophisticated performance benchmarking systems, capacity... 
    Performance
    Worldwide
    Relocation

    Apple

    Seattle, WA
    1 day ago
  • Staff + Sr. Software Engineer, Cloud Inference Launch Engineering San Francisco, CA | Seattle, WA About Anthropic Anthropic's mission is to create...  ...for every inference change — model launches, performance improvements, safeguard integrations — landing on cloud... 
    Performance
    Work at office
    Visa sponsorship
    Flexible hours

    Anthropic

    Seattle, WA
    2 days ago
  • $320k

     ...group of committed researchers, engineers, policy experts, and...  ...systems. About the role The Cloud Inference team scales and optimizes...  ...LLMs meet rigorous safety, performance, and security standards. Key...  ...qualifications Have significant software engineering experience, with... 
    Performance
    Work at office
    Local area
    Visa sponsorship
    Flexible hours

    Anthropic

    Seattle, WA
    2 days ago
  •  ...individual can thrive.Job DescriptionThe AI Inference Engineer plays a critical role in the AI lifecycle by bridging the gap between high-performance model development and optimized...  ...inference routing and orchestration. Ensure software solutions are optimized for peak... 
    Performance
    Full time
    Local area
    Immediate start

    F5 Networks

    Seattle, WA
    5 days ago
  •  ...CoreWeave combines superior infrastructure performance with deep technical expertise to...  ...Learn more at  Senior Deployment Engineer - LLM Inference & Optimization About the role We'...  ...ll bring ~5+ years of experience in software engineering, ML engineering, AI... 
    Performance
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    Seattle, WA
    8 days ago
  •  ...ElastixAI is an early-stage Software startup on a mission to reinvent AI inference infrastructure from the ground up...  ...philosophy is simple: the best performance comes from holistic co-design, where...  ...Inference Infrastructure Software Engineer to own and evolve the cloud and... 
    Performance
    Remote job
    Full time
    Work at office
    Flexible hours
    3 days per week

    elastix

    Seattle, WA
    7 days ago
  • $150k - $286k

     ...more efficient. We’re looking for a software optimization engineer to help us get more useful work done...  ...workloads as systems: measure how they perform, identify what limits useful...  ...the crowd. Optimizing AI training, inference, evaluation, or other large-scale GPU... 
    Performance
    Full time

    Nvidia

    Seattle, WA
    2 days ago
  •  ...advanced advertising solutions, including large-scale model training, high-performance inference, feature platforms, and embedding services. We are looking for exceptional Software Engineers who are passionate about AI infrastructure, high-performance computing,... 
    Performance
    Internship

    Mintegral

    Seattle, WA
    4 days ago
  •  ...technical leader to join the AI Software group. As a Fellow, you will...  ...to achieve industry-leading performance for our top-tier customers....  ...engagement, and software engineering, ensuring that AMD’s software...  ...of optimizing distributed inference and training at scale across... 
    Performance

    AMD

    Bellevue, WA
    1 day ago
  • $143.7k - $194.4k

     ...operates the recommendation engine serving every marketplace, every...  ...per day. We're looking for a Software Development Engineer to help...  ...real-time machine learning inference, AI-powered automated merchandising...  ..., all built for low-latency performance and resilience through the... 
    Performance
    Internship
    Worldwide
    Flexible hours
    Shift work

    Amazon

    Seattle, WA
    4 days ago
  • $143.7k - $194.4k

     ...customized stack of hardware, firmware, and software to deliver unparalleled virtualization...  ...EC2 Supercomputers, optimized for high-performance training and inference workloads. We are looking for an experienced software engineer to drive development for new EC2 machine... 
    Performance
    Internship
    Flexible hours

    Amazon

    Seattle, WA
    1 day ago
  • $168.1k - $227.4k

     ...NeuronWe build Amazon Neuron, the software development kit used to...  ....As a Senior Software Engineer on our Machine Learning Applications...  ...of machine learning, high-performance computing, and distributed...  ...efforts in building distributed inference support for Pytorch in the... 
    Performance
    Internship
    Work from home
    Flexible hours

    Amazon

    Seattle, WA
    4 days ago
  • $168.1k - $227.4k

    AWS Neuron is the complete software stack for the AWS Inferentia...  ...As the Software Development Engineer for the Neuron Foundation Tools...  ...develop and maintain high-performance monitoring and observability...  ...scale distributed training and inference solutions. This organization... 
    Performance
    Internship
    Work from home
    Flexible hours

    Amazon

    Seattle, WA
    1 day ago
  • $165.2k - $223.6k

     ...) is building a central pipeline of Software Development Engineer (SDE) talent for anticipated roles in...  ...new features- Identify and resolve performance bottlenecks and bugs- Participate in...  ...transformer architecture, training/inference lifecycles, and optimization techniques... 
    Performance
    Internship
    Local area
    Flexible hours
    Day shift

    Amazon

    Seattle, WA
    2 days ago
  • $168.1k - $227.4k

     ...dashboard summary can provide.As a Senior Software Development Engineer on the Log-Level Data team, you will...  ...and methodologies to improve system performance and capabilitiesBasic qualifications...  ..., including architecture, training/inference lifecycles, and optimization of... 
    Performance
    Internship
    Flexible hours

    Amazon

    Seattle, WA
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineer - Inference Performance. Be the first to apply!