Software Engineer - Voice AI (Inference Runtime)
Baseten
Baseten Voice AI Team Member Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. Voice is becoming the internet's next interface, but a production-grade Voice AI system is "hard to build". You'll join a small founding team of Baseten Voice AI, focused on bringing state-of-the-art open source models into production for Voice AI customers across productivity, customer service, clinical conversation, creator tools, education, and more. You'll make a meaningful impact on people's daily lives and help reshape these industries. This is a high-impact, high-ownership role. You will be the primary owner of Baseten Voice AI - our in-house inference stack to power Voice AI models - from product roadmap through engineering implementation. You'll partner closely with Forward Deployed Engineers, Model Performance Engineers, and sister engineering teams to push the boundaries of Voice AI. Example Initiatives Develop world-class model serving stack for state-of-the-art open-source voice models - reduce end-to-end and tail latency (p95/p99), increase throughput, and improve GPU efficiency via profiling, runtime tuning, and server-level optimizations. Build large-scale, real-time infrastructure for multi-model voice agents - orchestrate STT, TTS, and agent components with streaming I/O to meet customer SLOs. Design tight training and inference iteration loops for voice model customization - enable fast evaluation, safe rollout, and rapid experimentation for custom voice model development. Past projects: Baseten leads Coval's voice AI benchmark How Notion achieved 2-3x lower latency with AI Meeting Notes Cost-efficient, high-performance TTS with Qwen3-TTS Fine-tuning Qwen3-TTS for high-quality voice cloning The fastest Whisper — with streaming and diarization NVIDIA Nemotron 3 Diarization: real-time speaker labels at a cent per audio hour Responsibilities Own and lead Voice AI product areas end-to-end - from architecture and system design through implementation, rollout, and long-term production operations. Design, build, and operate real-time, large-scale, high-performance model serving systems for STT, TTS, and voice agent workloads for mission-critical customer deployments Drive cross-team collaboration with sister engineering teams to solve full-stack technical problems, align on priorities, and coordinate end-to-end delivery across the product surface area Mentor teammates through code reviews, design docs, and technical leadership. Requirements Bachelor's degree or higher in Computer Science or related field Proven track record owning production-grade real-time, large-scale systems where tail latency (p99) matters. Proficient coding abilities in one or more popular programming or scripting languages; Python proficiency is a plus. Good taste in product, particularly developer-oriented tools Interest in ML/AI infrastructure and willingness to learn Strong collaboration and communication skills Comfortable using AI coding assistants (e.g., Claude Code, Codex, Cursor) as a daily productivity multiplier — as an AI-native company, we see this as a must-have skill. Nice To Have Experience implementing pipeline-level model runtime optimizations such as dynamic batching, async scheduling, or decode-side throughput improvements. Experience building developer platforms: SDKs, CLIs, APIs, and self-serve workflows for ML or infrastructure products. Experience with containerization and orchestration technologies (Docker, Kubernetes), service meshes, or distributed scheduling. Familiarity with speech/audio ML models (STT, TTS, speech-to-speech) Familiarity with model-serving runtimes (vLLM, TensorRT, ONNX). Familiarity with systems-level performance profiling across host-device boundaries (e.g. PyTorch Profiler), diagnosing GPU utilization issues Exposure to customer-facing engineering: pre-sales prototyping, technical discovery, or working directly with customers to ship solutions. Benefits Competitive compensation, including meaningful equity (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) Paid parental leave Fertility and family-building stipend through Carrot (U.S. only) Company-facilitated 401(k) Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable). Baseten
- ...in critical industries through AI transformation. We specialize... ...two systems. The first is our inference control plane — open-weight models... ...go home — building the fork engine, the guest agent, and the... ...tested. Oversee the sandbox runtime and its host-side control plane...SuggestedFull timeRemote workWork visaFlexible hoursDay shift
$262.7k - $355.4k
As a Principal Software Engineer on our AI Inference Runtime team, you will set technical direction for critical components of distributed Inference runtime for running SOTA AI Models.You will lead hands-on work across scheduling, batching, KV-cache management, memory allocation...SuggestedWork at officeLocal area$168.1k - $227.4k
...Neuron is the complete software stack for AWS Inferentia... .... This senior software engineering role is part of the Machine Learning Inference Applications team and... ...compiler engineers, and runtime engineers to ensure end... ...Neuron, TPUs, or other AI accelerator hardware- Experience...SuggestedWork experience placementInternshipLocal areaFlexible hours$143.7k - $194.4k
...Neuron is the complete software stack for AWS Inferentia... ...the Machine Learning Inference Applications team to build... ...on Neuron chips.As an engineer on this team, you'll... ...performance, compiler, and runtime engineers to deliver... ..., Neuron, TPU or other AI acceleration hardware-...SuggestedInternshipFlexible hours$143.7k - $194.4k
AWS Neuron is the complete software stack for the AWS Inferentia... ...the Software Development Engineer for the Neuron Runtime Team, you will be responsible... ...learning applications and AI accelerators. You will work... ...scale distributed training and inference solutions. This...SuggestedInternshipWork from homeFlexible hours$120k - $160k
...is The Essential Cloud for AI™. Built for pioneers by... ...rendering, and real-time inference. Our stack is engineered for speed, scale, and cost... ...About the role As a Junior Software Engineer (IC1/IC2) on the... ...collaborate with peers across runtime, scheduling, and...Full timeTemporary workCasual workWork at officeRemote workFlexible hours- Inference Performance Engineer Baseten powers mission-critical inference for the world's most dynamic AI companies. By uniting applied AI research, flexible... ...inference engine and runtime through scheduling, serving... ...the performance of software systems, particularly in...Flexible hours
$150.4k - $277.6k
Role Number: 200690248Team: Software and ServicesLocation: Cupertino; SeattlePosted... ...SummaryApple is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute, the backbone of Siri AI and Apple Intelligence. You'll design...Remote workWorldwideRelocation$209.1k - $282.9k
As an engineer on Arm’s AI Inference Cloud team, you will shape the technical direction and develop highly... ...with AI compute, Inference Runtime, and product teams to enhance the performance... ...workload lifecycle management.Strong software and production engineering skills, including...Work at officeLocal areaVisa sponsorshipRelocation package$175k - $308.5k
Role Number: 200692468Team: Software and ServicesLocation: SeattleWeekly... ...are looking for senior engineer to build tooling, automation... ...that strengthen our AI inference platform. This role will focus... ...with ML serving frameworks and runtimes (e.g., Triton, TensorRT-LLM,...WorldwideRelocation- ...: ElastixAI is an early-stage Software startup on a mission to reinvent AI inference infrastructure from the ground up... ...Inference Infrastructure Software Engineer to own and evolve the cloud and... ...optimization, working closely with our ML, runtime, and hardware teams to expose the...Remote jobFull timeWork at officeFlexible hours3 days per week
$182k - $242k
Senior Engineer CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that... ...and hardware teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale. Key...Temporary workCasual workWork at officeFlexible hoursShift work$139k - $204k
...is The Essential Cloud for AI™. Built for pioneers by... ...rendering, and real-time inference. Our stack is engineered for speed, scale, and cost... ...We are looking for Senior Software Engineer to help us understand... ...across frameworks, runtimes, and hardware configurations...Full timeTemporary workCasual workWork at officeFlexible hours- ...interpretable, and steerable AI systems. We want AI to... ...committed researchers, engineers, policy experts, and... ...the role Anthropic's Inference organization serves... ...technical lead for Inference Runtime: the team that owns the... ...Have significant software engineering experience,...Full timeWork at officeVisa sponsorshipFlexible hours
$207.48k - $368.22k
...jobs daily, including AI and LLM workloads. The... ...like Ray on K8s, and LLM inference control plan project... ...'re seeking talented software engineers excited to optimize our... ...of container runtime environments to enhance... ...celebrating our diverse voices and to creating an environment...Temporary workInternshipLocal areaOverseas$167.8k - $246.8k
...customers are modelers and engineering teams across our... ...production, adopt open source software and models, and expand... ...retrieval with model inference on CPU and GPU... ...profiling, inference runtimes, or model deployment workflows... ...Regarding Use of AI and Automated Tools: To...Hourly payWork at officeLocal areaFlexible hours$177k - $216k
...are Metropolis is seeking a Senior Software Engineer to architect and build our AI agent swarm, orchestration... ...high-throughput, low-latency LLM inference routing, caching strategies, and semantic... ...inclusive culture where everyone has a voice and the best idea wins. You will...Temporary workWork at officeLocal area$175k - $308.5k
...SummaryWe are the Foundation Model Inference team within Cloud OS and AI Inference organization. We are on a... ...in inference efficiency, hardware/software codesign, systems architecture, and... ...set the technical direction for the engineers around you.This role sits within CloudOS...WorldwideRelocation- Staff + Sr. Software Engineer, Cloud Inference Launch Engineering San Francisco, CA | Seattle, WA About Anthropic Anthropic's mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a...Work at officeVisa sponsorshipFlexible hours
$181.63k - $254.28k
...Advanced Concepts and Enterprise Engineering (ACE), supporting Blue... ...seeking a highly skilled platform software engineer to own the Board... ...services, including the inference runtime and job orchestration layers... ...Passion for and understanding of AI and Machine Learning...Permanent employmentTemporary workLocal areaRemote work$143.7k - $194.4k
We build 1P AI capabilities — Deep Research, Artifact... ...within a unified cloud runtime. We're transforming how... ...— from prompt engineering and orchestration logic... ...perspectives, and passionate voices our teams bring to... ...internship professional software development experience-...InternshipFlexible hours$143.7k - $194.4k
We are looking for a Software Development Engineer II (SDE-2) to join the EKS Runtime Release team. In this role, you will design... ...your work will focus on enabling AI and ML workloads by implementing... ...to cutting edge AI training and inference.Mentorship: Mentor junior...InternshipWork from homeFlexible hours$168.1k - $227.4k
Shape the Future of AI Accelerators at AWS NeuronWe... ...Amazon Neuron, the software development kit used to... ...Trainium.As a Senior Software Engineer on our Machine Learning... ...building distributed inference support for Pytorch in... ...across compiler, runtime, framework, and hardware...InternshipWork from homeFlexible hours$320k
...interpretable, and steerable AI systems. We want AI to be safe... ...of committed researchers, engineers, policy experts, and business... ...systems. About the role The Cloud Inference team scales and optimizes... ...qualifications Have significant software engineering experience, with...Work at officeLocal areaVisa sponsorshipFlexible hours$175k - $308.5k
Role Number: 200694541Team: Software and ServicesLocation: SeattlePosted: 2026-11-12SummaryJoin the Apple Service Engineering (ASE) team and drive innovation that matters! The ASE team builds and provides systems and infrastructure that fuel Apple's services. As part of...RelocationShift work$143.7k - $194.4k
...builds AWS Neuron, the software development kit used to... ...includes an ML compiler, runtime, and application... ...enabling unparalleled ML inference and training performance... ...software boundary, our engineers build systematic infrastructure... ...of what's possible in AI acceleration.As part of...Work experience placementInternshipFlexible hours$167k - $209k
...developers to build, run, and scale AI-powered applications and autonomous coding agents. As a Senior Software Engineer on the MARS (Managed Agent Runtime Services) team, you will help build... ..., infrastructure, security, inference, and developer experience teams to...Local areaFlexible hours$146.88k - $220.32k
Senior Software Engineer, Agent Platform (AI for the Planet) Seattle, WA Our base salary range is $146,880 -... ...annotation; distributed training and inference; and a viewer so the outputs are... ...the application infrastructure: the runtime, the APIs, the tooling, and the operational...Work at officeLocal area$42.75 per hour
...training data orchestration to real-time online inference—that delivers seamless, high-frequency... ..., realtime online training, etc; Coach AI to perform release process and production... ...are committed to celebrating our diverse voices and to creating an environment that...Hourly paySummer workInternship$207.48k - $368.22k
...experience as a senior development engineer. 3. 3 years experience of... ...understanding of cloud infrastructure or AI infrastructure. 6. Familiar... ..., like training frameworks, Inference engines (vLLM, SGLang, etc),... ...to celebrating our diverse voices and to creating an environment...Temporary workLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer - Voice AI (Inference Runtime). Be the first to apply!
- senior software design engineer Seattle, WA
- software web developer Seattle, WA
- senior software engineer Seattle, WA
- software developer fintech Seattle, WA
- software support engineer Seattle, WA
- security software engineer Seattle, WA
- software developer night shift Seattle, WA
- scientific software engineer Seattle, WA
- software engineer - web development Seattle, WA
- new graduate software engineer Seattle, WA


