Inference Software Engineer
$2,000 per monthEtched
About Etched
Etched is building AI chips that are hard-coded for individual model architectures. Our first product (Sohu) only supports transformers, but has an order of magnitude more throughput and lower latency than a B200. With Etched ASICs, you can build products that would be impossible with GPUs, like real-time video generation models and extremely deep & parallel chain-of-thought reasoning agents.
Key responsibilities
- Contribute to the architecture and design of the Sohu host software stack
- Implement high-performance, modular code across the complete Etched software stack, consisting of a mix of Rust, C++ and Python.
- Interface with firmware and drivers teams delivering highest-performance HW/SW stack.
- Work with AI model researchers and product-facing teams building out the Etched serving front-end.
Representative projects
- Build scheduling logic for handling continuous batching and real time inference
- Implement inference-time acceleration techniques such as speculative decoding, tree search, KV cache sharing, etc.
- Implement distributed networking primitives for efficient multi-server inference
You may be a good fit if you have
- Experience with C++ and Python
- Familiarity with transformer model architectures and inference serving stacks (vLLM, SGLang, etc.) or experience working in distributed inference/training environments
- Experience working cross-functionally in large software and hardware organizations
Strong candidates may also have
- Experience with Rust
- Familiarity with GPU kernels, the CUDA compilation stack and related tools, or other hardware accelerators
- Understanding of distributed systems, networking, and parallel programming
Benefits
- Full medical, dental, and vision packages, with 100% of premium covered
- Housing subsidy of $2,000/month for those living within walking distance of the office
- Daily lunch and dinner in our office
- Relocation support for those moving to Cupertino
How we’re different
Etched believes in the Bitter Lesson . We think most of the progress in the AI field has come from using more FLOPs to train and run models, and the best way to get more FLOPs is to build model-specific hardware. Larger and larger training runs encourage companies to consolidate around fewer model architectures, which creates a market for single-model ASICs.
We are a fully in-person team in Cupertino, and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both as needed.
- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...ultra high-speed inference.About the RoleWe're hiring a Software Engineer to help contribute to projects on our Inference Platform...Suggested
- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...with ultra-fast decode on the Cerebras Wafer-Scale Engine.We are hiring a Software Engineer to productionize and optimize our GPU serving stack...Suggested
- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...high-speed inference.About the RoleWe’re hiring a Senior Software Engineer for our Inference Cloud team. This team owns the cloud layer...Suggested
$150.4k - $277.6k
Role Number: 200690248Team: Software and ServicesLocation: Cupertino; SeattlePosted: 2026-10-19SummaryApple is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute, the backbone of Siri AI and Apple Intelligence...SuggestedRemote workWorldwideRelocation$193.3k - $261.5k
...layer is what our team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and... ...will:Build and optimize the low-level data-movement software across accelerators, servers, and heterogeneous...SuggestedInternshipLocal areaFlexible hours$165.2k - $223.6k
...layer is what our team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and... ...will:Build and optimize the low-level data-movement software that transfers KV cache and activations across...InternshipLocal areaFlexible hours$127.1k - $185k
...Services (AWS) builds the AWS Neuron SDK, the software development kit used to accelerate deep... ...learning accelerators.The Neuron Inference organization is at the forefront of optimizing... ...to the hardware-software boundary, our engineers build systematic infrastructure, develop...InternshipLocal areaFlexible hours$165.2k - $223.6k
...Services (AWS) builds the AWS Neuron SDK, the software development kit used to accelerate deep... ...learning accelerators.The Neuron Inference organization is at the forefront of optimizing... ...to the hardware-software boundary, our engineers build systematic infrastructure, develop...InternshipLocal areaFlexible hours$184k - $287.5k
NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a Senior Software Engineer - AI Inference Performance to advance innovative LLM and VLM inference. You will push workloads toward practical performance limits on NVIDIA GPU-accelerated...Full time- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ....About the TeamThe Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure...
$152k - $241.5k
NVIDIA is seeking talented engineers to join the TensorRT team and advance high-performance AI inference on NVIDIA GPUs. Our team develops software that accelerates state-of-the-art generative AI models for local and enterprise deployments. We solve challenging problems...Full timeLocal area- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ...with ultra-fast decode on the Cerebras Wafer-Scale Engine.We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes...
$193.3k - $261.5k
We develop AWS Neuron, the complete software stack for Trainium, Amazon's custom cloud-scale machine learning accelerators. Join... ...on the Trainium hardware.As a Sr. Software Development Engineer on the Inference Model Enablement team, you will onboard and optimize state...InternshipLocal areaFlexible hours- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ....About the TeamThe Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure...Remote work
$165.2k - $223.6k
...Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep... ...and JAX enabling unparalleled ML inference and training performance.The Inference Enablement... ...the hardware-software boundary, our engineers build systematic infrastructure, innovate...Work experience placementInternshipLocal areaFlexible hours$192k - $278k
...definition and optimization of the hardware/software stack to enable performant training and... ...:Bachelor's degree in Electrical Engineering, Computer Engineering, Computer Science... ...performance optimizations for ML training and inference.Experience with deep learning...Worldwide- ...-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture... ...on a global scale. From chips and software to rack-scale systems, our team is tackling... ....The role: Principal System Software Engineer, AI Inference ExecutionWhat you will do:...3 days per week
$160.36k - $240.54k
...components. Develop observability to track ML model lifecycles from data generation to on-road validation.Maintain an in-house ML inference platform to serve large language models efficiently.Maintain an in-house ML compiler platform to compile, deploy, and validate Nuro...Immediate startFlexible hours$152k - $241.5k
...company”.We are looking for an AI & Deep Learning Compiler Engineer. NVIDIA is hiring software engineers for its Deep Learning & AI Compiler (DLC) team... ...of AI, our DLC has been the backbone of NVIDIA’s inference engine, spanning across data centers, personal devices,...Full timeRemote work- ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU-... ...About the RoleWe’re hiring a senior backend engineer to build and scale the services that... ...experience building and operating production software.Education. Bachelor’s or master’s degree...
$135k - $175k
...is actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, INFERENCE (AI DATA ENGINEERING) The application software team is the central nervous system of SpaceX - we create mission...Temporary workRemote workWorldwideWeekend work$120k - $160k
...GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a... ...most demanding workloads. About the role As a Junior Software Engineer (IC1/IC2) on the Inference team, you'll own a single...Full timeTemporary workCasual workWork at officeRemote workFlexible hours$194k - $349k
...capabilities into the product and use AI tools throughout your engineering workflow. Leadership & Impact As a technical leader, you... ...knowledge of LLM fundamentals applied to real systems: inference behavior, hallucination patterns, non-determinism, context management...Permanent employmentWork experience placementWork at officeLocal area$182k - $242k
...) in March 2025. Learn more at . About the Team Senior engineers are area owners who lead designs raise engineering standards and... ...and hardware teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale. Key Responsibilities...Full timeTemporary workCasual workWork at officeFlexible hoursShift work$139k - $204k
...GPU workloads across AI/ML, VFX, rendering, and real-time inference. Our stack is engineered for speed, scale, and cost efficiency—providing a... ...workloads. About the role We are looking for Senior Software Engineer to help us understand, measure, and improve the...Full timeTemporary workCasual workWork at officeFlexible hours$152k - $204k
...in March 2025. Learn more at . What You’ll Do: Senior engineers are area owners who lead designs, raise engineering standards,... ...orchestration, and hardware teams to evolve our Kubernetes-native inference platform and meet strict P99 SLAs at scale. About the role:...Full timeTemporary workCasual workWork at officeFlexible hoursShift work$175k - $308.5k
...08-13SummaryWe are the Foundation Model Inference team within Cloud OS and AI Inference organization... ...in inference efficiency, hardware/software codesign, systems architecture, and... ...help set the technical direction for the engineers around you.This role sits within CloudOS...WorldwideRelocation$200k - $375k
...scaling, batch scheduling, and global KV caching. Optimize model inference latency, throughput, concurrency, reliability, and tail latency... ...scale. Benchmark, fine-tune, and accelerate inference engines through GPU kernel work and code generation. Develop tools to...Full timeTemporary work$224k - $356.5k
We are now looking for a Senior System Software Engineer to work on Dynamo. NVIDIA is hiring software engineers for its GPU-accelerated deep... ...processing. We are a fast-paced team building Generative AI inference platform to make design and deployment of new AI models...Full time$155k - $235k
...-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture... ...on a global scale. From chips and software to rack-scale systems, our team is tackling... ...by hard problems.Staff Test & Release Engineer, Inference CloudLocation: Santa Clara,...Remote workShift work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Inference Software Engineer. Be the first to apply!
- senior software design engineer Cupertino, CA
- senior software engineer Cupertino, CA
- software developer fintech Cupertino, CA
- senior software engineer ruby on rails Cupertino, CA
- intel software engineer Cupertino, CA
- software developer positions Cupertino, CA
- software data engineer Cupertino, CA
- software development engineer aws Cupertino, CA
- junior software developer remote Cupertino, CA
- remote entry level software developer Cupertino, CA



