ML Software Engineer, Data Plane

Salary: $165,200 - 223,600 per year Requirements:

• Bachelors degree or equivalent
• 4 years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations
• Knowledge of computer architecture, operating systems, and parallel computing
• Strong understanding of Linux fundamentals
• Strong proficiency in C/C++
• Experience building compute kernels for GPUs, DSPs, or custom accelerators
• Demonstrated success owning and delivering complex software features end to end
• Knowledge of machine learning and LLM fundamentals, including transformer architecture, training and inference lifecycles, and optimization methods
• Familiarity with ML frameworks such as JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, and TensorRT
• Experience developing and deploying LLMs in production on GPUs, Neuron, TPU, or other AI acceleration hardware
• Experience with distributed systems, including collective communication, RDMA, or high-speed interconnect programming
• Experience with hardware simulation environments and model validation workflows
• Demonstrated use of AI-assisted development tools, including LLMs or code-generation agents, in daily work

• Develop and optimize compute kernels for a custom ML accelerator architecture to achieve production-level performance for large language model inference
• Implement and validate LLM architectures end to end, from PyTorch model definition through distributed execution on custom hardware
• Integrate custom accelerator backends into open-source ML serving frameworks such as vLLM and PyTorch, including scheduler extensions, memory management, and model parallelism
• Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets
• Profile and tune inference workloads by identifying bottlenecks, instrumenting critical paths, and improving latency and throughput from simulation through hardware bring-up
• Own features from design through implementation, testing, and integration into the broader software stack
• Contribute to CI/CD pipelines that gate model and kernel changes on correctness and performance regressions

• AI
• CI/CD
• Hardware
• LLM
• Linux
• Machine Learning
• PyTorch
• RDMA
• vLLM
• Support

More:

We are the MLIL DataPlane team, building the software that enables large models to run efficiently on custom hardware across model execution, memory management, data movement, and serving integration. Our work spans the full inference path, from integrating serving engines with custom hardware to developing high-performance compute kernels and driving models from early validation into production. We operate at frontier scale with large distributed models, in a ground-up environment with rapidly evolving hardware and software. We offer comprehensive benefits including health insurance, 401(k) matching, paid time off, parental leave, sign-on payments, and restricted stock units. The base salary range for this role in Cupertino, CA is USD 165,200 to 223,600 annually.

last updated 37 week of 2026