Sr. Software Dev Engineer AI/ML, AWS Neuron

Salary: $193,300 - 261,500 per year Requirements:

• 5+ years of programming experience in a modern language such as Java, C++, or C#, with object-oriented design experience
• 5+ years of experience leading design or architecture for new and existing systems, including design patterns, reliability, and scaling
• 5+ years of full software development lifecycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations
• 5+ years of non-internship professional software development experience
• Experience mentoring, serving as a tech lead, or leading an engineering team
• Masters degree in computer science or an equivalent background
• Experience with machine learning and large language model fundamentals, including architecture, training and inference lifecycles, and model execution optimization

• Build high-performance models using distributed inference libraries
• Improve performance optimization and system reliability across our Neuron ecosystem
• Mentor team members and provide technical leadership across multiple work streams
• Influence architectural decisions that affect our entire Neuron serving stack
• Partner with customers, product owners, and engineering teams to shape technical strategy
• Write technical documentation, design proposals, and architectural guidelines
• Lead design reviews and architectural discussions
• Troubleshoot complex performance issues across the stack with our compiler and runtime teams
• Coach junior engineers on system design and model optimization across model enablement teams
• Drive technical choices that shape the future of our inference stack

• AWS
• C#
• Cloud
• Java
• Machine Learning
• vLLM
• Architect
• DevOps
• GitHub
• Hardware
• Support

More:

We develop AWS Neuron, the complete software stack for Trainium, our custom cloud-scale machine learning accelerators. In this Senior Software Development Engineer role on the Inference Model Enablement team, we focus on onboarding and optimizing open-source and customer LLMs, including dense and MoE models, for inference on Trainium. Our team releases its models in the vLLM Neuron plugin. We offer comprehensive benefits, including health coverage, 401(k) matching, paid time off, parental leave, sign-on payments, and RSUs. The listed base salary range for Cupertino, CA is 193,300.00 to 261,500.00 USD annually.

last updated 37 week of 2026