Software Development Engineer, AI/ML, AWS Neuron

Salary: $129,300 - 223,600 per year Requirements:

• A Bachelors degree in computer science or equivalent field.
• At least 5 years of professional software development experience (non-internship).
• Minimum of 5 years in system design or architecture, focusing on design patterns, reliability, and scalability of both new and existing systems.
• Fundamental knowledge of machine learning and large language models (LLMs), including their architecture, training, and inference processes, along with experience in optimizations for enhanced model execution.
• Proficiency in software development using C++ or Python (experience in at least one language is essential).
• Strong understanding of system performance, memory management, and principles of parallel computing.
• Experience in debugging, profiling, and applying best practices in software engineering for large-scale systems.

Responsibilities:

• Design, develop, and enhance machine learning models and frameworks for deployment on custom ML hardware accelerators.
• Engage in all phases of the ML system development lifecycle, including distributed computing architecture design, implementation, performance profiling, hardware-specific optimizations, testing, and production deployment.
• Create infrastructure for systematic analysis and onboarding of diverse models.
• Develop and implement high-performance kernels and features for ML operations, utilizing the Neuron architecture and programming models.
• Conduct performance analysis across various generations of Neuron hardware and use profiling tools to identify and address performance bottlenecks.
• Apply optimizations such as fusion, sharding, tiling, and scheduling.
• Execute comprehensive testing, including unit and end-to-end model testing with continuous deployment.
• Collaborate directly with customers to enable and optimize their ML models on AWS accelerators.
• Coordinate with various teams to innovate optimization techniques.

Technologies:

• AI
• AWS
• Hardware
• Machine Learning
• Python
• Web
• Cloud
• Architect
• Backbone
• CUDA
• GitHub
• Support
• LLM
• PyTorch

More:

Our Annapurna Labs team at Amazon Web Services (AWS) is focused on building AWS Neuron, the software development kit that accelerates deep learning and Generative AI workloads on our custom machine learning accelerators, Inferentia and Trainium. We are dedicated to accelerating models and supporting novel architectures, ensuring our infrastructure is optimized for demanding workloads.

We combine deep hardware knowledge with machine learning expertise, pushing the limits of AI acceleration. You will have the unique opportunity to work at the intersection of machine learning, high-performance computing, and distributed architectures, influencing the future of AI technology. Our team fosters a culture of innovation and experimentation, emphasizing collaboration, technical ownership, and continuous learning.

In this role, you can expect to work within a small, agile team, mentored by experienced engineers, as we tackle challenging infrastructure problems in AI/ML. We celebrate knowledge sharing and prioritize the growth of our team members. Join us as we strive to solve some of the most pressing challenges in the field today.

last updated 37 week of 2026