Sr. ML Kernel Performance Engineer, AWS Neuron

Salary: $193,500 - 193,500 per year Requirements:

• 5+ years of non-internship professional software development experience
• 5+ years of programming experience in at least one software language
• 5+ years of experience leading design or architecture for new and existing systems, including design patterns, reliability, and scaling
• 5+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations
• Experience serving as a mentor, tech lead, or engineering team lead
• Bachelors degree in computer science or an equivalent background
• 6+ years of full software development experience
• Expertise in accelerator architectures for ML or HPC, such as GPUs, CPUs, FPGAs, or custom architectures
• Experience with GPU kernel optimization and GPGPU computing, such as CUDA, NKI, Triton, OpenCL, SYCL, or ROCm
• Demonstrated experience with NVIDIA PTX and/or AMD GPU ISA
• Experience building high-performance libraries for HPC applications
• Proficiency in low-level performance tuning for GPUs
• Experience developing LLVM/MLIR backends for GPUs
• Knowledge of ML frameworks such as PyTorch or TensorFlow and their GPU backends
• Experience with parallel programming and optimization methods
• Understanding of GPU memory hierarchies and optimization strategies

• Design and build high-performance compute kernels for ML operations using the Neuron architecture and programming models
• Analyze and improve kernel-level performance across multiple generations of Neuron hardware
• Use profiling tools to perform detailed performance analysis and identify bottlenecks
• Develop compiler optimizations including fusion, sharding, tiling, and scheduling
• Partner directly with customers to enable and optimize their ML models on AWS accelerators
• Work across teams to create innovative kernel optimization techniques
• Architect and implement business-critical features
• Publish cutting-edge research
• Mentor experienced engineers

• AWS
• Architect
• CUDA
• Hardware
• LLVM
• Machine Learning
• PyTorch
• TensorFlow
• Web
• Cloud
• AI
• Backbone
• Backend
• Flow
• Support

More:

We are the Annapurna Labs team at Amazon Web Services, building AWS Neuron, the software development kit that accelerates deep learning and GenAI workloads on our custom machine learning accelerators, Inferentia and Trainium. Our Acceleration Kernel Library team focuses on maximizing performance at the hardware-software boundary, crafting high-performance kernels for ML functions so every FLOP counts for customer workloads. We work across the Neuron Compiler organization, spanning frameworks, compilers, runtime, and collectives, while also contributing to future architecture designs and collaborating closely with customers on model enablement. This role offers the chance to work on cutting-edge products at the intersection of machine learning, high-performance computing, and distributed architectures. We value an innovative, agile, and collaborative culture, and we offer flexible working arrangements, a strong work-life balance, inclusive team programs, and comprehensive benefits including health coverage, retirement matching, paid time off, parental leave, and sign-on payments and RSUs. The position is based in Cupertino, California, with a listed annual base salary of 193,500.00 USD.

last updated 37 week of 2026