AI Infrastructure & Accelerator Architect

Salary: $65,000 - 102,000 per year Requirements:

• We require a bachelors degree in Computer Science, Electrical Engineering, or equivalent practical experience; a masters degree or Ph.D. is preferred.
• We are looking for 15+ years of software engineering experience, including 8+ years in distributed systems, AI infrastructure, or high-performance computing architecture.
• You should have 7+ years of experience designing and building complex software systems in C++ or Python.
• We need 5+ years of experience leading the architecture, design, and delivery of large-scale software products, frameworks, or developer ecosystems from concept through production.
• You should have a proven track record of architecting performance-critical systems at the kernel level, connecting hardware accelerators with higher-level software frameworks.
• Deep expertise in TPU/GPU execution optimization using low-level kernel languages or abstractions such as Pallas, Mosaic, Triton, or CUDA is highly valued.
• Strong knowledge of modern ML frameworks such as JAX and PyTorch, as well as attention mechanisms, Mixture of Experts, model quantization, and low-precision arithmetic, is preferred.
• Advanced understanding of accelerator architectures, including heterogeneous compute, memory hierarchies, data movement optimization, and multi-node scale-out fabrics, is highly desirable.
• We prefer candidates with strong compiler fundamentals and familiarity with code generation and toolchains such as MLIR, OpenXLA, and LLVM.
• Experience building and scaling developer infrastructure, widely adopted open-source libraries, and extensible high-performance APIs is a plus.
• Exceptional strategic communication and stakeholder management skills, with a history of influencing cross-functional engineering teams, researchers, and executive leadership, are important.

• We define and lead the multi-year technical roadmap for high-performance AI kernels, custom operations, and hardware-software co-design targeting TPU and GPU architectures.
• We scale and mentor a world-class technical practice by setting architectural governance, engineering standards, and best practices across the organization.
• We act as the principal technical liaison, partnering with ML researchers, core framework architects such as JAX and PyTorch, and compiler engineering teams including XLA and MLIR to remove systemic bottlenecks and shape future hardware and software requirements.
• We architect foundational infrastructure, including enterprise-grade benchmarking suites, automated autotuning frameworks, regression analysis pipelines, and thorough documentation that supports the global developer community.
• We track industry shifts in hardware architectures, emerging model topologies, and compiler innovations to identify new opportunities for improving AI training and inference efficiency.

• AI
• Architect
• Cloud
• CUDA
• Hardware
• LLVM
• PyTorch
• Python
• NodeJS
• OSS

More:

We are seeking a Principal Architect to shape and execute our technical strategy for AI performance, optimization, and hardware-software co-design. In this highly visible role, we set the architectural vision for AI training and serving infrastructure, with broad impact across Google Cloud, major enterprise customers, and the open-source ecosystem. We lead a Center of Excellence and help advance next-generation TPU and GPU platforms, state-of-the-art ML models, and advanced compiler toolchains. This role is ideal for someone who thrives on solving difficult performance bottlenecks and pushing the boundaries of AI infrastructure.

last updated 39 week of 2026