Head of AI Infrastructure

Job Description

Head of AI Infrastructure

\n

\n

New York City | Hybrid / Onsite

\n

\n

We are hiring a Head of AI Infrastructure to lead the architecture, deployment and operation of a rapidly scaling GPU and HPC platform.

\n

\n

This is a senior technical leadership role with end-to-end ownership across compute, networking, cluster software, workload orchestration and production operations.

\n

\n

You will set the technical direction for the infrastructure function, remain close to the technology and help define how the next generation of large-scale GPU systems are designed, deployed and operated.

\n

\n

Responsibilities:

\n

\n

Own the technical strategy and architecture for large-scale GPU and HPC infrastructure supporting demanding AI training and inference workloads.

\n

Design and scale modern NVIDIA GPU environments, including HGX/DGX platforms and H100, H200 and Blackwell-generation systems.

\n

Lead infrastructure architecture across InfiniBand, RoCE, RDMA, Ethernet, NCCL, NVLink/NVSwitch and high-performance storage.

\n

Establish standards across bare-metal provisioning, firmware, cluster deployment, burn-in, benchmarking, validation and production acceptance.

\n

Own cluster performance, reliability and availability across GPU compute environments.

\n

Drive improvements across GPU utilisation, node health, telemetry, observability, failure detection and automated remediation.

\n

Lead workload orchestration across technologies such as Slurm, Kubernetes and containerised environments.

\n

Develop an automation-first approach to provisioning, cluster operations, diagnostics and infrastructure management.

\n

Work directly with datacentre operators, OEMs, networking vendors, infrastructure suppliers and engineering teams.

\n

Build and lead a high-performing AI infrastructure team across GPU systems, HPC, networking and platform engineering.

\n

Define technical standards, engineering processes and operational best practices as the infrastructure estate scales.

\n

\n

Experience:

\n

We are looking for someone with deep experience building and operating large-scale infrastructure across several of the following areas:

\n

\n

NVIDIA HGX/DGX and H100/H200/Blackwell -class GPU platforms

\n

InfiniBand, RoCE, RDMA and high-performance Ethernet

\n

NCCL, NVLink, NVSwitch, CUDA and DCGM

\n

Slurm, Kubernetes, Docker and Linux systems engineering

\n

Bare-metal provisioning and infrastructure automation

\n

HPC benchmarking, GPU validation and production acceptance

\n

High-performance and distributed storage

\n

Datacentre and rack-scale infrastructure

\n

Python, Go, Bash or Ansible

\n

Distributed AI training and inference environments

\n

\n

Background:

\n

You may currently be working as a Head of Infrastructure, Director of Infrastructure, GPU Infrastructure Lead, Head of HPC, Principal HPC Engineer, Staff or Principal Infrastructure Engineer, AI Infrastructure Architect or senior technical leader within a neocloud, hyperscaler, AI lab or HPC environment.

\n

\n

Compensation: $275,000–$350,000 base salary plus bonus and equity, depending on experience.