Principal Core Infrastructure Engineer - GPU Validation

Join the HPC team to develop automated tools and validation frameworks that ensure GPU racks are ready for production. The role includes building NCCL performance benchmarks, GPU infrastructure and NVLink connectivity tests, Slurm-based validation workflows, and automated FDT and cable diagnostics.

Designs, implements, and optimizes components in distributed systems with an emphasis on scalability, resiliency, and operability.

Designs, implements, and optimizes components in distributed systems with an emphasis on scalability, resiliency, and operability. Delivers features and load/performance tests; leverages data plane platforms and distributed state tools for high-volume retrieval, storage, and processing; and reviews peers' implementations for scalability compliance. Builds fault-tolerant paths (redundancy, replication, automatic failover), applies recovery-oriented principles, and implements retries, circuit breakers, and timeouts. Proactively detects and mitigates issues via tests, alarms, dashboards, and telemetry; authors runbooks and participates in incident response and RCAs. Implements standard replication and synchronization, develops automation/IaC for troubleshooting and maintenance, and applies advanced security controls (encryption, access, remediation) while ensuring change, compliance, and documentation standards are met.

Must live or be willing to relocate so you are able to regularly go into the Nashville, TN office.