Principal Core Infrastructure Engineer - GPU Validation

Salary: $114,600 - 234,600 per year Requirements:

• Live in or be willing to relocate to regularly work from our Nashville, Tennessee office.

• Design and build scalable validation frameworks for GPU racks and high-performance computing infrastructure.
• Create automated benchmarks and diagnostics for NCCL performance, GPU health, NVLink connectivity, InfiniBand, and other high-speed networks.
• Develop Slurm-based workflows for distributed validation and performance testing.
• Build in retries, timeouts, recovery mechanisms, and fault-tolerant behavior; develop functional, load, and fault-injection tests.
• Create dashboards, telemetry, and alerts to track infrastructure health and validation outcomes.
• Develop Linux diagnostic, log-analysis, hardware fault-isolation, FDT, and cable-validation tools; turn incident findings into reusable diagnostics and preventive automation.
• Write recovery runbooks and contribute to incident response and root-cause investigations.
• Develop and maintain Bash, Python, and Infrastructure as Code tools for deployment and configuration.
• Implement access controls and securely handle infrastructure credentials and diagnostic data.
• Automate patching, updates, and rollback workflows in line with change-management practices; maintain technical documentation and expand automation coverage.
• Translate project requirements into development plans and deliverables, manage work to agreed timelines and quality standards, and adapt to changing priorities and dependencies.
• Collaborate with HPC, infrastructure, and engineering teams to define requirements and align on outcomes.
• Investigate issues using logs, metrics, and diagnostic results; resolve problems, escalate when needed, and document findings.
• Develop expertise in GPU systems, HPC technologies, and software development; recommend and implement improvements to validation, reliability, and efficiency.
• Share reusable tools and best practices, and incorporate team feedback into ongoing improvements.

• Bash
• Hardware
• InfiniBand
• Linux
• Oracle
• Python
• DevOps
• Support
• Security

More:

We are partnering with hackajob to connect exceptional professionals with Oracle. In this HPC role, you will help ensure GPU racks are production-ready by building automated tools and validation frameworks, while contributing to scalable, resilient, and operable distributed systems. The position is based around regular work at our Nashville, Tennessee office. The U.S. hiring range is $114,600–$234,600 annually, with potential eligibility for bonus, equity, and deferred compensation. Our U.S. benefits include medical, dental, and vision coverage; disability and life insurance; flexible spending accounts; commuter benefits; a 401(k) with company match; paid time off, holidays, and sick leave; parental leave and adoption assistance; an employee stock purchase plan; financial planning and legal services; and voluntary insurance options. Compensation and benefits depend on location and individual circumstances.

last updated 42 week of 2026