Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior HPC & AMD Infrastructure Engineer

$180k - $220k
Full-time

evergrid

United States | Hybrid or Remote | Full-time

The role

You'll own the health, reliability, and performance of Evergrid's AMD-only GPU compute clusters.

You're the primary custodian of our high-density accelerator environments. The work spans hardware operations, Linux systems engineering, distributed infrastructure, and ML workloads. It covers GPU bring-up and kernel-level debugging, as well as maintaining and optimizing the ROCm-based ML stack behind production-scale AI. If you like getting maximum performance out of hardware, debugging GPUs at scale, and shipping world-class AI infrastructure, this role is for you.

What you'll own

System health and reliability (SRE)

  • Primary on-call response for outages, GPU failures, node crashes, and cluster-wide incidents.

  • Being the key point person for POC and active customers' GPU clusters.

  • Fast diagnosis and resolution that minimizes downtime and keeps SLA-level reliability.

  • Monitoring for GPU health, thermals, PCIe topology, memory errors, and cluster load.

  • Repairs, RMAs, and physical maintenance, coordinated with data center operators, hardware vendors, and on-site technicians.

Linux and network administration

  • Installing, patching, and maintaining Linux (Ubuntu, CentOS, RHEL) across large GPU node fleets.

  • Kernel tuning, consistent OS configuration, and fleet automation at scale.

  • Secure networking: VPNs, firewalls (iptables/firewalld), SSH hardening, and routing.

  • Identity and access systems (LDAP, FreeIPA, Active Directory).

  • Distributed storage (NFS, GPFS, Lustre).

AMD GPU and ML stack engineering (ROCm-first)

  • Deployment and bring-up of new GPU nodes, including BIOS configuration, NUMA tuning, and topology validation.

  • AMD GPU drivers, kernel modules, and the ROCm runtime across production fleets.

  • The AMD ML stack: ROCm, PyTorch (ROCm builds), JAX (ROCm/XLA), RCCL, hipBLAS/hipDNN, MIOpen, and supporting runtimes.

  • Debugging complex failures across GPUs, compilers, ML frameworks, and distributed training and inference. Examples include RCCL hangs, HIP memory faults, ROCm kernel crashes, framework build and link issues, and vLLM build failures on ROCm.

  • Infrastructure that supports both research iteration and production reliability, built with the ML and platform teams.

What you bring

  • 5+ years in HPC, GPU cluster operations, Linux systems engineering, or similar roles.

  • A bachelor's or master's in Computer Science, Computer Engineering, Electrical Engineering, or a related field.

  • Deep hands-on experience with AMD MI-series GPUs, including driver and kernel-level debugging.

  • Strong grasp of Linux internals, kernel modules, hardware bring-up, and performance tuning.

  • Experience securing and operating production infrastructure: VPNs, firewalls, SSH, and identity systems.

  • Proficiency in Bash and Python for automation, tooling, and operations.

  • Strong familiarity with ML stacks and runtime behavior in ROCm environments (ROCm/HIP, MIOpen, RCCL, PyTorch, JAX).

  • Experience debugging high-performance networking and RDMA (InfiniBand or RoCE), including cluster-level communication failures that affect distributed training.

Bonus

  • Schedulers and orchestration (Slurm, Kubernetes).

  • Model serving and inference optimization on ROCm (vLLM, SGLang).

  • Configuration management and IaC (Ansible, SaltStack, Terraform).

  • Supporting ML research or production AI teams at a startup or high-growth company.

Benefits & Compensation

  • Compensation: $180,000 - $220,000

  • Health insurance

  • Paid time off and paid holidays

  • Home office stipend

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Senior HPC & AMD Infrastructure Engineer in New York, NY vacancy
  •  ...zlokalizowany w Polsce. Twoje zadania obejmują projektowanie i wdrażanie architektur hybrydowych i chmurowych, doradztwo w zakresie infrastruktury oraz wspieranie dużych migracji. Znajomość Linux, AWS, Kubernetes, Docker oraz HPC będzie kluczowa. #J-18808-Ljbffr Sopra Steria
    Senior

    Sopra Steria

    New York, NY
    4 days ago
  • Philip Morris International ищет инженера по инфраструктуре облака для создания и эксплуатации облачных платформ на базе AWS и HPC, поддержки контейнеризации Docker и Kubernetes, автоматизации через Terraform и Ansible, а также проектирования безопасных сетевых решений.... 
    Senior

    Philip Morris International

    New York, NY
    5 days ago
  • $200k - $300k

     ...River Trading’s High Performance Computing (HPC) Network Engineering team designs and engineers the low-latency communications infrastructure that underpins our incredibly large GPU...  ...documentation Mentor others as a senior resource Profile 10+ years of relevant experience... 
    Suggested
    Work at office
    Local area
    Immediate start
    Worldwide

    Hudson River Trading

    New York, NY
    5 days ago
  •  ...Job Title: HPC Network Engineer | High-Performance Infrastructure Location: New York or Chicago preferred | Other US locations considered (Flexible work arrangements) Salary: Up to $800,000 TC (Depending on experience) Do you want to join a global quantitative... 
    Suggested
    Flexible hours

    Hamilton Barnes 🌳

    New York, NY
    3 days ago
  • $60k - $67k

     ...is the Kubernetes-native AI infrastructure company, enabling organizations...  ..., Mirantis empowers platform engineering teams to deliver composable,...  ...are looking for a motivated HPC (High Performance Computing)...  ...technologies. You will work alongside senior engineers to operate,... 
    Suggested
    Remote work

    Mirantis

    New York, NY
    5 days ago
  • Mirantis, Inc. in the United States seeks a motivated HPC Network Engineer to support and grow with our infrastructure team. You will work on InfiniBand, Ethernet, and...  ...RDMA, MPI workloads, and network security under senior engineers, with a clear growth path toward a... 

    Mirantis, Inc.

    New York, NY
    9 hours ago
  •  ...Head of Engineering (GPU Infrastructure / HPC) Investment Profile Greylock-backed company in New York building a cloud simulation platform for robotics, autonomy, and physical AI. The platform helps engineering teams develop and evaluate autonomous systems, generate... 
    Full time
    Relocation

    Greylock Partners

    New York, NY
    2 days ago
  •  ...Senior Infrastructure Engineer Build and modernize the secure, resilient infrastructure that keeps a firm’s people, systems, and data connected and protected. In this role, you’ll shape hybrid and cloud-first platforms, advance Microsoft technologies, automate operations... 
    Senior
    Full time

    ESP Legal

    New York, NY
    3 days ago
  • $130k - $150k

     ...Senior Critical Infrastructure Engineer | Commercial Office, NYC | $130k-$150k base Your New Company Hays have partnered with a leading FM service provider who are looking to hire a Senior Critical Infrastructure Engineer based in New York City. This position will... 
    Senior
    Full time
    For contractors
    Work at office

    Hays

    New York, NY
    3 days ago
  • $168k - $270.25k

     ...Enterprise Experience (NVEX) Solutions Engineering team is looking for a senior Computer or Software Engineer. This...  ...-X that link GPUs and AI compute infrastructure. Candidates must have a software...  ...Background with AI infrastructure and HPC networkingExperience programming... 
    Senior
    Full time
    Weekend work

    Nvidia

    New York, NY
    4 days ago
  •  ...to ingest digitized legacy records (PDFs, old databases) and surface them cleanly in the application. Own the Infrastructure: Help shape our engineering culture, best practices, deployment pipelines, and overall system architecture as we scale from prototype to enterprise... 
    Senior
    Full time

    Airbase

    New York, NY
    1 day ago
  • $182k - $242k

     ...CoreWeave combines superior infrastructure performance with deep technical...  ...the internal and customer engineering teams, offering valuable insights...  ...how well they perform. As a Senior Specialist Field Engineer for...  ...the data-center fabric and HPC interconnects (InfiniBand, RoCE... 
    Senior
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    New York, NY
    7 days ago
  • RESPONSIBILITIES: Kforce has a client in NYC that is seeking a Senior Network Engineer. Responsibilities: * Support and operate a large-...  ...Cisco SDA fabric deployments and Catalyst Center infrastructure * Administer and support Cisco ISE services, including... 
    Senior
    Hourly pay
    Contract work

    Kforce Technology Staffing

    New York, NY
    2 days ago
  • $110k - $160k

    We’re seeking a Senior Machine Learning Infrastructure Engineer to join us in our mission to transform the discovery of therapeutics and materials. As a member...  ...or restricted-egress cloudsArgo Workflows or batch/HPC schedulingRegulated environments (SOC 2, ISO 27001, GxP... 
    Senior
    Work at office
    Flexible hours

    Schrödinger

    New York, NY
    9 hours ago
  •  ...We are seeking an experienced Senior Network Engineer to join our IT Infrastructure team in a dynamic, fast-paced financial environment. This role is responsible for designing, implementing, and maintaining a secure, scalable, and high-performing network that supports... 
    Senior
    Remote work

    TBG | The Bachrach Group

    New York, NY
    a month ago
  • $80k - $100k

     ...postgraduate in Technology with Computers or Information Technology (IT) stream. Or A degree in Computer Science, IT, Systems Engineering or a related qualification. Preferred Qualification: Professional expert level certification for Fortinet FCSS, Fortinet... 
    Senior
    Flexible hours

    Genpact

    New York, NY
    12 days ago
  •  ...The Ideal candidate is one that has prior worked on network engineering designs and testing plus evaluated products within a network lab...  ...Kubernetes and modern cloud architecture Experience: ~ Senior level, experienced, can lead projects and work independently.... 
    Senior
    Full time
    Night shift

    ApTask

    New York, NY
    4 days ago
  • Blackstone Infrastructure Partners is seeking a 2023 Senior Associate in Infrastructure Private Equity. You will evaluate and execute investments, lead due diligence, and coordinate with Senior Advisors and portfolio teams to drive value across large-scale infrastructure... 
    Senior

    Financeroles

    New York, NY
    4 days ago
  •  ...Role : Senior Network Test & Certification Engineer Location: New York, NY (Hybrid; local candidates only) Duration: Contract Summary...  ...GenAI, LLMs, Agentic AI, AI-assisted Automation OS / Infrastructure: Linux, Kubernetes, CNI, Cilium, eBPF, Cloud Networking... 
    Senior
    Contract work
    Local area

    InfiCare Staffing

    New York, NY
    1 day ago
  •  ...environment spanning large-scale Linux infrastructure, GPUs, Kubernetes and purpose-built...  ...My client is looking to hire 2–3 Senior / Staff-level Security Engineers, working on a hybrid basis (three...  ...existing products. Experience with HPC, distributed systems or large-scale... 
    Senior
    Full time
    3 days per week

    Iceberg

    New York, NY
    2 days ago
  • $200k - $300k

     ...River Trading’s High Performance Computing (HPC) Network Engineering team designs and engineers the low-latency communications infrastructure that underpins our incredibly large GPU...  ...documentation Mentor others as a senior resource Profile10+ years of relevant experienceProven... 
    Work at office
    Local area
    Immediate start
    Worldwide

    Hudson River Trading

    New York, NY
    4 days ago
  • $135k - $165k

     ...As the only vertically integrated AI infrastructure company built from the ground up, we...  ..., low-cost GPU compute power. As a Senior Cloud Support Engineer, you'll play a crucial role in empowering...  ...competing escalations. HPC Knowledge: Understanding of HPC technologies... 
    Senior
    Full time
    Temporary work
    Shift work
    Weekend work
    Weekday work

    Crusoe

    New York, NY
    16 days ago
  •  ...manager with an established track record of investing across infrastructure and real assets. The firm is expanding its investment team in...  ...Position Overview The firm is seeking an Associate / Senior Associate to join its growing investment team in New York. The... 
    Senior

    Madison Hunt

    New York, NY
    22 days ago
  • $14 per hour

     ...HPC Systems Engineer | NYC | Stealth Fund One of the most low-key, high-firepower quant trading firms in the game is hiring - elite infra, elite people, and a research/trading stack that runs like a Swiss watch.... (Making nearly $14milllion per headcount) The... 
    Full time
    Remote work

    Hunter Bond

    New York, NY
    1 day ago
  • $60 per hour

     ...supporting, securing, and maintaining the BEP IT network infrastructure and systems to ensure a sustainable, efficient, effective...  ...contractors. ESSENTIAL DUTIES AND RESPONSIBILITIES The Senior Network Engineer shall be responsible for leading the entire Network Engineering... 
    Senior
    For contractors
    Work at office

    Tatitlek Corporation

    New York, NY
    4 days ago
  •  ..., efficiency, and compliance through exceptional user experience and cutting-edge engineering. The Role We are seeking an experienced New York-based Senior Backend / Infrastructure Engineer who wants to own the systems that power Zenda’s SaaS Platform. You will be... 
    Senior
    Full time
    Local area
    Relocation
    Relocation package

    ZENDA

    New York, NY
    8 days ago
  • $150k - $200k

     ...Sr. Infrastructure Automation Engineer — Terraform & Ansible Remote | Permanent | $150,000–$200,000 base salary + annual bonus and full benefits I’m recruiting for a Sr. Infrastructure Automation Engineer to join an established automation team and help shape automation... 
    Senior
    Permanent employment
    Full time
    Remote work

    Cannon Search

    New York, NY
    2 days ago
  •  ...headquartered in San Francisco with offices in New York, Seattle, Washington D.C., Raleigh, London, and Amsterdam. As an Infrastructure Engineer on Network Infrastructure, you'll own large parts of the network that every Plaid product depends on. You'll plan and lead... 
    Senior
    Full time
    Work experience placement
    Local area

    plaid

    New York, NY
    3 days ago
  •  ...Senior Vice President – Mainframe Infrastructure Engineer Overview BNY is seeking an experienced Senior Vice President, Infrastructure Engineer to join our Mainframe Services organization based in New York, NY or Jersey City, NJ. This role is responsible for the engineering... 
    Senior
    Full time
    Flexible hours

    BNY

    New York, NY
    11 days ago
  •  ...Visionaryz is looking for a senior, hands-on architect to help shape how one of the country’s largest transit organizations adopts agentic...  ...The ability to explain architecture clearly to executives and engineers alike ⭐ Nice to have: Cloud certifications (Azure, AWS)... 
    Senior

    Visionaryz Inc.

    New York, NY
    16 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior HPC & AMD Infrastructure Engineer. Be the first to apply!