Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

HPC/ML Infrastructure Engineer

Spellbrush

Job Description

Job Description

We’re looking for an experienced HPC infrastructure engineer to lead bringup, administration, and operations on is probably the largest anime AI training cluster in the world . You’ll serve as the bridge between our researchers and the bare GPU machines, helping to make sure that SLURM jobs are running, parallel filesystems are serving, network is transmitting, and that the anime models are training.

You may be a good fit if:You love anime and the anime aesthetic.

This probably one of the only jobs in the world where you will get to combine your love of anime and large-scale GPU systems.

You’re familiar with the modern HPC software landscape

Once upon a time, our team could install SLURM on a few bare metal nodes and get away with it. Now the landscape has become unbelievable complex, with SLURM deploys through Slinky on K8s, provisioning through warewulf/MAAS/ansible, filesystems through WEKA/VAST/Ceph, VPN and access through tailscale, and monitoring via the Grafana/Prometheus stack. We’re looking for someone with relevant experience up and down the stack (and maybe a papercut or two to show for it!)

As well as the traditional sysadmin landscape

Bringing up and managing cluster still requires good old linux sysadmin skills, including wrangling ldap, triaging dmesg, and setting sticky bits on directories for misbehaving users and tools.

You're not afraid of physical computers

We’re building out edge datacenters and our CEO is still personally racking, stacking, and provisioning HGX-based nodes in our living room. Also his VLAN design sucks and he’s bad at fiber routing. Please send help.

And you're comfortable working on small, fast-paced teams.

We currently have a very tiny research team, and you’ll be directly helping some of the AI researchers in the world train the best anime image model in the world.

We also believe in the unmatched speed of in-person teams, and prefer on-site collaboration in either our primary research office in Tokyo (downtown Akihabara), or San Francisco (dogpatch!). Bay area is strongly preferred as we have physical hardware in the Bay Area. Visa sponsorships are available.

Vacancy posted 7 days ago
Similar jobs that could be interesting for youBased on the HPC/ML Infrastructure Engineer in San Francisco, CA vacancy
  •  ...Job Description Job Description We’re looking for an experienced HPC infrastructure engineer to lead bringup, administration, and operations on is probably the largest anime AI training cluster in the world . You’ll serve as the bridge between our researchers and... 
    Suggested
    Work at office
    Visa sponsorship

    Spellbrush

    San Francisco, CA
    22 days ago
  • $190k - $210k

     ...backed by climate-tech and Silicon Valley investors. For more information, please visit  Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out... 
    Suggested
    Local area

    Gridware

    San Francisco, CA
    27 days ago
  •  ...oprecruiting.comPhone: (***) ***-****Job Title: Machine Learning Infrastructure EngineerLocation: San Francisco, CA Metro Area (100% On-Site)...  ...company is seeking a Machine Learning Infrastructure Engineer to help architect the compute, training, and execution frameworks... 
    Suggested
    Full time
    Work at office
    Flexible hours

    Objective Paradigm

    San Francisco, CA
    2 days ago
  • $350k

     ...AI Infrastructure EngineerThe mission of Thinking Machines is to build AI that extends human will...  ...it.We're hiring an AI Infrastructure Engineer to keep our post-training and reinforcement...  ...observability tooling purpose-built for ML training, not just general... 
    Suggested
    Permanent employment
    Visa sponsorship
    Work visa
    Relocation package
    Shift work

    Thinking Machines Lab

    San Francisco, CA
    1 day ago
  •  ...Job Description Job Description The AI Infrastructure team at Zensors builds the engine that powers our visual sensing platform. We provide the tools to...  ...video streams. As a Machine Learning Engineer in ML Runtime & Optimization , you will develop technologies... 
    Suggested

    Zensors

    San Francisco, CA
    23 days ago
  • $200k - $400k

     ...Job Description Job Description Machine Learning Infrastructure Engineer San Francisco, CA · On-site (5 days/week) · Full-time Compensation...  ...be doing Design, deploy, and maintain large distributed ML training and inference clusters Build efficient, scalable... 
    Full time
    Visa sponsorship
    Relocation package

    David Joseph & Company

    San Francisco, CA
    13 days ago
  • $166k - $244k

    A leading technology company is seeking a Research Software Engineer to develop next-generation technologies that enhance communication through advanced software and hardware. This role involves collaborating with researchers, optimizing machine learning performance, and... 

    Google

    San Francisco, CA
    5 days ago
  • $148.7k - $201.2k

     ...multi-disciplinary team of scientists, engineers, and technicians, on a mission to develop...  ...quantum computer.We are looking to hire an HPC Platform Engineer to develop, automate,...  ...high-performance computing (HPC) infrastructure on AWS that CQC scientists and engineers... 
    Local area
    Flexible hours

    Amazon

    San Francisco, CA
    4 days ago
  • A decentralized AI platform company in the United States is seeking an experienced ML Training Platform Engineer to design and build robust infrastructure for ML training. The ideal candidate has over 5 years in infrastructure and platform engineering, with expertise in... 

    Pluralis Research

    San Francisco, CA
    5 days ago
  •  ...Francisco is seeking a Machine Learning Engineer to develop tools for interpretable AI systems. The role involves optimizing ML pipelines, implementing research into production...  ...have over 5 years of experience in ML infrastructure, with expertise in Python and distributed... 

    Goodfire

    San Francisco, CA
    1 day ago
  • $275k

     ...generation of managed GPU compute and inference infrastructure. Backed by leading venture investors,...  .... This company is seeking a Founding Engineer to take ownership of core GPU cloud...  ...experience operating large-scale AI or HPC infrastructure in production ~ Proven... 
    Full time
    Relocation
    San Francisco, CA
    a month ago
  • $292k - $417.2k

     ...ads optimization that shape the future of streaming.We are seeking a Director of Machine Learning Engineering and Infrastructure to lead a hybrid team bridging advanced ML engineering with world-class infrastructure design. In this role, you will own the strategic direction... 
    Full time
    Temporary work
    Local area
    Flexible hours

    Tubi TV

    San Francisco, CA
    1 day ago
  • $230k

     ...over unchecked growth.About the roleAs a software engineer on the Fleet High Performance Computing (HPC) team, you will be responsible for the reliability...  ...maintain the health and efficiency of our supercomputing infrastructure.Our team empowers strong engineers with a high... 
    Work at office
    Local area
    Flexible hours

    OpenAI

    San Francisco, CA
    5 days ago
  • $157k - $234k

     ...machine learning models. - Collaborate with data scientists and engineers to understand model requirements and optimize pipeline...  ...functional teams.   Bonus/nice to have:  - Experience with infrastructure-as-code (IaC) tools such as Terraform or Crossplane. -... 
    Full time
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    7 days ago
  •  ...still be proud of in ten years from now. Machine Learning Engineer, Platform About the Role So much of the work society depends...  ...a Machine Learning Engineer on Platform, you'll build the core ML capabilities every product we ship stands on — built once,... 
    Full time
    Shift work

    Brain Corp

    San Francisco, CA
    9 hours ago
  • Intuitive is seeking a Senior Software Engineer - Simulation & ML Platform to build scalable software that supports simulation-based data generation and ML development for robotic motion AI. You will collaborate with simulation data scientists and ML teams to translate... 

    Socket.dev

    San Francisco, CA
    3 days ago
  •  ...Job Title:  Machine Learning Engineer, Training Infrastructure Position Type: Full time Location: San Francisco, CA, USA Salary Range: $...  ...Job ID#: 158135 Job Description: We are looking for an ML Engineer with 3+ YOE in high-performance computing systems... 
    Full time
    Work experience placement

    Intellipro Group

    San Francisco, CA
    9 hours ago
  •  ...in the enterprise through purpose-built ML systems that learn from Rippling's proprietary...  ...at Rippling.As a Staff Machine Learning Engineer, you will own the end-to-end ML...  ...Build robust evaluation and experimentation infrastructure: offline benchmarks, A/B testing, and... 
    Work at office
    3 days per week

    Rippling

    San Francisco, CA
    4 days ago
  • $246.5k - $339k

     ...this roleAs a Staff Machine Learning Platform Engineer, you will help design, improve, and operate a scalable ML platform to accelerate model training, deployment...  ....What You Will DoDesign and operate ML infrastructure, including workspaces, clusters, jobs, and workflowsProductionize... 
    Work experience placement
    Work at office
    Local area
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    1 day ago
  • $295k - $405.5k

     ...roleAs the Senior Staff Machine Learning Platform Engineer, you will own the technical vision and evolution of Faire’s ML platform. You will set standards, influence...  ...background in distributed systems, ML infrastructure, and cloud architecture.Demonstrated technical... 
    Work experience placement
    Work at office
    Local area
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    4 days ago
  • $212k - $318k

     ...creator economy and are looking for a Senior Machine Learning Engineer, Infrastructure to support our mission.This role is based in San Francisco...  ...TeamYou'll join the Relevance team, whose mission is to build the ML systems that power how fans discover creators and how... 
    Work at office
    Local area
    Remote work
    Worldwide
    Flexible hours
    2 days per week
    3 days per week

    Patreon

    San Francisco, CA
    2 days ago
  •  ...Sciforium is an AI infrastructure company developing next-generation multimodal AI models and...  ...from AMD with hands-on support from AMD engineers the team is scaling rapidly to build the...  ...applications. About the role As an ML Engineer at Sciforium, you will operate... 
    Full time
    Flexible hours

    Sciforium

    San Francisco, CA
    9 hours ago
  •  ...leave a clean trail of tests and dashboards behind. Required Qualifications: Expert-level PyTorch. Proven software engineer who loves ML; comfortable writing production code across the stack. Hands-on experience training or fine-tuning large language or other... 
    Full time
    Contract work
    Flexible hours
    Shift work

    Sesame, L.l.c.

    San Francisco, CA
    9 hours ago
  •  ...Gentoro was founded by a team with deep experience in enterprise infrastructure and AI, with leadership roots at companies including Splunk,...  ...About the Role We are looking for a visionary Senior ML Engineer who will bridge the gap between high-level architecture and... 
    Full time
    Shift work

    Palm Venture Studios

    San Francisco, CA
    9 hours ago
  •  ...their patients. Position Overview We are hiring two ML Engineers / Researchers to help build the next generation of Knowtex's...  ...clinical outcomes Build datasets, benchmarks, and evaluation infrastructure that make model improvements measurable and reproducible... 
    Full time

    Knowtex

    San Francisco, CA
    9 hours ago
  •  ...creating a genuinely open, collaborative path to frontier‑scale AI. We’re looking for an ML Training Platform Engineer to architect, build, and scale the foundational infrastructure powering our decentralised ML training platform. You will own core systems spanning... 
    Work experience placement

    Pluralis Research

    San Francisco, CA
    5 days ago
  • $128.7k - $261.3k

     ...approaches to model export, kernel development, and performance engineering so that every cycle on our accelerators translates into better...  ...tooling that makes that path fast, reliable, and effortless for ML engineers across the AV organization to compile their models.... 
    Full time
    Local area
    Remote work
    Work from home
    Relocation package
    Flexible hours

    General Motors

    San Francisco, CA
    4 days ago
  • $170.1k - $258.3k

     ...export, kernel development, and performance engineering so that every cycle on our accelerators...  ...that sit at the heart of our on‑vehicle ML inference for ADAS and autonomous...  ...workloads.Build and improve tooling and infrastructure that make it easier to profile, debug, and... 
    Full time
    Local area
    Remote work
    Work from home
    Relocation package
    Flexible hours

    General Motors

    San Francisco, CA
    2 days ago
  •  ...000s of developers and enterprise users.ML performance, quality, and systems acumen...  ...Qualifications 7+ years of software engineering experience building and operating enterprise...  ...building the next layer of enterprise AI infrastructure.Job SummaryCategory: Engineering
    Work at office

    Atlassian

    San Francisco, CA
    4 days ago
  • $148.5k - $223.9k

     ...the right place! Agentforce is the future of AI, and you are the future of Salesforce.This role is for a Senior Machine Learning Engineer within the Trust Intelligence Platform team who will architect data-driven strategies for threat detection across the security organization... 
    Full time

    Salesforce

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to HPC/ML Infrastructure Engineer. Be the first to apply!