Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Inference Platform Engineer - Data Center

$300k
Full-time

Join a stealth-mode hyperscale data center startup building an AI and cloud platform, powered by thousands of H100s, H200s, and B200s,  ready to go for experimentation, full-scale model training, or inference. 

Our client operates high-performance GPU clusters powering some of the most advanced AI workloads worldwide. They’re now building a serverless inference platform,  beginning with cost-efficient batch inference and expanding into low-latency, real-time inference and custom model hosting. This is a unique chance to join at an early stage and help define the architecture, scalability, and technical direction of that platform.

If you are interested in this opportunity, get in tuch!  You don't want to miss this opportunity! 

Key Responsibilities

  • Take ownership of the inference platform architecture, from batch to low-latency workloads.
  • Design, build, and optimise distributed inference systems to maximise GPU utilisation and minimise cold starts.
  • Integrate, tune, and operate inference engines such as vLLM, SGLang, and TensorRT-LLM across multiple model types.
  • Develop APIs, orchestration layers, and autoscaling logic to support both multi-tenant and dedicated deployments.
  • Collaborate with cross-functional teams to translate business and customer needs into robust technical solutions.
  • Stay up to date with the latest models, serving frameworks, and optimisation techniques, applying best practices in performance and efficiency.
  • Implement monitoring, alerting, and observability workflows for production systems.

Requirements:

  • 5+ years’ experience building large-scale, fault-tolerant distributed systems (ML inference, HPC, or similar).
  • Proficiency in Python, Go, Rust, or a comparable language.
  • Strong understanding of GPU software stacks (CUDA, Triton, NCCL) and Kubernetes orchestration.
  • Practical experience with model-serving frameworks such as vLLM, SGLang, TensorRT-LLM, or custom PyTorch deployments.
  • Knowledge of performance optimisation techniques, including batching, speculative decoding, quantisation, and caching.
  • Familiarity with Infrastructure-as-Code tools (Terraform, Helm) and low-level OS performance tuning.

Nice to Have

  • Experience with event-driven or serverless architectures.
  • Exposure to hybrid cloud or multi-cluster environments.
  • Contributions to open-source ML or inference systems projects.
  • Proven track record of cost optimisation in high-performance compute environments.

Benefits:

  • Equity

Salary:

  • $300,000 gross per year
Vacancy posted more than 2 months ago
Similar jobs that could be interesting for youBased on the Senior Inference Platform Engineer - Data Center in San Francisco, CA vacancy
  • $200k - $220k

     ...experts across energy, manufacturing, data center construction, and cloud services. If...  ...This Role: Join Crusoe Energy as a Senior Data Engineer, an early and pivotal hire on our...  ...architect and build the foundational data platform infrastructure that powers Crusoe's AI... 
    Senior
    Full time
    Temporary work
    Work at office
    Remote work

    Crusoe

    San Francisco, CA
    4 days ago
  • $245k - $295k

     ...experts across energy, manufacturing, data center construction, and cloud services...  ....About the RoleWe are seeking a Senior Manager, Infrastructure Platform Engineering to lead a team building core...  ...GPU clusters, AI training, and inference workloadsWorking knowledge of... 
    Senior
    Temporary work
    Immediate start

    Crusoe

    San Francisco, CA
    a month ago
  •  ...they need through the platforms they already sell on....  ...for a seasoned software engineer to join Parafin’s...  ...of our next-generation Data Platform. This role is...  ...small businesses.As a Senior Software Engineer, you...  ...retraining), real-time inference.Contributions to internal... 
    Senior
    Flexible hours

    Parafin

    San Francisco, CA
    a month ago
  • $240k - $360k

     ...the RoleWe're looking for a Senior Staff Data Engineer to be the technical backbone of our Data & ML Platform team — the foundation powering...  ...supports model training and inference workflows reliably.Raise the...  .... With an AI-powered human-centered care model, Hinge Health... 
    Senior
    Work at office
    Local area
    Immediate start
    Remote work
    Worldwide
    3 days per week

    Hinge Health

    San Francisco, CA
    a month ago
  • $170k - $205k

     ...across energy, manufacturing, data center construction, and cloud...  ...ambitious and experienced Senior Software Engineer to join their team. You'll...  ...teams to define the AI platform roadmap. Influence the...  ...infrastructure (training, inference, ETL pipelines). ~ Software... 
    Senior
    Temporary work
    Work at office

    Crusoe

    San Francisco, CA
    3 days ago
  • $193.4k - $290k

     ...frontier agentic AI, an enterprise-grade platform, and deep domain expertise, we're reshaping...  ...Harvey is generating far more data than we currently know how to use well. Product...  ...frameworks, tooling, and paved paths that product engineers, data engineers, and analysts all build... 
    Senior
    Night shift

    Harvey

    San Francisco, CA
    22 hours ago
  • $168k - $280k

     ...Rippling.com addresses. About the team Rippling's Data Platform team builds and operates the infrastructure, tools, and...  ...cost-effectively. About the role We are hiring a Senior Software Engineer to work across core data and ML infrastructure, workflow... 
    Senior
    Work at office
    3 days per week

    Rippling

    San Francisco, CA
    3 days ago
  • $220.5k - $269.5k

     ...millions of daily active users who use the platform for many different reasons, but there’s...  ...Go Live, every Quest completed, there's data, petabytes of it, telling the story of how...  ...lovable products for Discord users and Discord engineers. We’re building the next generation Data... 
    Senior
    Full time
    Work at office
    Worldwide
    Relocation
    Relocation package
    2 days per week
    1 day per week

    Discord

    San Francisco, CA
    22 hours ago
  • $148k - $185k

     ...and move rapidly to test new ideas and products.As a Data Scientist expert in causal inference and marketing mix models (MMM), you will lead our efforts...  ...Expertise in SQL and experience with large-scale data platforms.Proficiency in Python and working within production... 
    Senior
    Hourly pay
    Work at office
    Local area
    3 days per week

    Lyft

    San Francisco, CA
    a month ago
  •  ...media not just possible, but practical: a unified platform where high-performance inference, orchestration, and observability come together to...  ...ambitious teams build on. About this role: As a Senior Data Engineer at fal, you will build the data infrastructure that... 
    Senior
    Currently hiring
    Relocation package

    features and labels

    San Francisco, CA
    3 days ago
  • A leading compensation platform company in San Francisco is seeking experienced backend engineers to build and maintain robust, data-rich systems for compensation processes. Ideal candidates have strong product intuition and hands-on experience with technologies like Node... 
    Senior
    Flexible hours

    Jobleads-US

    San Francisco, CA
    4 days ago
  • $215k - $260k

     ...Engineering Manager For The Data Team Crusoe is on a mission to accelerate the abundance of energy and...  ...across energy, manufacturing, data center construction, and cloud services....  ...engineers, the reliability of our data platform, and your ability to execute a roadmap... 
    Temporary work

    Crusoe

    San Francisco, CA
    1 day ago
  •  ...Senior Software Engineer, Developer Platform Houston, TX or San Francisco Bay Area Company Introduction At...  ...simulation and machine learning training to data pipelines and CI/CD, our teams...  ...cloud (AWS) and on-prem data center environments, handling scheduling,... 
    Senior
    Temporary work

    Bot Auto

    San Francisco, CA
    4 days ago
  • $120k - $148k

    Joining ArupWe are seeking a Senior Electrical Engineer to join one of either our Los Angeles or San Francisco office with a focus on data center project delivery. This role involves leading electrical engineering design, collaborating with multidisciplinary teams, and... 
    Senior
    Work at office
    Remote work
    Flexible hours

    Arup

    San Francisco, CA
    3 days ago
  • $101k - $175k

     ...delivering sustainable design, engineering, and consultancy solutions...  ...re looking for an experienced Senior Fire Protection Engineer with a strong background in data center and Industrial environments to...  ...tools such as Revit, BIM platforms, AutoCAD, Navisworks, and fire... 
    Senior
    Full time
    Part time

    Arcadis

    San Francisco, CA
    4 days ago
  • $179k - $218k

     ...experts across energy, manufacturing, data center construction, and cloud services....  ...be bridged. We are seeking a Senior Staff Data Center Operations Engineer, GPU Hardware Architecture to be...  ...technical authority on GPU platforms within the Data Center Engineering... 
    Senior
    Temporary work

    Crusoe

    San Francisco, CA
    a month ago
  • $148k - $260k

     ...learn more visit: As a Senior/Staff Software Engineer embedded within our Autonomy...  ...build the scalable ML data pipelines necessary to train...  ...Waabi’s autonomous driving platform. Working closely with...  ...and/or highly optimized ML inference pipelines.   - Experience... 
    Senior
    Full time
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    24 days ago
  • $295k - $405.5k

     ...a technology wholesale platform built on the belief that...  ...using the power of tech, data, and machine learning to...  ...Wisconsin.About this roleAs the Senior Staff Machine Learning Platform Engineer, you will own the...  ...including training, inference, feature management, governanceEstablish... 
    Senior
    Work experience placement
    Work at office
    Local area
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    14 days ago
  • $160k - $230k

     ...Acceleration Cloud, an end-to-end platform for the full generative AI...  ..., combining the fastest LLM inference engine with state-of-the-art AI cloud infrastructure. As a Senior Backend Engineer, you will...  ..., spanning dozens of data centers across the world. Some of... 
    Senior
    Full time
    Remote work

    Together Ai

    San Francisco, CA
    22 hours ago
  • $152k - $221k

     ...Specialists, Product, and Engineering teams to package...  ..., C or C++) including data structures, algorithms...  ...such as Google Cloud Platform (GCP).Ability to travel...  ...running AI/ML training and inference workloads on GPU/TPU...  ..., web-scale data centers, and software infrastructure... 

    Google

    San Francisco, CA
    7 days ago
  • $250k - $300k

     ...across energy, manufacturing, data center construction, and cloud...  ...seeking Sr. Staff Software Engineers to serve as the Managed Platform Services (MAPS) organization’s most senior floating technical leaders....  ...customer profiles (training and inference) to identify what features... 
    Senior
    Permanent employment
    Temporary work
    Shift work

    Crusoe

    San Francisco, CA
    a month ago
  • $41.5k - $133.3k

     ...Data Center Sales Engineer HPC-Industrial, powered by Clean Harbors, is looking for a Data Center Sales Engineer to join their safety conscious...  ...-pronged safety system, innovative career development platform, committed customer service, deep industry expertise, and... 
    Full time
    Temporary work
    For contractors
    Local area
    Remote work
    Work from home

    Clean Harbors

    San Francisco, CA
    15 hours ago
  •  ..., Raleigh, London, and Amsterdam.Making data-driven decisions is key to Plaid's culture...  ...datasets and tooling to teams across engineering, product, and business and help them explore...  ...DE team and from the broader Data Platform team. You will collaborate with and have... 
    Senior
    Full time
    Work experience placement
    Work at office
    Local area
    Flexible hours

    Plaid Financial

    San Francisco, CA
    14 days ago
  • $300 per month

     ...experts across energy, manufacturing, data center construction, and cloud services....  ...About the Role: We are seeking Senior Software Engineers to design and develop internal datacenter...  ...management processes and enhancing our cloud platform’s overall performance as we... 
    Senior
    Full time
    Temporary work

    Crusoe

    San Francisco, CA
    22 hours ago
  • $160k - $260k

     ...AirwallexAirwallex is the only unified payments and financial platform for global businesses. Powered by our unique combination of...  ...banking. If that sounds like you, let’s build what’s next.Senior Data Engineer Hiring Location: San FranciscoPart 1. Data ModelingDesign and... 
    Senior
    Temporary work
    Work at office
    Local area
    Worldwide

    Airwallex

    San Francisco, CA
    a month ago
  • $182k - $242k

     ...pioneers, CoreWeave delivers a platform of technology, tools, and...  ...at  What You'll Do: The Data Engineering Team builds and operates the...  ...the role: We're seeking a Senior Data Engineer to design and...  ...day in our office and data center locations ~ A casual work... 
    Senior
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    San Francisco, CA
    3 days ago
  •  ...Lambda Inc. in San Francisco is seeking a Storage Engineer to own the reliability, performance, and capacity of our production storage fleet across multiple data centers, using a software-defined data plane. You will build monitoring, dashboards, and alerting for storage... 
    Senior

    Lambda

    San Francisco, CA
    4 days ago
  • $168k - $210k

     ...establish trust at the very first interaction! As a Senior Software Engineer, you will build the platform where every customer journey begins. You are building...  ...scalable orchestration layers for customer data. Migrate core business logic out of Salesforce into... 
    Senior
    Full time
    Work at office
    Local area

    Ripple Labs Inc.

    San Francisco, CA
    4 days ago
  • A leading AI research organization in San Francisco seeks a skilled Full-Stack Engineer to enhance data acquisition tools and interfaces. You will develop robust applications in collaboration with cross-functional teams to streamline data workflows. The ideal candidate... 
    Senior

    Jobleads-US

    San Francisco, CA
    4 days ago
  •  ...Role Our generative AI-powered products are revolutionizing the practice of medicine, and we're looking for a highly motivated Data Engineer to join our growing US-based Data Engineering team. In this crucial role, you will build and optimize large scale data... 
    Senior
    Hourly pay
    Full time
    Flexible hours

    Abridge

    San Francisco, CA
    22 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Inference Platform Engineer - Data Center. Be the first to apply!