Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Inference Platform Engineer - Data Center

$300k
Full-time

Join a stealth-mode hyperscale data center startup building an AI and cloud platform, powered by thousands of H100s, H200s, and B200s,  ready to go for experimentation, full-scale model training, or inference. 

Our client operates high-performance GPU clusters powering some of the most advanced AI workloads worldwide. They’re now building a serverless inference platform,  beginning with cost-efficient batch inference and expanding into low-latency, real-time inference and custom model hosting. This is a unique chance to join at an early stage and help define the architecture, scalability, and technical direction of that platform.

If you are interested in this opportunity, get in tuch!  You don't want to miss this opportunity! 

Key Responsibilities

  • Take ownership of the inference platform architecture, from batch to low-latency workloads.
  • Design, build, and optimise distributed inference systems to maximise GPU utilisation and minimise cold starts.
  • Integrate, tune, and operate inference engines such as vLLM, SGLang, and TensorRT-LLM across multiple model types.
  • Develop APIs, orchestration layers, and autoscaling logic to support both multi-tenant and dedicated deployments.
  • Collaborate with cross-functional teams to translate business and customer needs into robust technical solutions.
  • Stay up to date with the latest models, serving frameworks, and optimisation techniques, applying best practices in performance and efficiency.
  • Implement monitoring, alerting, and observability workflows for production systems.

Requirements:

  • 5+ years’ experience building large-scale, fault-tolerant distributed systems (ML inference, HPC, or similar).
  • Proficiency in Python, Go, Rust, or a comparable language.
  • Strong understanding of GPU software stacks (CUDA, Triton, NCCL) and Kubernetes orchestration.
  • Practical experience with model-serving frameworks such as vLLM, SGLang, TensorRT-LLM, or custom PyTorch deployments.
  • Knowledge of performance optimisation techniques, including batching, speculative decoding, quantisation, and caching.
  • Familiarity with Infrastructure-as-Code tools (Terraform, Helm) and low-level OS performance tuning.

Nice to Have

  • Experience with event-driven or serverless architectures.
  • Exposure to hybrid cloud or multi-cluster environments.
  • Contributions to open-source ML or inference systems projects.
  • Proven track record of cost optimisation in high-performance compute environments.

Benefits:

  • Equity

Salary:

  • $300,000 gross per year
Vacancy posted more than 2 months ago
Similar jobs that could be interesting for youBased on the Senior Inference Platform Engineer - Data Center in San Francisco, CA vacancy
  • $170k - $205k

     ...experts across energy, manufacturing, data center construction, and cloud services....  ...About This Role: Join Crusoe Energy as a Senior Data Engineer, an early and pivotal hire on our growing...  ...and build the foundational data platform infrastructure that powers Crusoe's AI... 
    Senior
    Full time
    Temporary work
    Work at office
    Remote work

    Crusoe

    San Francisco, CA
    11 days ago
  • $230k - $265k

     ...they need through the platforms they already sell on....  ...for a seasoned software engineer to join Parafin’s...  ...of our next-generation Data Platform. This role is...  ...small businesses. As a Senior Software Engineer, you...  ...), real-time inference. Contributions to internal... 
    Senior
    Full time
    Work from home
    Flexible hours

    Parafin

    San Francisco, CA
    1 day ago
  • $245k - $295k

     ...experts across energy, manufacturing, data center construction, and cloud services...  ....About the RoleWe are seeking a Senior Manager, Infrastructure Platform Engineering to lead a team building core...  ...GPU clusters, AI training, and inference workloadsWorking knowledge of... 
    Senior
    Temporary work
    Immediate start

    Crusoe

    San Francisco, CA
    a month ago
  • $160k - $230k

     ...Acceleration Cloud, an end-to-end platform for the full generative AI...  ..., combining the fastest LLM inference engine with state-of-the-art AI cloud infrastructure. As a Senior Backend Engineer, you will...  ..., spanning dozens of data centers across the world. Some of... 
    Senior
    Full time
    Remote work

    Together Ai

    San Francisco, CA
    1 day ago
  • $240k - $360k

     ...the RoleWe're looking for a Senior Staff Data Engineer to be the technical backbone of our Data & ML Platform team — the foundation powering...  ...supports model training and inference workflows reliably.Raise the...  .... With an AI-powered human-centered care model, Hinge Health... 
    Senior
    Work at office
    Local area
    Immediate start
    Remote work
    Worldwide
    3 days per week

    Hinge Health

    San Francisco, CA
    a month ago
  • $300 per month

     ...experts across energy, manufacturing, data center construction, and cloud services....  ...About the Role: We are seeking Senior Software Engineers to design and develop internal datacenter...  ...management processes and enhancing our cloud platform’s overall performance as we... 
    Senior
    Full time
    Temporary work

    Crusoe

    San Francisco, CA
    1 day ago
  • $192k - $240k

     ...Brex is the intelligent finance platform that enables companies to...  ...need to grow your career. Engineering at Brex Engineering at Brex...  ...intention. Our teams span Software, Data, Security, and IT, and...  ...grade backend systems. As a Senior Software Engineer, you’ll... 
    Senior
    Full time
    Work at office
    Remote work
    Work from home

    Brex Inc.

    San Francisco, CA
    1 day ago
  • $213k - $263k

     ...hail service and can also be applied to a range of vehicle platforms and product use cases. The Waymo Driver has provided over...  ...that drives the car. We are looking for experienced data-minded software engineers and data scientists to help us improve how we characterize... 
    Senior
    Full time
    Remote work

    Waymo

    San Francisco, CA
    1 day ago
  •  ...Persona is the configurable identity platform built for businesses in a digital-first...  ...role We are looking for passionate engineers who are excited to build the data foundation for the identity layer of the internet. As a senior engineer on the Data Platform team at... 
    Senior
    Full time
    For contractors
    Internship
    Immediate start

    Persona

    San Francisco, CA
    1 day ago
  • $170k - $230k

     ...company that sets the standard in modern card platform management. As an all-in-one card issuer...  ...it happen. Job Description The Data Platform Team is part of the Data...  ...push the boundaries of productivity and engineering outcomes. Bonus skills Experience... 
    Senior
    Full time
    Work at office
    Local area
    Home office
    Flexible hours

    Highnote

    San Francisco, CA
    1 day ago
  • $170k - $205k

     ...across energy, manufacturing, data center construction, and cloud...  ...ambitious and experienced Senior Software Engineer to join their team. You'll...  ...teams to define the AI platform roadmap. Influence the...  ...infrastructure (training, inference, ETL pipelines). ~ Software... 
    Senior
    Temporary work
    Work at office

    Crusoe

    San Francisco, CA
    4 days ago
  • $193.4k - $290k

     ...frontier agentic AI, an enterprise-grade platform, and deep domain expertise, we're reshaping...  ...Harvey is generating far more data than we currently know how to use well. Product...  ...frameworks, tooling, and paved paths that product engineers, data engineers, and analysts all build... 
    Senior
    Night shift

    Harvey

    San Francisco, CA
    5 days ago
  • $148k - $185k

     ...and move rapidly to test new ideas and products.As a Data Scientist expert in causal inference and marketing mix models (MMM), you will lead our efforts...  ...Expertise in SQL and experience with large-scale data platforms.Proficiency in Python and working within production... 
    Senior
    Hourly pay
    Work at office
    Local area
    3 days per week

    Lyft

    San Francisco, CA
    a month ago
  •  ...media not just possible, but practical: a unified platform where high-performance inference, orchestration, and observability come together to...  ...ambitious teams build on. About this role: As a Senior Data Engineer at fal, you will build the data infrastructure that... 
    Senior
    Currently hiring
    Relocation package

    features and labels

    San Francisco, CA
    4 days ago
  • $182k - $242k

     ...pioneers, CoreWeave delivers a platform of technology, tools, and...  ...more at What You’ll Do The Data Engineering Team builds and operates the...  ...About The Role We’re seeking a Senior Data Engineer to design and...  ...day in our office and data center locations A casual work... 
    Senior
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    San Francisco, CA
    3 days ago
  • A leading compensation platform company in San Francisco is seeking experienced backend engineers to build and maintain robust, data-rich systems for compensation processes. Ideal candidates have strong product intuition and hands-on experience with technologies like Node... 
    Senior
    Flexible hours

    Jobleads-US

    San Francisco, CA
    5 days ago
  • $190k - $240k

    Senior Data Platform Engineer San Francisco, California, United States Tatari is on a mission to revolutionize TV advertising. Founded in 2016 to help transform the antiquated world of TV advertising through the intelligent application of AI and machine learning, Tatari... 
    Senior
    Work at office
    2 days per week

    Tatari

    San Francisco, CA
    2 days ago
  • $120k - $148k

    Joining ArupWe are seeking a Senior Electrical Engineer to join one of either our Los Angeles or San Francisco office with a focus on data center project delivery. This role involves leading electrical engineering design, collaborating with multidisciplinary teams, and... 
    Senior
    Work at office
    Remote work
    Flexible hours

    Arup

    San Francisco, CA
    4 days ago
  • $179k - $218k

     ...experts across energy, manufacturing, data center construction, and cloud services....  ...be bridged. We are seeking a Senior Staff Data Center Operations Engineer, GPU Hardware Architecture to be...  ...technical authority on GPU platforms within the Data Center Engineering... 
    Senior
    Temporary work

    Crusoe

    San Francisco, CA
    a month ago
  • $215k - $260k

     ...experts across energy, manufacturing, data center construction, and cloud services.If you...  ...with us at Crusoe.About the Role:As the Engineering Manager for the Data Team, you will lead...  ...engineers, the reliability of our data platform, and your ability to execute a roadmap... 
    Temporary work
    Work at office

    Crusoe

    San Francisco, CA
    a month ago
  • $148k - $260k

     ...learn more visit: As a Senior/Staff Software Engineer embedded within our Autonomy...  ...build the scalable ML data pipelines necessary to train...  ...Waabi’s autonomous driving platform. Working closely with...  ...and/or highly optimized ML inference pipelines.   - Experience... 
    Senior
    Full time
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    25 days ago
  • $120.4k - $158.03k

     ...effective and transformative projects to the electric vehicle, data center, consumer goods and products, metals, pulp and paper and...  ...and every location. We\'re looking for a high-energy, Senior HVAC Engineer to focus on our Data Center market. Join us, and you\'ll impact... 
    Senior
    Full time
    Remote work

    Jacobs

    San Francisco, CA
    3 days ago
  • $101k - $175k

     ...delivering sustainable design, engineering, and consultancy solutions...  ...re looking for an experienced Senior Fire Protection Engineer with a strong background in data center and Industrial environments to...  ...tools such as Revit, BIM platforms, AutoCAD, Navisworks, and fire... 
    Senior
    Full time
    Part time

    Arcadis

    San Francisco, CA
    5 days ago
  • $295k - $405.5k

     ...a technology wholesale platform built on the belief that...  ...using the power of tech, data, and machine learning to...  ...Wisconsin.About this roleAs the Senior Staff Machine Learning Platform Engineer, you will own the...  ...including training, inference, feature management, governanceEstablish... 
    Senior
    Work experience placement
    Work at office
    Local area
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    15 days ago
  • $152k - $221k

     ...Specialists, Product, and Engineering teams to package...  ..., C or C++) including data structures, algorithms...  ...such as Google Cloud Platform (GCP).Ability to travel...  ...running AI/ML training and inference workloads on GPU/TPU...  ..., web-scale data centers, and software infrastructure... 

    Google

    San Francisco, CA
    8 days ago
  • $250k - $300k

     ...across energy, manufacturing, data center construction, and cloud...  ...seeking Sr. Staff Software Engineers to serve as the Managed Platform Services (MAPS) organization’s most senior floating technical leaders....  ...customer profiles (training and inference) to identify what features... 
    Senior
    Permanent employment
    Temporary work
    Shift work

    Crusoe

    San Francisco, CA
    a month ago
  • $230k - $340k

     ...tied to it. We fuse public records, IRS data, sanctions lists, web signals, and...  ...business is real and worth trusting: gig platforms, marketplaces, AI companies, and commerce...  ...pipeline, not a research demo. We're hiring a Senior AI Engineer to own a slice of this enrichment... 
    Senior
    Full time
    Work at office
    Flexible hours

    Baselayer

    San Francisco, CA
    1 day ago
  •  ...Senior Data EngineerHiring Location: San FranciscoPart 1. Data ModelingDesign and implement robust and scalable data models that...  ...transformation and delivery.Able to work closely with Data Platform Engineers (DPEs) and Product Managers (PMs) to quickly identify root... 
    Senior
    Work at office

    Airwallex

    San Francisco, CA
    5 days ago
  •  ...About Persona Persona is the configurable identity platform built for businesses in a digital-first world. Verifying...  ...us. About the Role We are looking for a Senior Software Engineer to join our Online Data Stores team, a critical part of the Infrastructure team... 
    Senior
    Remote job
    Full time
    For contractors
    Internship

    Persona

    San Francisco, CA
    1 day ago
  • $164k

     ...About the Role The Data engineering team has been created keeping in mind the importance and criticality of data for the overall success...  ..., and free. Through our user-friendly tools and intuitive platforms, we empower our members to take control of their finances... 
    Senior
    Full time
    Work at office
    Local area
    Remote work
    Night shift

    Chime Financial

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Inference Platform Engineer - Data Center. Be the first to apply!