Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Principal AI Product Engineer

$290k

Nscale

 

 

About Nscale

Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.

About the Role

Nscale is looking for a  Principal AI Engineer (Specialised) to lead the inference and post-training pillar of our AI systems engineering organization. You’ll define the multi-year technical roadmap for how models are served, evaluated, and post-trained on Nscale’s GPU cloud, across dedicated and serverless inference and bring-your-own-model deployments. You’ll lead the most consequential architectural programmes in that space and set the engineering standards that 20–50+ engineers build to.

As a Principal engineer, you are one of the deepest technical authorities in the company on AI systems. Your decisions set the cost, latency, and reliability at which Nscale serves tokens and runs post-training workloads, and those numbers have to compete with the world’s leading AI infrastructure providers. The problems span the full stack: kernel efficiency on state-of-the-art GPU systems, fleet-level KV cache and serving architecture, the evals that prove model quality, and RL loops where inference and training share hardware. You frame the solutions the organization executes against, including the API contracts customers see.

How We Work

  • Dog years. We move quickly and compress a lot of learning into a short time.
  • Don’t let perfect be the enemy of good. Ship, measure, iterate.
  • Be relentless. Own the problem end to end and see it through.
  • One team, one mission. Outcomes over process, and no “not my job”.

Responsibilities

  • Define and own the multi-year technical roadmap for Nscale’s inference, evals, and post-training platform, and translate it into architecture that multiple teams can execute against
  • Lead company-scale architectural initiatives in the pillar, such as next-generation serving (disaggregated prefill/decode, KV cache orchestration across GPU, host, and storage tiers, speculative decoding, multi-tenant scheduling), GPU kernel and model efficiency work (custom kernels, FP8/NVFP4/INT8/4 quantization, sparsity, distillation, MoE serving), evals and benchmarking frameworks, and post-training and RL infrastructure
  • Establish engineering standards adopted across all AI teams: API design and compatibility guarantees, benchmarking and evals methodology, training stability norms, and performance testing practices
  • Own the framework by which cost, latency, throughput, and model quality trade-offs are made and measured across the pillar
  • Identify long-horizon systemic risks early (serving engine and framework bets, accelerator support, capability gaps) and resolve them before they block the organization
  • Align AI engineering, research, product, and infrastructure leadership on multi-team technical strategy; frame technical trade-offs in product and commercial terms
  • Mentor and develop Staff and Senior AI Engineers, and grow the next generation of inference technical leaders at Nscale
  • Represent Nscale’s technical approach externally: open-source leadership in the frameworks we depend on, publications, conference talks, and partnerships with GPU vendors and AI labs

Requirements

  • 10–15 years of engineering experience, with a clear track record of pillar-level impact on production AI systems
  • 4+ years of hands-on work with LLMs in inference, GPU performance, evals, or post-training and RL, in production or research
  • Demonstrated ability to define multi-year technical strategy for complex, multi-team AI systems organizations
  • World-class depth in production LLM inference, GPU performance, evals, and/or post-training and RL infrastructure, with strong working knowledge across the rest
  • Demonstrated ownership of the architecture of a large-scale production inference or training platform
  • Proven ability to create architectural frameworks and engineering standards adopted across large engineering organizations
  • Deep understanding of the hardware/software boundary for AI accelerators: CUDA or ROCm, memory bandwidth and interconnect constraints, and distributed compute paradigms
  • Strong history of growing technical leaders (Staff and above) and multiplying technical capability across teams
  • External recognition in the AI systems community through research, open source, or industry contribution

Preferred

  • Prior experience at a top-tier AI lab or major hyperscaler AI infrastructure team
  • Maintainer or core contributor to a foundational inference, kernel, or RL framework (vLLM, SGLang, TensorRT-LLM, LMCache, FlashInfer, Triton, verl, OpenRLHF, TRL, DeepSpeed, Megatron-LM, etc.)
  • Hands-on depth in RL for LLMs (DPO/GRPO-style methods, reward modelling, multi-turn and tool-use RL) and the interaction between inference and training infrastructure
  • Experience defining developer API platforms adopted at scale by external developers
  • Deep experience with control plane / data plane architecture and cell-based deployment patterns in large-scale inference infrastructure
  • Published work in AI systems: MLSys, NeurIPS Systems Track, OSDI, EuroSys, SC, or equivalent
  • Experience with hardware-software co-design: custom accelerator kernels (CUDA, Triton), compiler-level optimization, AI hardware roadmap engagement
  • Experience defining pricing, SLO, and capacity models for a commercial inference product

 

The range below reflects the base salary for the position. Actual compensation may vary based on job-related factors such as skill set, experience, education, and location. In addition to base salary, this role may be eligible for bonus, equity, and/or commission programs. Nscale may offer a competitive benefits package including medical, dental, vision, flexible paid time off, parental leave, and retirement plan participation.

Salary Range

$290,000—$520,000 USD

For information on how Nscale handles candidate personal data, please see our Employee & Candidate Privacy Notice:  Here.

Nscale does not accept unsolicited candidate submissions from recruitment agencies.

Vacancy posted 8 hours ago
Similar jobs that could be interesting for youBased on the Principal AI Product Engineer in San Francisco, CA vacancy
  •  ...About the Role As a Principal AI Engineer at Innovaccer, you will be one of our most senior engineers and you will stay deep in the code...  ...the AI/ML-powered features that increasingly define our products — while setting the technical bar for how those systems are... 
    Principal
    Temporary work

    Innovaccer Analytics

    San Francisco, CA
    4 days ago
  •  ...workforce through an intelligent, auditable AI platform that spans the entire...  ...confidential search, Scovai is seeking a Principal Generative AI Engineer to serve as the technical cornerstone...  ...role, you will architect and deliver production-grade generative AI systems —... 
    Principal
    Full time

    CONFIDENTIAL Scovai

    San Francisco, CA
    5 days ago
  • $250k - $350k

     ...t fall through the cracks. We believe AI should do more. Thorin is building proactive...  ...workflows. Own features from the initial product decision through implementation and...  ...others, and raise the quality of a small engineering pod. Turn product and customer signal... 
    Suggested
    Full time

    thorin

    San Francisco, CA
    3 days ago
  • $150k - $200k

    San Francisco, CaliforniaEngineering - AI Engineering /US Full-time Salaried /On-siteSamba is a media intelligence company. We know what...  ...selectionWHO YOU AREYou’ve built AI agents and shipped them to production—not just prototypesYou’ve deployed agents in cloud... 
    Suggested
    Full time
    Local area

    Samba TV

    San Francisco, CA
    4 days ago
  • $90k - $110k

     ...develop, and operationalize secure agentic AI capabilities in risk & security...  ...where AI agents can accelerate analyst productivity and improve outcomesContinuously validate...  ...experience leading enterprise-scale security engineering initiatives5+ years of progressive experience... 
    Principal
    Full time

    Axis Capital

    San Francisco, CA
    4 days ago
  • $215.25k - $287.91k

     ...DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive...  ...compelling business stories. You will connect products, models, agents, and data into...  ...individual-contributor role combining software engineering, applied AI, solution architecture, and storytelling... 
    Principal
    Full time

    Salesforce

    San Francisco, CA
    1 day ago
  • $157.25k - $212.75k

     ...Francisco, CAEngineering /Full Time /RemoteThe Principal Product Manager, Clinical Data Analytics, will...  .... This person will work closely with engineering, clinical informatics, and customer-...  ...’s marketplace growth. Under Tendo’s AI-First R&D model, this role operates as... 
    Principal
    Full time
    Temporary work
    Local area
    Flexible hours

    Tendo Systems

    San Francisco, CA
    1 day ago
  • $200k - $320k

     ...Job Description Job Description Senior AI Product Engineer Location: San Francisco Bay Area (Hybrid/Flexible – Occasional Office Visits) Compensation: $200,000–$320,000 Base Salary + Competitive Equity Employment: Full-Time Industry: Artificial Intelligence... 
    Full time
    Work at office
    Remote work
    Flexible hours

    Kastel Staffing

    San Francisco, CA
    4 days ago
  • $127.2k - $201.3k

     ...as we are, join our team.KPMG is currently seeking a Manager, AI Product Security Leader to join our Enterprise Security Services...  ...secure model- and tool-interaction patternsLead complex security engineering initiatives end-to-end with minimal oversight, from risk and... 
    H1b
    Local area

    KPMG

    San Francisco, CA
    2 days ago
  • $200k - $250k

     ...About Us At You.com, we are building the AI Search Infrastructure that powers modern AI systems. Our goal is to create...  ...reliable, transparent, and useful. Our team includes engineers, researchers, product builders, and operators who care about solving meaningful... 
    Remote job
    Immediate start
    Work from home
    Flexible hours
    Early shift

    You.com

    San Francisco, CA
    8 hours ago
  • $140k - $180k

     ...Latch builds rigorous scientific benchmarks ( benchmarks.bio ) for AI agents across biology. We work with frontier labs and pharma to...  ...We're seeking an AI Biologist to join our Protein Design & Engineering team, working at the frontier of what artificial intelligence can... 
    Full time
    For contractors
    H1b
    Work at office
    Remote work
    Night shift

    LatchBio

    San Francisco, CA
    2 days ago
  •  ...About Town Town is building the AI coworker for every team. Everyone gets their own...  ...previously led EPD at Plaid and applied AI product at Google.   About the Role The core...  ...anyone automate real work, a suggestion engine that discovers what to automate before you... 
    Full time

    town

    San Francisco, CA
    3 days ago
  •  ...is building the TikTok for mini apps — an AI-driven consumer platform where people...  ...shape the next generation of consumer social products. Download our app and give it a try!...  ...We are looking for a product-centric engineer who cares deeply about user experience, moves... 
    Full time
    Remote work

    sekai

    San Francisco, CA
    3 days ago
  • Samba is a media intelligence company. We know what the world is watching, reading, and thinking about — in real time, at scale, across every screen. Our data exists with the consent of over a billion people, organized into the most complete picture of consumer attention...
    Full time

    sambatv

    San Francisco, CA
    3 days ago
  • $200k - $300k

     ...About the Company Our client is a fast-growing, Series A legal AI company building automation for plaintiff law firms, covering...  ..., and insurance. About the Role We are hiring AI Product Engineers to design, build, and iterate on the company's core AI-driven... 
    Full time
    H1b
    Work at office
    Visa sponsorship

    David Joseph & Company

    San Francisco, CA
    2 days ago
  •  ...About Brain Co. Brain Co. builds AI-native operating systems for large, regulated...  ...Brain Co. is entering its next phase of production deployments on a national scale with an elite...  ...About the Role As an AI Product Engineer, you will work closely with cross-functional... 
    Remote job
    Full time
    Internship
    Local area

    brainco

    San Francisco, CA
    3 days ago
  • $180k - $325k

     ...About Brain Co. Brain Co. builds AI-native operating systems for large, regulated...  ...Brain Co. is entering its next phase of production deployments on a national scale with an elite...  ...About The Role As an AI Product Engineer at Brain Co., you will play a pivotal role... 
    Remote job
    Full time
    Local area
    Worldwide

    brainco

    San Francisco, CA
    3 days ago
  • $170k - $220k

     ...About Tavrn: Tavrn is a pioneering legal AI company transforming the plaintiff law sector through automation of medical chronologies...  ...Overview: We're seeking innovative and highly motivated Product Engineers to join our dynamic product team. As a Product Engineer, you'... 
    Remote job
    Full time

    tavrn

    San Francisco, CA
    3 days ago
  • $200k - $260k

     ...organizations use to build their own clinical AI agents. Our mission is to make excellent...  ...meets the real world. Our core engineering team builds the platform. We take it into...  ...of it, as durable software that runs in production. As a Senior Applied AI Engineer, you... 
    Full time
    Work at office
    Flexible hours

    Concurrence

    San Francisco, CA
    10 days ago
  •  ...Get AI-powered advice on this job and more exclusive features. About The Role Work on large-scale AI systems, model infrastructure...  ...level Employment type ~ Full-time Job function ~ Engineering and Information Technology Industries ~ Technology,... 
    Full time

    Future Is Now

    San Francisco, CA
    4 days ago
  • $35 - $58 per hour

     ...team that's shaping an exciting array of product solutions, including Video-Based Safety,...  ...the long term. About Samsara Hardware Engineering: Samsara's hardware systems bridge the...  ...asset trackers, vehicle ADAS systems with AI/ML, telematics systems, and rackmount video... 
    Full time
    Internship
    Work at office
    Remote work
    Relocation package
    Monday to Friday
    Flexible hours

    Samsara

    San Francisco, CA
    6 hours ago
  •  ...The Principal Product Manager, Clinical Data Analytics, will play a pivotal role in shaping Tendo...  .... This person will work closely with engineering, clinical informatics, and customer-facing...  ...’s marketplace growth. Under Tendo’s AI-First R&D model, this role operates as... 
    Principal
    Remote job
    Full time

    tendo

    San Francisco, CA
    2 days ago
  • $200k - $250k

     ...Description About the Role Join an early-stage AI company building software for physical...  ..., from early experiments through production use. What You'll Do Build agent harnesses...  ...For Four to ten years of software engineering experience, including ownership of... 
    Remote work

    Wintermeyer Ventures

    San Francisco, CA
    6 days ago
  • Zipporin is seeking an exceptional Senior Full-Stack AI Product Engineer to join our team in San Francisco, CA on a hybrid schedule. In this role, you will bridge the gap between cutting-edge foundational AI models and polished, customer-facing product experiences. You... 
    Contract work

    Zipporin

    San Francisco, CA
    13 days ago
  • Zipporin is seeking an exceptional Senior Full-Stack AI Product Engineer to join our team in San Francisco on a hybrid basis. In this role, you will lead the architecture and deployment of intelligent, user-facing AI applications, transforming cutting-edge generative AI... 
    Contract work

    Zipporin

    San Francisco, CA
    1 day ago
  •  ...Job Description Job Description Founding Senior AI Engineer — Agent Quality & Evaluation On behalf of an early-stage startup...  ...Requirements What we're looking for ~3+ years building production ML or AI systems, with recent hands-on shipping of LLM products... 
    Full time

    Fruition Executive Search

    San Francisco, CA
    28 days ago
  • $220k - $330k

     ...the data centers, software, and applications that power today’s AI stack using sustainable technology solutions. We thrive on a...  ...do.   About the Role Nscale is looking for a  Staff AI Product Engineer to drive technical direction across the AI product domain. Working... 
    Contract work
    Flexible hours

    Nscale

    San Francisco, CA
    8 hours ago
  • $220k - $330k

     ...software, and applications that power today's AI stack using sustainable technology...  ...Role Nscale is looking for a  Staff AI Engineer (Specialised)to set technical direction for...  ...quality broadly Work with research, product, and infrastructure leadership so the platform... 
    Flexible hours

    Nscale

    San Francisco, CA
    8 hours ago
  • $180k - $275k

     ...Job Description Job Description Senior AI Engineer Company: Confidential — consumer and business banking Location: San Francisco...  ...end to end, from ideation to deployment, and help shape the product and engineering culture. What You Will Do Build and ship... 
    Full time
    Work at office
    Visa sponsorship

    Rumi

    San Francisco, CA
    2 days ago
  • $254k - $407k

     ...partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments...  ...realize their greatest potential. Title and Summary Senior Principal AI Engineer- AI Center of Excellence Our Purpose: Mastercard powers... 
    Principal
    Full time
    Part time
    Remote work
    Worldwide
    Flexible hours

    Mastercard

    San Francisco, CA
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Principal AI Product Engineer. Be the first to apply!