Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Staff AI Product Engineer

$220k - $330k

Nscale

About Nscale

Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.

 

About the Role

Nscale is looking for a  Staff AI Engineer (Specialised) to set technical direction for part of the inference platform at the core of our AI cloud: dedicated and serverless inference, bring-your-own-model deployments, and the post-training services that sit alongside them.

You’ll be the technical authority for one or more focus areas, working across the teams that build serving, post-training, and platform. Your decisions shape the latency, throughput, and cost of the tokens Nscale serves, and whether we can say with evidence that the models we run are the right ones. You’ll take on open questions, such as how KV cache should move between GPUs, nodes, and storage, or where hand-written kernels beat the open-source defaults, and your answers become the standards others build on.

We run state-of-the-art GPU systems, and much of the open-source ecosystem hasn’t caught up with them yet. This role is for people who want to close that gap hands-on while growing the engineers around them.

How We Work

  • Dog years. We move quickly and compress a lot of learning into a short time.
  • Don’t let perfect be the enemy of good. Ship, measure, iterate.
  • Be relentless. Own the problem end to end and see it through.
  • One team, one mission. Outcomes over process, and no “not my job”.

Focus Areas

We’re hiring for depth. You don’t need all of these; we want people who are exceptional in  one or more , and we’re deliberately hiring people with different focus areas:

  • KV cache offloading and inference performance: KV cache orchestration across GPU, host memory, and storage; cross-instance cache sharing (LMCache or similar); KV-aware routing; disaggregated prefill/decode; speculative decoding; quantisation (FP8, NVFP4, INT8/4); MoE serving
  • GPU kernels and performance: writing and tuning kernels in CUDA, Triton, CUTLASS or ROCm; attention, MoE and GEMM optimisation; multi-GPU and multi-node communication on NVLink-scale systems
  • Evals and benchmarking: designing evaluation frameworks, turning customer requirements into custom benchmarks, and producing objective model comparisons we can stand behind
  • Post-training and RL: fine-tuning and preference optimisation as a service; RL for LLMs (PPO/GRPO-style, reward modelling, multi-turn and tool-use RL); rollout generation, weight synchronisation, and sharing infrastructure between inference and training
  • Serving platform and APIs: serving engines (vLLM, SGLang, TensorRT-LLM), model onboarding, and OpenAI-compatible APIs that other engineers and customers build on

Responsibilities

  • Set technical direction for your focus areas and turn it into work that multiple teams can deliver
  • Lead the resolution of systemic performance and reliability problems across the serving stack, from kernel bottlenecks to fleet-level capacity and multi-tenant isolation
  • Own the trade-offs between cost, latency, throughput, and model quality in your area, and back them with measurement
  • Build reusable frameworks, benchmarks, and tooling that make other AI engineers at Nscale more effective
  • Evaluate emerging serving engines, kernel libraries, RL frameworks, and accelerators, and make clear build/adopt/contribute recommendations
  • Coach and grow engineers across teams, and raise engineering quality broadly
  • Work with research, product, and infrastructure leadership so the platform tracks customer demand
  • Represent your area in cross-team technical reviews and planning

Requirements

  • 8–12 years of engineering experience, with significant depth in production AI systems or ML research at scale
  • 4+ years of hands-on work with LLMs in at least one of the focus areas above, in production or research
  • Deep, demonstrated expertise in at least one of the focus areas above
  • Enough working knowledge of the full serving stack to debug across it: API, router, scheduler, engine, kernel, and cluster
  • A track record of setting technical direction and creating standards or tools adopted beyond your own team
  • Strong Python and PyTorch, with a track record of maintainable, well-tested, production-grade systems
  • Experience operating AI workloads in containerised, distributed environments (Kubernetes, large GPU clusters)
  • Solid understanding of GPU performance: memory bandwidth, interconnect, and parallelism strategies (data, tensor, pipeline, expert)
  • Deep understanding of transformer and LLM architectures and how they behave under production load

Preferred

  • Contributions to widely used open-source inference, kernel, or RL projects (vLLM, SGLang, TensorRT-LLM, LMCache, FlashInfer, Triton, verl, OpenRLHF, TRL, DeepSpeed, etc.)
  • Experience at an AI lab, hyperscaler AI team, or leading ML infrastructure company
  • Published research or technical writing on inference, kernels, evals, or RL (MLSys, NeurIPS, ICLR, blog posts)
  • Hands-on custom kernel work (CUDA, Triton, CUTLASS) if kernels aren’t already your focus area
  • Experience designing developer-facing APIs and SDKs used by external customers
  • Experience with control plane / data plane separation and cell-based architecture for scale-out and blast-radius isolation

The range below reflects the base salary for the position. Actual compensation may vary based on job-related factors such as skill set, experience, education, and location. In addition to base salary, this role may be eligible for bonus, equity, and/or commission programs. Nscale may offer a competitive benefits package including medical, dental, vision, flexible paid time off, parental leave, and retirement plan participation.

Salary Range

$220,000—$330,000 USD

For information on how Nscale handles candidate personal data, please see our Employee & Candidate Privacy Notice:  Here.

Nscale does not accept unsolicited candidate submissions from recruitment agencies.

Vacancy posted 8 hours ago
Similar jobs that could be interesting for youBased on the Staff AI Product Engineer in San Francisco, CA vacancy
  • $250k - $350k

     ...through the cracks. We believe AI should do more. Thorin is...  ...Own features from the initial product decision through...  ...raise the quality of a small engineering pod. Turn product and customer...  ...What We’re Looking For Staff-level engineering judgment, whether... 
    Suggested
    Full time

    thorin

    San Francisco, CA
    3 days ago
  • $150k - $200k

    San Francisco, CaliforniaEngineering - AI Engineering /US Full-time Salaried /On-siteSamba is a media intelligence company. We know what...  ...selectionWHO YOU AREYou’ve built AI agents and shipped them to production—not just prototypesYou’ve deployed agents in cloud... 
    Suggested
    Full time
    Local area

    Samba TV

    San Francisco, CA
    4 days ago
  • $127.2k - $201.3k

     ...as we are, join our team.KPMG is currently seeking a Manager, AI Product Security Leader to join our Enterprise Security Services...  ...secure model- and tool-interaction patternsLead complex security engineering initiatives end-to-end with minimal oversight, from risk and... 
    Suggested
    H1b
    Local area

    KPMG

    San Francisco, CA
    2 days ago
  • $140k - $180k

     ...scientific benchmarks ( benchmarks.bio ) for AI agents across biology. We work with...  ...AI Biologist to join our Protein Design & Engineering team, working at the frontier of what artificial...  ...interview with a Member of Technical Staff (pod lead). # Culture interview with... 
    Suggested
    Full time
    For contractors
    H1b
    Work at office
    Remote work
    Night shift

    LatchBio

    San Francisco, CA
    2 days ago
  •  ...About Brain Co. Brain Co. builds AI-native operating systems for large, regulated...  ...Brain Co. is entering its next phase of production deployments on a national scale with an elite...  ...About the Role As an AI Product Engineer, you will work closely with cross-functional... 
    Suggested
    Remote job
    Full time
    Internship
    Local area

    brainco

    San Francisco, CA
    3 days ago
  • $200k - $250k

     ...About Us At You.com, we are building the AI Search Infrastructure that powers modern AI systems. Our goal is to create...  ...reliable, transparent, and useful. Our team includes engineers, researchers, product builders, and operators who care about solving meaningful... 
    Remote job
    Immediate start
    Work from home
    Flexible hours
    Early shift

    You.com

    San Francisco, CA
    8 hours ago
  •  ...About Town Town is building the AI coworker for every team. Everyone gets their own...  ...previously led EPD at Plaid and applied AI product at Google.   About the Role The core...  ...anyone automate real work, a suggestion engine that discovers what to automate before you... 
    Full time

    town

    San Francisco, CA
    3 days ago
  •  ...is building the TikTok for mini apps — an AI-driven consumer platform where people...  ...shape the next generation of consumer social products. Download our app and give it a try!...  ...We are looking for a product-centric engineer who cares deeply about user experience, moves... 
    Full time
    Remote work

    sekai

    San Francisco, CA
    3 days ago
  • $200k - $320k

     ...Job Description Job Description Senior AI Product Engineer Location: San Francisco Bay Area (Hybrid/Flexible – Occasional Office Visits) Compensation: $200,000–$320,000 Base Salary + Competitive Equity Employment: Full-Time Industry: Artificial Intelligence... 
    Full time
    Work at office
    Remote work
    Flexible hours

    Kastel Staffing

    San Francisco, CA
    4 days ago
  • Samba is a media intelligence company. We know what the world is watching, reading, and thinking about — in real time, at scale, across every screen. Our data exists with the consent of over a billion people, organized into the most complete picture of consumer attention...
    Full time

    sambatv

    San Francisco, CA
    3 days ago
  •  ...Benefits & Perks Comprehensive medical, dental, and vision coverage Career growth at a rapidly scaling AI startup About This Role As an AI Product Engineer, you will design, build, and iterate on our core AI-driven product. Our small engineering team works... 
    Full time
    H1b
    Work at office
    Visa sponsorship

    Cephas Consultancy Services Private Limited

    San Francisco, CA
    5 days ago
  • $180k - $325k

     ...About Brain Co. Brain Co. builds AI-native operating systems for large, regulated...  ...Brain Co. is entering its next phase of production deployments on a national scale with an elite...  ...About The Role As an AI Product Engineer at Brain Co., you will play a pivotal role... 
    Remote job
    Full time
    Local area
    Worldwide

    brainco

    San Francisco, CA
    3 days ago
  • $170k - $220k

     ...About Tavrn: Tavrn is a pioneering legal AI company transforming the plaintiff law sector through automation of medical chronologies...  ...Overview: We're seeking innovative and highly motivated Product Engineers to join our dynamic product team. As a Product Engineer, you'... 
    Remote job
    Full time

    tavrn

    San Francisco, CA
    3 days ago
  • $35 - $58 per hour

     ...team that's shaping an exciting array of product solutions, including Video-Based Safety,...  ...the long term. About Samsara Hardware Engineering: Samsara's hardware systems bridge the...  ...asset trackers, vehicle ADAS systems with AI/ML, telematics systems, and rackmount video... 
    Full time
    Internship
    Work at office
    Remote work
    Relocation package
    Monday to Friday
    Flexible hours

    Samsara

    San Francisco, CA
    6 hours ago
  • $290k

     ...applications that power today's AI stack using sustainable...  ...is looking for a  Principal AI Engineer (Specialised) to lead the inference...  ...AI engineering, research, product, and infrastructure leadership...  ...commercial terms Mentor and develop Staff and Senior AI Engineers, and... 
    Flexible hours

    Nscale

    San Francisco, CA
    8 hours ago
  • $220k - $330k

     ...data centers, software, and applications that power today’s AI stack using sustainable technology solutions. We thrive on a...  ...all we do.   About the Role Nscale is looking for a  Staff AI Product Engineer to drive technical direction across the AI product domain.... 
    Contract work
    Flexible hours

    Nscale

    San Francisco, CA
    8 hours ago
  •  ...Monaco is building an AI-native revenue platform that replaces the fragmented GTM stack...  ...already 65 people, and have strong early product-market fit generating millions in ARR...  ...We're looking for an AI Backend Product Engineer to help build and shape Monaco's core AI... 
    Full time
    Work at office

    monaco

    San Francisco, CA
    3 days ago
  • $200k - $260k

     ...to build their own clinical AI agents. Our mission is to make...  ...meets the real world. Our core engineering team builds the platform. We...  ...software that runs in production. As a Senior Applied AI Engineer...  ...AI ladder runs from Senior to Staff or Tech Lead. You move up by... 
    Full time
    Work at office
    Flexible hours

    Concurrence

    San Francisco, CA
    10 days ago
  • Zipporin is seeking an exceptional Senior Full-Stack AI Product Engineer to join our team in San Francisco on a hybrid basis. In this role, you will lead the architecture and deployment of intelligent, user-facing AI applications, transforming cutting-edge generative AI... 
    Contract work

    Zipporin

    San Francisco, CA
    1 day ago
  • Zipporin is seeking an exceptional Senior Full-Stack AI Product Engineer to join our team in San Francisco, CA on a hybrid schedule. In this role, you will bridge the gap between cutting-edge foundational AI models and polished, customer-facing product experiences. You... 
    Contract work

    Zipporin

    San Francisco, CA
    13 days ago
  • $200k - $250k

     ...Job Description Job Description About the Role Join a small team building AI-powered advertising products in the SaaS sector. You will shape production agents that create personalized interactive and video ads, then connect their output to live recommendation systems... 

    Wintermeyer Ventures

    San Francisco, CA
    5 days ago
  •  ...Area, California | Fully On-site We are seeking an Applied AI Engineer to join an early-stage technology company building AI systems...  ...applied AI, computer vision, evaluation systems, and production engineering. Key Responsibilities Build and deploy agentic... 
    Work at office
    Visa sponsorship

    MaxIT Consulting - Max Corporate Group

    San Francisco, CA
    15 days ago
  • $135k - $175k

     ...era of high-performance, Made in America production. About the team: Formic's Business...  ...data across platforms, and develop AI-powered tools that make information more...  ...career path. You might come from software engineering, applied AI, technical product development... 
    Full time
    Temporary work
    Remote work
    Home office
    Flexible hours
    Shift work

    Formic

    San Francisco, CA
    8 days ago
  •  ...keep reading. About the Role We are looking for an AI Enablement Engineer to accelerate AI adoption across engineering by turning standards...  ...at the intersection of AI tooling enablement, developer productivity, engineering standards, lightweight training, feedback... 
    Work at office
    Immediate start

    Emerald Development Managers

    San Francisco, CA
    3 days ago
  • $200k - $250k

     ...Description About the Role Join an early-stage AI company building software for physical...  ..., from early experiments through production use. What You'll Do Build agent harnesses...  ...For Four to ten years of software engineering experience, including ownership of... 
    Remote work

    Wintermeyer Ventures

    San Francisco, CA
    6 days ago
  • $175k - $275k

     ...company Our client is a seed-stage AI company building the data and infrastructure...  ...and application-layer companies. Its products support post-training, reinforcement learning...  ...verification. The role As AI Engineer, RL & Evals, you will build reinforcement... 
    Full time
    H1b
    Work at office

    Raydar

    San Francisco, CA
    5 days ago
  •  ...Job Description Job Description Founding Senior AI Engineer — Agent Quality & Evaluation On behalf of an early-stage startup...  ...Requirements What we're looking for ~3+ years building production ML or AI systems, with recent hands-on shipping of LLM products... 
    Full time

    Fruition Executive Search

    San Francisco, CA
    28 days ago
  • $120k - $180k

     ...Job Description About the Role This is a founding engineering role at a seed-stage AI startup building the intelligence layer for the construction...  ...vision, LLMs, and multimodal AI to prototype and ship production systems on real-world, noisy construction documents.... 
    Full time
    Work at office
    Relocation

    Clera

    San Francisco, CA
    18 days ago
  • $180k - $200k

     ...system into their real, regulated production — then you watch the business...  ...it. Embedded is not staff-augmented. Autonomous. You own...  ...how leading enterprises adopt AI, from strategy through first...  ...you measured and why. Strong engineering fundamentals — dropped into an... 
    Full time
    Work at office
    Visa sponsorship

    Provectus

    San Francisco, CA
    4 days ago
  • $180k - $275k

     ...Job Description Job Description Senior AI Engineer Company: Confidential — consumer and business banking Location: San Francisco...  ...end to end, from ideation to deployment, and help shape the product and engineering culture. What You Will Do Build and ship... 
    Full time
    Work at office
    Visa sponsorship

    Rumi

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Staff AI Product Engineer. Be the first to apply!