Principal AI Product Engineer
$290kNscale
About Nscale
Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.
About the Role
Nscale is looking for a Principal AI Engineer (Specialised) to lead the inference and post-training pillar of our AI systems engineering organization. You’ll define the multi-year technical roadmap for how models are served, evaluated, and post-trained on Nscale’s GPU cloud, across dedicated and serverless inference and bring-your-own-model deployments. You’ll lead the most consequential architectural programmes in that space and set the engineering standards that 20–50+ engineers build to.
As a Principal engineer, you are one of the deepest technical authorities in the company on AI systems. Your decisions set the cost, latency, and reliability at which Nscale serves tokens and runs post-training workloads, and those numbers have to compete with the world’s leading AI infrastructure providers. The problems span the full stack: kernel efficiency on state-of-the-art GPU systems, fleet-level KV cache and serving architecture, the evals that prove model quality, and RL loops where inference and training share hardware. You frame the solutions the organization executes against, including the API contracts customers see.
How We Work
- Dog years. We move quickly and compress a lot of learning into a short time.
- Don’t let perfect be the enemy of good. Ship, measure, iterate.
- Be relentless. Own the problem end to end and see it through.
- One team, one mission. Outcomes over process, and no “not my job”.
Responsibilities
- Define and own the multi-year technical roadmap for Nscale’s inference, evals, and post-training platform, and translate it into architecture that multiple teams can execute against
- Lead company-scale architectural initiatives in the pillar, such as next-generation serving (disaggregated prefill/decode, KV cache orchestration across GPU, host, and storage tiers, speculative decoding, multi-tenant scheduling), GPU kernel and model efficiency work (custom kernels, FP8/NVFP4/INT8/4 quantization, sparsity, distillation, MoE serving), evals and benchmarking frameworks, and post-training and RL infrastructure
- Establish engineering standards adopted across all AI teams: API design and compatibility guarantees, benchmarking and evals methodology, training stability norms, and performance testing practices
- Own the framework by which cost, latency, throughput, and model quality trade-offs are made and measured across the pillar
- Identify long-horizon systemic risks early (serving engine and framework bets, accelerator support, capability gaps) and resolve them before they block the organization
- Align AI engineering, research, product, and infrastructure leadership on multi-team technical strategy; frame technical trade-offs in product and commercial terms
- Mentor and develop Staff and Senior AI Engineers, and grow the next generation of inference technical leaders at Nscale
- Represent Nscale’s technical approach externally: open-source leadership in the frameworks we depend on, publications, conference talks, and partnerships with GPU vendors and AI labs
Requirements
- 10–15 years of engineering experience, with a clear track record of pillar-level impact on production AI systems
- 4+ years of hands-on work with LLMs in inference, GPU performance, evals, or post-training and RL, in production or research
- Demonstrated ability to define multi-year technical strategy for complex, multi-team AI systems organizations
- World-class depth in production LLM inference, GPU performance, evals, and/or post-training and RL infrastructure, with strong working knowledge across the rest
- Demonstrated ownership of the architecture of a large-scale production inference or training platform
- Proven ability to create architectural frameworks and engineering standards adopted across large engineering organizations
- Deep understanding of the hardware/software boundary for AI accelerators: CUDA or ROCm, memory bandwidth and interconnect constraints, and distributed compute paradigms
- Strong history of growing technical leaders (Staff and above) and multiplying technical capability across teams
- External recognition in the AI systems community through research, open source, or industry contribution
Preferred
- Prior experience at a top-tier AI lab or major hyperscaler AI infrastructure team
- Maintainer or core contributor to a foundational inference, kernel, or RL framework (vLLM, SGLang, TensorRT-LLM, LMCache, FlashInfer, Triton, verl, OpenRLHF, TRL, DeepSpeed, Megatron-LM, etc.)
- Hands-on depth in RL for LLMs (DPO/GRPO-style methods, reward modelling, multi-turn and tool-use RL) and the interaction between inference and training infrastructure
- Experience defining developer API platforms adopted at scale by external developers
- Deep experience with control plane / data plane architecture and cell-based deployment patterns in large-scale inference infrastructure
- Published work in AI systems: MLSys, NeurIPS Systems Track, OSDI, EuroSys, SC, or equivalent
- Experience with hardware-software co-design: custom accelerator kernels (CUDA, Triton), compiler-level optimization, AI hardware roadmap engagement
- Experience defining pricing, SLO, and capacity models for a commercial inference product
The range below reflects the base salary for the position. Actual compensation may vary based on job-related factors such as skill set, experience, education, and location. In addition to base salary, this role may be eligible for bonus, equity, and/or commission programs. Nscale may offer a competitive benefits package including medical, dental, vision, flexible paid time off, parental leave, and retirement plan participation.
Salary Range
$290,000—$520,000 USD
For information on how Nscale handles candidate personal data, please see our Employee & Candidate Privacy Notice: Here. Nscale does not accept unsolicited candidate submissions from recruitment agencies.
- ...About the Role As a Principal AI Engineer at Innovaccer, you will be one of our most senior engineers and you will stay deep in the code... ...the AI/ML-powered features that increasingly define our products — while setting the technical bar for how those systems are...PrincipalTemporary work
- ...workforce through an intelligent, auditable AI platform that spans the entire... ...confidential search, Scovai is seeking a Principal Generative AI Engineer to serve as the technical cornerstone... ...role, you will architect and deliver production-grade generative AI systems —...PrincipalFull time
$250k - $350k
...t fall through the cracks. We believe AI should do more. Thorin is building proactive... ...workflows. Own features from the initial product decision through implementation and... ...others, and raise the quality of a small engineering pod. Turn product and customer signal...SuggestedFull time$150k - $200k
San Francisco, CaliforniaEngineering - AI Engineering /US Full-time Salaried /On-siteSamba is a media intelligence company. We know what... ...selectionWHO YOU AREYou’ve built AI agents and shipped them to production—not just prototypesYou’ve deployed agents in cloud...SuggestedFull timeLocal area$90k - $110k
...develop, and operationalize secure agentic AI capabilities in risk & security... ...where AI agents can accelerate analyst productivity and improve outcomesContinuously validate... ...experience leading enterprise-scale security engineering initiatives5+ years of progressive experience...PrincipalFull time$215.25k - $287.91k
...DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...compelling business stories. You will connect products, models, agents, and data into... ...individual-contributor role combining software engineering, applied AI, solution architecture, and storytelling...PrincipalFull time$157.25k - $212.75k
...Francisco, CAEngineering /Full Time /RemoteThe Principal Product Manager, Clinical Data Analytics, will... .... This person will work closely with engineering, clinical informatics, and customer-... ...’s marketplace growth. Under Tendo’s AI-First R&D model, this role operates as...PrincipalFull timeTemporary workLocal areaFlexible hours$200k - $320k
...Job Description Job Description Senior AI Product Engineer Location: San Francisco Bay Area (Hybrid/Flexible – Occasional Office Visits) Compensation: $200,000–$320,000 Base Salary + Competitive Equity Employment: Full-Time Industry: Artificial Intelligence...Full timeWork at officeRemote workFlexible hours$127.2k - $201.3k
...as we are, join our team.KPMG is currently seeking a Manager, AI Product Security Leader to join our Enterprise Security Services... ...secure model- and tool-interaction patternsLead complex security engineering initiatives end-to-end with minimal oversight, from risk and...H1bLocal area$200k - $250k
...About Us At You.com, we are building the AI Search Infrastructure that powers modern AI systems. Our goal is to create... ...reliable, transparent, and useful. Our team includes engineers, researchers, product builders, and operators who care about solving meaningful...Remote jobImmediate startWork from homeFlexible hoursEarly shift$140k - $180k
...Latch builds rigorous scientific benchmarks ( benchmarks.bio ) for AI agents across biology. We work with frontier labs and pharma to... ...We're seeking an AI Biologist to join our Protein Design & Engineering team, working at the frontier of what artificial intelligence can...Full timeFor contractorsH1bWork at officeRemote workNight shift- ...About Town Town is building the AI coworker for every team. Everyone gets their own... ...previously led EPD at Plaid and applied AI product at Google. About the Role The core... ...anyone automate real work, a suggestion engine that discovers what to automate before you...Full time
- ...is building the TikTok for mini apps — an AI-driven consumer platform where people... ...shape the next generation of consumer social products. Download our app and give it a try!... ...We are looking for a product-centric engineer who cares deeply about user experience, moves...Full timeRemote work
- Samba is a media intelligence company. We know what the world is watching, reading, and thinking about — in real time, at scale, across every screen. Our data exists with the consent of over a billion people, organized into the most complete picture of consumer attention...Full time
$200k - $300k
...About the Company Our client is a fast-growing, Series A legal AI company building automation for plaintiff law firms, covering... ..., and insurance. About the Role We are hiring AI Product Engineers to design, build, and iterate on the company's core AI-driven...Full timeH1bWork at officeVisa sponsorship- ...About Brain Co. Brain Co. builds AI-native operating systems for large, regulated... ...Brain Co. is entering its next phase of production deployments on a national scale with an elite... ...About the Role As an AI Product Engineer, you will work closely with cross-functional...Remote jobFull timeInternshipLocal area
$180k - $325k
...About Brain Co. Brain Co. builds AI-native operating systems for large, regulated... ...Brain Co. is entering its next phase of production deployments on a national scale with an elite... ...About The Role As an AI Product Engineer at Brain Co., you will play a pivotal role...Remote jobFull timeLocal areaWorldwide$170k - $220k
...About Tavrn: Tavrn is a pioneering legal AI company transforming the plaintiff law sector through automation of medical chronologies... ...Overview: We're seeking innovative and highly motivated Product Engineers to join our dynamic product team. As a Product Engineer, you'...Remote jobFull time$200k - $260k
...organizations use to build their own clinical AI agents. Our mission is to make excellent... ...meets the real world. Our core engineering team builds the platform. We take it into... ...of it, as durable software that runs in production. As a Senior Applied AI Engineer, you...Full timeWork at officeFlexible hours- ...Get AI-powered advice on this job and more exclusive features. About The Role Work on large-scale AI systems, model infrastructure... ...level Employment type ~ Full-time Job function ~ Engineering and Information Technology Industries ~ Technology,...Full time
$35 - $58 per hour
...team that's shaping an exciting array of product solutions, including Video-Based Safety,... ...the long term. About Samsara Hardware Engineering: Samsara's hardware systems bridge the... ...asset trackers, vehicle ADAS systems with AI/ML, telematics systems, and rackmount video...Full timeInternshipWork at officeRemote workRelocation packageMonday to FridayFlexible hours- ...The Principal Product Manager, Clinical Data Analytics, will play a pivotal role in shaping Tendo... .... This person will work closely with engineering, clinical informatics, and customer-facing... ...’s marketplace growth. Under Tendo’s AI-First R&D model, this role operates as...PrincipalRemote jobFull time
$200k - $250k
...Description About the Role Join an early-stage AI company building software for physical... ..., from early experiments through production use. What You'll Do Build agent harnesses... ...For Four to ten years of software engineering experience, including ownership of...Remote work- Zipporin is seeking an exceptional Senior Full-Stack AI Product Engineer to join our team in San Francisco, CA on a hybrid schedule. In this role, you will bridge the gap between cutting-edge foundational AI models and polished, customer-facing product experiences. You...Contract work
- Zipporin is seeking an exceptional Senior Full-Stack AI Product Engineer to join our team in San Francisco on a hybrid basis. In this role, you will lead the architecture and deployment of intelligent, user-facing AI applications, transforming cutting-edge generative AI...Contract work
- ...Job Description Job Description Founding Senior AI Engineer — Agent Quality & Evaluation On behalf of an early-stage startup... ...Requirements What we're looking for ~3+ years building production ML or AI systems, with recent hands-on shipping of LLM products...Full time
$220k - $330k
...the data centers, software, and applications that power today’s AI stack using sustainable technology solutions. We thrive on a... ...do. About the Role Nscale is looking for a Staff AI Product Engineer to drive technical direction across the AI product domain. Working...Contract workFlexible hours$220k - $330k
...software, and applications that power today's AI stack using sustainable technology... ...Role Nscale is looking for a Staff AI Engineer (Specialised)to set technical direction for... ...quality broadly Work with research, product, and infrastructure leadership so the platform...Flexible hours$180k - $275k
...Job Description Job Description Senior AI Engineer Company: Confidential — consumer and business banking Location: San Francisco... ...end to end, from ideation to deployment, and help shape the product and engineering culture. What You Will Do Build and ship...Full timeWork at officeVisa sponsorship$254k - $407k
...partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments... ...realize their greatest potential. Title and Summary Senior Principal AI Engineer- AI Center of Excellence Our Purpose: Mastercard powers...PrincipalFull timePart timeRemote workWorldwideFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Principal AI Product Engineer. Be the first to apply!
- principal cloud engineer San Francisco, CA
- principal battery engineer San Francisco, CA
- senior civil engineer project manager San Francisco, CA
- principal developer San Francisco, CA
- principal data engineer San Francisco, CA
- technical director engineering San Francisco, CA
- principal security engineer San Francisco, CA
- principal engineer San Francisco, CA
- project engineer assistant project manager San Francisco, CA
- director systems engineering San Francisco, CA






