Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Staff Software Engineer - AI/ML Inference Engineering

$201.3k - $352.3k
Full-time

ServiceNow

Company Description It all started when engineer Fred Luddy wrote code that automated a tedious task for his coworker, Phyllis. She cried tears of joy. That moment inspired Fred to build a company that could do that for everyone—freeing people from busywork so they could focus on meaningful work. Today, ServiceNow is the AI control tower for business reinvention. Our ServiceNow AI platform brings together any AI, any data, and any workflow— helping 85% of the Fortune 500® work smarter, faster, and better. We're building an AI-native culture where technology and talent are unstoppable together. And we're just getting started. Join us to put AI to work for people.

Job Description

What you get to do in this role: About the team - The AI Services Platform Group at ServiceNow is a customer-focused innovation group building intelligent software and smart user experiences using existing and latest advanced technologies to enable end-to-end, industry-leading work experiences for customers. We are a group of researchers, applied scientists, engineers, and product managers with a dual mission. We build and evolve the AI platform, and partner with teams to build products and end-to-end AI-powered work experiences. In equal measure, we lay the foundations, research, experiment, and de-risk AI technologies that unlock new work experiences in the future. We are seeking a Senior Staff Software Engineer - AI/ML Inference Engineering to lead the architecture, optimization, and operational excellence of our AI inference platform. You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure. You'll also architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing. This role combines deep technical expertise in GPU compute, LLM APIs, inference frameworks, containerization, orchestration, and API gateway architecture with a bias toward pragmatic engineering and measurable performance impact.

Qualifications

To be successful in this role you have:
  • Expertise in Go or Python, OOP, Design Patterns, time and space-efficient algorithms
  • Model-serving and runtime experience, such as vLLM, TensorRT-LLM, Triton Inference Server.
  • Exposure to LLM APIs or ML-serving systems, or a clear ability to ramp up quickly
  • Experience building new products that use challenging algorithms
  • Expertise in coding efficient, object-oriented, modularized and quality software
  • Knowledge of core AI/ML techniques and algorithms
  • Knowledge of unit testing, profiling, and code tuning
AI skills Artificial intelligence strategy is a critical skill at Experienced proficiency for IC5 — it applies to how you build and to what you build.
  • Confident use of AI coding assistants such as GitHub Copilot or Cursor, paired with the critical eye to review AI-generated code as rigorously as any other — catching logic errors, security anti-patterns, and missed edge cases rather than treating model output as production-ready.
  • AI-assisted debugging and trace analysis, and a clear understanding of the data privacy rules governing what goes into a prompt.
Mandatory minimum
  • 8+ years of Backend software engineering experience
  • 4+ years of designing and operating distributed systems
  • Bachelor’s degree in computer science, software engineering, or a closely related technical field required.
  • A master’s degree may offset up to one year of the experience minimum. Equivalent practical experience is considered where technical depth is clearly demonstrable.
Desired skills:
  • Deploy, scale, and operate services in production Kubernetes environments, including Helm-based deployments and CI/CD pipelines that make releases repeatable and safe to roll back.
  • Own observability for what you build — meaningful metrics, useful logs, real tracing, and alerts that fire on customer impact rather than on noise.
  • Debug and resolve production incidents independently, participate in on-call, run root-cause analysis, and operate against defined service level objectives.
For positions in this location, we offer a base pay of $201,300 - $352,300 , plus equity (when applicable), variable/incentive compensation and benefits. Sales positions generally offer a competitive On Target Earnings (OTE) incentive compensation structure. Please note that the base pay shown is a guideline, and individual total compensation will vary based on factors such as qualifications, skill level, competencies, and work location. We also offer health plans, including flexible spending accounts, a 401(k) Plan with company match, ESPP, matching donations, a flexible time away plan and family leave programs. Compensation is based on the geographic location in which the role is located and is subject to change based on work location.

Additional Information

Work Personas We approach our distributed world of work with flexibility and trust. Work personas (flexible, remote, or required in office) are categories that are assigned to ServiceNow employees depending on the nature of their work and their assigned work location. Learn more here. To determine eligibility for a work persona, ServiceNow may confirm the distance between your primary residence and the closest ServiceNow office using a third-party service. Equal Opportunity Employer ServiceNow is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, national origin, age, disability, gender identity, veteran status, or any other category protected by law. In addition, all qualified applicants with arrest or conviction records will be considered for employment in accordance with legal requirements. Accommodations We strive to create an accessible and inclusive experience for all candidates. If you require a reasonable accommodation to complete any part of the application process, or are unable to use this online application and need an alternative method to apply, please contact [email protected] for assistance. Export Control Regulations For positions requiring access to controlled technology subject to export control regulations, including the U.S. Export Administration Regulations (EAR), ServiceNow may be required to obtain export control approval from government authorities for certain individuals. All employment is contingent upon ServiceNow obtaining any export license or other approval that may be required by relevant export control authorities. From Fortune. ©2026 Fortune Media IP Limited. All rights reserved. Used under license.
Vacancy posted 15 days ago
Similar jobs that could be interesting for youBased on the Senior Staff Software Engineer - AI/ML Inference Engineering in Santa Clara, CA vacancy
  • $193.93k - $352.29k

     ...most immediate and profound opportunity for AI to drive positive change in the physical...  ...other leading investors.About the RoleThe ML Infrastructure team is responsible for building...  ...-road validation.Maintain an in-house ML inference platform to serve large language models... 
    Senior
    Immediate start
    Flexible hours

    Nuro

    Mountain View, CA
    15 hours ago
  •  ...Systems builds the world's largest AI chip, 56 times larger than GPUs...  ...industry-leading training and inference speeds; over 10 times faster...  ...decode on the Cerebras Wafer-Scale Engine.We are hiring a Software Engineer to build and evolve the ML API layer that makes this... 
    Suggested

    Cerebras Systems

    Sunnyvale, CA
    2 days ago
  •  ...builds the world's largest AI chip, 56 times larger than...  ...industry-leading training and inference speeds; over 10 times...  ...on the Cerebras Wafer-Scale Engine.We are hiring a Software Engineer to productionize and...  ...LLM, or another open-source ML systems project.Experience... 
    Suggested

    Cerebras Systems

    Sunnyvale, CA
    3 days ago
  •  ...InworldInworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We...  ...contributions to major inference engines, or deep-dive technical write-ups.Full...  ...practical experience building backend or ML systems.Who Thrives HereYou don’t... 
    Senior
    Full time
    Work at office
    Relocation package

    Inworld AI

    Mountain View, CA
    2 days ago
  •  ...At Rhoda AI, we’re building the next generation of generalist intelligent robots...  ...accelerators. We're looking for an Inference Optimization engineer to make Rhoda's models run fast, correctly...  ...experience in inference optimization, ML systems, ML compilers, or a closely... 
    Senior
    Full time

    rhoda-ai

    Mountain View, CA
    7 days ago
  • $250k - $300k

     ...Job Description Job Description Senior Machine Learning Engineer, Inference Job Summary For our client, a fast-growing AI company building realtime voice and LLM technology,...  ...Learning Engineer with strong expertise in ML inference optimization to join its inference... 
    Senior
    Permanent employment
    Full time
    Contract work
    H1b
    Local area
    Remote work
    Relocation
    Relocation package

    AIDA Recruitment

    Mountain View, CA
    3 days ago
  • $130k - $220k

    Santa Clara, CASoftware Engineering - Motion Planning /Full...  ...is a Physical AI company pioneering AI-based virtual driver software for factory-built autonomous...  ...production runtime pipeline for ML-based autonomous...  ...stack.Optimize end-to-end inference performance through... 
    Senior
    Full time

    Plus.ai

    Santa Clara, CA
    4 days ago
  • $152k - $241.5k

    NVIDIA is seeking talented engineers to join the TensorRT team and advance high-performance AI inference on NVIDIA GPUs. Our team develops software that accelerates state-of-the-art generative...  ...and hardware configurations.As a Senior Software Engineer, you will develop... 
    Senior
    Full time
    Local area

    Nvidia

    Santa Clara, CA
    4 days ago
  • $184k - $287.5k

    NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a Senior Software Engineer - AI Inference Performance to advance innovative LLM and VLM inference. You will push workloads toward practical performance limits on NVIDIA GPU-accelerated... 
    Senior
    Full time

    Nvidia

    Santa Clara, CA
    15 hours ago
  •  ...Systems builds the world's largest AI chip, 56 times larger than GPUs. This...  ...deliver industry-leading training and inference speeds; over 10 times faster than...  ...Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure... 
    Senior

    Cerebras Systems

    Sunnyvale, CA
    3 days ago
  •  ...Systems builds the world's largest AI chip, 56 times larger than...  ...industry-leading training and inference speeds; over 10 times faster...  ...decode on the Cerebras Wafer-Scale Engine.We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes this... 
    Senior

    Cerebras Systems

    Sunnyvale, CA
    2 days ago
  •  ...builds the world's largest AI chip, 56 times larger than...  ...-leading training and inference speeds; over 10 times faster...  ...Infrastructure team builds the software systems that power engineering workflows across Cerebras....  ..., CI, release, quality, ML systems, and product... 
    Senior
    Remote work

    Cerebras Systems

    Sunnyvale, CA
    2 days ago
  • $175k - $308.5k

     ...SummaryWe are the Foundation Model Inference team within Cloud OS and AI Inference organization. We are on a...  ...in inference efficiency, hardware/software codesign, systems architecture, and...  ...set the technical direction for the engineers around you.This role sits within CloudOS... 
    Senior
    Worldwide
    Relocation

    Apple

    Santa Clara, CA
    1 day ago
  • $262k - $364k

     ...technical goal, roadmap, and software architecture for hardware/software...  ..., optimizing large-scale ML inference and training systems.Design,...  ...in Computer Science, Computer Engineering, Electrical Engineering, a...  ...work to shape the future of AI/ML hardware acceleration. You... 
    Senior
    Worldwide

    Google

    Sunnyvale, CA
    1 day ago
  •  ...the world's largest AI chip, 56 times...  ...leading training and inference speeds; over 10...  ...RoleWe're hiring a Staff Engineer to help lead, drive...  ...maintain production software, with...  ...effectiveness of senior engineers through...  ...Influence. Partner with ML, Product, Infrastructure... 

    Cerebras Systems

    Sunnyvale, CA
    1 day ago
  • $262k - $364k

     ...architecture and design of the inference and training AI infrastructure from SRE...  ...while working closely with senior technical leads in the development...  ...8 years of experience with software development in one or more...  ...in Computer Science or Engineering.Site Reliability... 
    Senior

    Google

    Sunnyvale, CA
    7 hours ago
  •  ...builds the world's largest AI chip, 56 times larger than GPUs...  ...-leading training and inference speeds; over 10 times faster...  ...About the RoleWe’re hiring a Senior Software Engineer for our Inference Cloud team...  ...operation, partnering closely with ML, Product, and Infrastructure... 
    Senior

    Cerebras Systems

    Sunnyvale, CA
    3 days ago
  • $262k - $364k

     ...coach a distributed engineering team, fostering...  ...and deploy scalable software and advanced...  ...accelerate HPC, RDMA, and ML workloads....  ...RDMA, storage, and AI/ML, staying ahead of training and inference advancements.Collaborative...  ...(GDS).As a Senior Staff Software Engineer,... 
    Senior
    Remote work
    Worldwide

    Google

    Sunnyvale, CA
    3 days ago
  • $150.4k - $277.6k

    Role Number: 200690248Team: Software and ServicesLocation: Cupertino; SeattlePosted...  ...-19SummaryApple is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute, the backbone of Siri AI and Apple Intelligence. You'll... 
    Senior
    Remote work
    Worldwide
    Relocation

    Apple

    Cupertino, CA
    1 day ago
  • $152k - $241.5k

     ...recently, GPU deep learning ignited modern AI — the next era of computing — with...  ...for an AI & Deep Learning Compiler Engineer. NVIDIA is hiring software engineers for its Deep Learning & AI...  ...has been the backbone of NVIDIA’s inference engine, spanning across data centers... 
    Senior
    Full time
    Remote work

    Nvidia

    Santa Clara, CA
    3 days ago
  •  ...builds the world's largest AI chip, 56 times larger than GPUs...  ...-leading training and inference speeds; over 10 times faster...  ...About the RoleWe’re hiring a senior backend engineer to build and scale the services...  ...building and operating production software.Education. Bachelor’s or... 
    Senior

    Cerebras Systems

    Sunnyvale, CA
    15 hours ago
  • $193.93k - $352.29k

     ...most immediate and profound opportunity for AI to drive positive change in the physical...  ...leading investors. About The Role The ML Infrastructure team is responsible for...  ...road validation. Maintain an in-house ML inference platform to serve large language models efficiently... 
    Senior
    Immediate start
    Flexible hours

    Nuro

    Mountain View, CA
    4 days ago
  • $193.93k - $352.29k

     ...most immediate and profound opportunity for AI to drive positive change in the physical...  ...leading investors. About the Role The ML Infrastructure team is responsible for...  ...road validation. Maintain an in-house ML inference platform to serve large language models efficiently... 
    Senior
    Immediate start
    Flexible hours

    Nuro

    Mountain View, CA
    11 days ago
  •  ...sits at the intersection of Generative AI, Information Retrieval, and Knowledge Graphs. We're looking for a senior machine learning engineer to own the conversational intelligence...  ...and tool-calling, the low-latency LLM inference behind it, and the knowledge services that... 
    Senior

    Apple

    Cupertino, CA
    2 days ago
  • $165.2k - $223.6k

    Amazon AI is looking for a machine learning engineer to be part of inference optimization team working on optimizing large language...  ...existing and/or new inference software. List of responsibilities...  ...other employees, supervisors, and staff; adhere to standards of excellence... 
    Internship
    Local area
    Work from home
    Flexible hours

    Amazon

    Santa Clara, CA
    4 days ago
  • $174.72k - $295.68k

     ..., integrating advanced AI and autonomous driving...  ...-time Machine Learning Engineer - AI Foundation, with deep...  ...a state-of-art ML infrastructure for training...  ...accelerating model training/inference. Our mission is to...  ...with a team of talented software engineers, machine learning... 
    Senior
    Full time

    XPENG Motors

    Santa Clara, CA
    3 days ago
  • $200k - $322k

     ...people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our...  ...to do their best work. We are seeking a highly skilled Senior Staff Software Engineer to join our dynamic team. Our company is at the forefront of... 
    Senior
    Full time
    Remote work

    NVIDIA

    Santa Clara, CA
    4 days ago
  • $224k - $356.5k

    NVIDIA is looking for a Machine Learning Engineer to join the GPU accelerated Apache...  ...scale workloads for ETL, SQL, and ML/DL model training and inference pipelines, spanning many domains and...  ...with GPUs. You will apply the latest ML/AI methods to empower enterprises to migrate... 
    Senior
    Full time

    Nvidia

    Santa Clara, CA
    4 days ago
  • $184k - $287.5k

     ...the boundless capabilities of AI to build the next era of...  ...systems at scale. We seek a Senior ML Engineer to compose and deliver next-...  ...quantization, and real-time inference optimization for production...  ...creative and autonomous Systems Software Engineer who loves... 
    Senior
    Full time

    Nvidia

    Santa Clara, CA
    15 hours ago
  •  ...Senior MLOps Engineer (LLM / Agentic Systems) We are seeking a Senior MLOps...  ...platforms for next-generation AI systems. This role focuses on...  ...regressions Integrate AI/ML models into scalable backend...  ...distributed systems for large-scale inference Optimize systems for... 
    Senior

    United IT

    Cupertino, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Staff Software Engineer - AI/ML Inference Engineering. Be the first to apply!