Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Staff Inference Engineer

$175k - $218.7k

Uniphore

Uniphore is the Business AI company. Our sovereign, composable and secure AI platform connects enterprise data, fine-tunes AI models and deploys agentic AI across the enterprise. We empower every worker to boost productivity and help businesses grow faster, operate smarter and reduce costs. Trusted by more than 2,000 businesses globally, and recognized on the Deloitte Fast 500, Uniphore delivers on the promise of AI as a transformative force for business. Job Description:Staff Inference Engineer –Uniphore is building an enterprise AI pla4orm where agents operate across voice, video, text, and structured data in real time. We are looking for a Senior or Staff Inference Engineer to own the inference layer — combining deep model inference expertise with systems engineering to deliver a context-aware runtime serving finetuned SLMs across cloud, on-premises, and widely used GPU platforms.This is a hands-on individual contributor role working closely with Uniphore’s research, model engineering, knowledge systems, agent, and pla4orm teams.What You Will Build Context-aware runtime: Integrate context selection, compression, conversation compaction, and persistent task state with model execution. Adapt context budgets, model selection, and reasoning effort to each task.KV-cache and computation optimization: Build paged KV-cache and prompt-prefix-cache management, cache-aware routing, retention, eviction, and offloading. Preserve correctness as context, permissions, model versions, and adapters change.Inference performance: Implement and tune speculative decoding, attention kernels, continuous batching, quantization, chunked prefill, and multi-GPU parallel execution.SLM and adapter lifecycle: Build model and adapter lifecycle management, multi-adapter serving, zero-downtime hot swapping, and resource scheduling.Production reliability: Implement observability (DCGM, Prometheus, Grafana), tenant isolation, overload protection, failure recovery, and safe rollout and rollback.Cross-environment deployment: Build portable runtime integrations across NVIDIA and AMD GPU pla4orms, cloud and on-premises environments, and single- or multi-GPU configurations.Technical ExpertiseModel inference internals: Autoregressive generation, prefill vs. decoding, attention architectures, compute- vs. memory-bound execution, numerical precision.Speculative decoding: Draft-and-verify execution, acceptance/rejection sampling, adaptive speculation, and throughput tradeoffs.GPU kernels: Flash attention-style kernels, tiling, operator fusion, memory access patterns.CUDA, HIP, Triton, or CUTLASS proficiency.KV and prefix caching: Paged KV storage, fragmentation, cache quantization, offloading, cross request prefix reuse, tenant isolation.Context-window management: Token budgeting, context selection, prompt compression, conversation compaction, sliding window and sparse attention.Serving and scaling: Continuous batching, chunked prefill, admission control, quantization, multi-GPU parallelism, prefill/decode disaggregation.Qualifications Required:• 5+ years engineering, including 3+ years hands-on with GPU systems, distributed inference, or ML serving• Production experience with vLLM, SGLang, or TensorRT-LLM• Strong Python and C++ (C++17/20); ability to read and debug CUDA C++ or HIP kernels• Hands-on GPU profiling and performance optimizationPreferred:• Custom GPU kernel development (CUDA, HIP, Triton, CUTLASS)• PyTorch internals — torch. Compile, custom operators, MoE or disaggregatedprefill/decode• LoRA/SLM adapter serving and zero-downtime hot swapping• NVIDIA and AMD Instinct hardware in production• Telemetry with DCGM, AMD SMI, Prometheus, and GrafanaHiring Pay Range:$175,000 - $218,700Benefits: In addition to competitive base pay, this position also includes an annual incentive opportunity based on target achievement, pre-IPO stock options, benefits including medical, dental, vision, 401(k) with a match, and more, plus generous paid time off, paid holidays, paid day off for your birthday and other paid leave policies to support employees through all phases of life.Location preference:USA - CA - Palo AltoUniphore is an equal opportunity employer committed to diversity in the workplace. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, disability, veteran status, and other protected characteristics.For more information on how Uniphore uses AI to unify—and humanize—every enterprise experience, please visit  USA - CA - Palo AltoType: Full time

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Staff Inference Engineer in Palo Alto, CA vacancy
  • About InworldInworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We build first-party...  ...projects, open-source contributions to major inference engines, or deep-dive technical write-ups.Full-cycle ownership. You can... 
    Suggested
    Full time
    Work at office
    Relocation package

    Inworld AI

    Mountain View, CA
    3 days ago
  • Join d-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture that brings compute directly into...  ...s unique computational fabric. We are an applied research and engineering team that moves fast, ships real systems, and works directly... 
    Suggested

    d-Matrix

    Santa Clara, CA
    3 days ago
  • $175k - $265k

    Join d-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture that brings compute directly...  ...driven by execution and energized by hard problems.Senior Staff Performance Engineer, Inference CloudLocation: Santa Clara, CA / Remote, USRole... 
    Suggested
    Permanent employment
    Remote work
    Shift work

    d-Matrix

    Santa Clara, CA
    2 days ago
  •  ...Role Mission As HAI's LLM Inference Engineer, you will own the serving infrastructure that determines whether our breakthrough healthcare AI reaches patients efficiently and reliably. You'll optimize the systems that translate raw model capability into sub-100ms responses... 
    Suggested
    Full time
    Work at office

    Hippocratic%20AI

    Menlo Park, CA
    9 days ago
  • $202.5k - $274k

     ...to make that possible.Job OverviewIntuit is looking for a Staff Machine Learning Engineer to own the data and platform layer beneath our consumer...  ...data path, training and evaluation frameworks, real-time inference serving, and the handoff into our decision engine, across... 
    Suggested
    Worldwide

    Intuit

    Mountain View, CA
    3 days ago
  • $207k - $340k

     ...review of remote status and approval. We’re hiring a Principal Staff Software Engineer to lead LinkedIn’s GPU-Based Retrieval Platform, a...  ...performance tuning.3+ years of experience with NCCL, distributed inference, multi-GPU communication, or optimizing workloads on modern... 
    For contractors
    Work at office
    Remote work
    Work from home
    Flexible hours

    Linkedin

    Mountain View, CA
    1 day ago
  •  ...workflow automation with Moveworks’ Reasoning Engine and natural language capabilities, we...  ...at scale. We’re looking for a hands-on Staff Engineer who can move machine-learning models...  ...across infrastructure, data pipelines, inference services, model quality, and product... 
    Full time
    Work at office
    Immediate start
    Remote work
    Flexible hours

    ServiceNow

    Mountain View, CA
    17 days ago
  • $155k - $235k

    Join d-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture that brings compute directly...  ...are driven by execution and energized by hard problems.Staff Test & Release Engineer, Inference CloudLocation: Santa Clara, Ca / Remote,... 
    Remote work
    Shift work

    d-Matrix

    Santa Clara, CA
    2 days ago
  •  ...relentless focus on outcomes. ASAPP’s AI Engineering team is seeking an enterprising,...  ...agentic systems. We're looking for a Staff Machine Learning Engineer to own and grow...  ...experience with LLM-centric services (e.g., inference, orchestration, evaluation, monitoring)... 
    Work at office
    Immediate start

    ASAPP

    Mountain View, CA
    10 days ago
  • $251k - $310k

    Staff Machine Learning Engineer - Vision-Language Foundation Models Waymo is an autonomous driving technology company with the mission to be the world...  ...the VLM Data Flywheel: Architect the highly scalable inference and evaluation pipelines that leverage these trained Gemini... 
    Full time
    Remote work

    Waymo

    Mountain View, CA
    3 days ago
  • $218.4k - $283.9k

     ...text, and 3D primitives into interactive worlds. As our Staff Machine Learning Engineer, you will be a core technical leader bringing state-of-the...  ...deployment spanning cloud GPUs through to efficient on-device inference where the product requires it. Architecture /& Research... 
    Work at office
    Worldwide

    Unity Technologies

    Mountain View, CA
    3 days ago
  • $227k - $300k

     ...-grade AI on the Edge. We are looking for a great Senior Staff Machine Learning Engineer to join our seasoned AI team and lead the development of...  ...and decent working knowledge of modern C++ (C++14/17 for inference). Deep proficiency with PyTorch or TensorFlow, and experience... 
    Work at office
    Worldwide
    Flexible hours
    Shift work
    3 days per week

    Energy Jobline ZR

    Mountain View, CA
    3 days ago
  • $230k - $345k

    Machine Learning Engineer At AI Core, we are scaling the impact of our AI initiatives to become the primary driver of Nu's most critical...  ...critical use cases like Credit, RecSys, GenAI, and real-time inference. Address difficult and ambiguous modeling problems that... 
    Work at office
    Work from home
    Relocation package
    Flexible hours

    QED Investors

    Palo Alto, CA
    3 days ago
  • $206k - $258k

     ...member of the ML Compiler team working on software tools to enable inference of deep learning networks hardware on Rivian Hardware...  ...autonomy requirements. Qualifications Ph.D. or M.S. in Computer Engineering or a related field. Excellent C/C++ and Python programming skills... 
    Full time
    Contract work
    Temporary work
    Part time
    Local area
    Shift work

    Rivian

    Palo Alto, CA
    3 days ago
  • $298k - $368k

     ...fidelity, platform-agnostic Ground Truth (GT) engine that powers the next generation of...  ...scale offline data engineering. As a Senior Staff Machine Learning Engineer, you will serve...  ..." of high-fidelity 3D perception and ML inference at a petabyte scale. Key Responsibilities... 
    Full time
    Remote work

    Waymo

    Mountain View, CA
    3 days ago
  •  ...Pragmatic Optimism Excellence without Ego Proactive Collaboration Job Overview Mainspring is seeking Staff Level Mechanical Engineers with a focus on designing systems that require thermal and fluids analysis. In this role, you will design system architecture... 
    Local area
    Flexible hours

    Mainspring Energy

    Menlo Park, CA
    6 days ago
  • $200k - $221k

     ...with ones on vehicles that are more affordable, more enjoyable and 10-50x more efficient. ALSO is looking for a Staff Mechanical Design Engineer, Vehicle Integration to join our Vehicle Integration team, owning product architecture and layout from the earliest... 
    Local area
    Flexible hours

    ALSO

    Palo Alto, CA
    a month ago
  •  ...Pragmatic Optimism Excellence without Ego Proactive Collaboration About the Role Mainspring is seeking a Mechanical Engineer to own defined-scope mechanical engineering tasks, workstreams, or moderate-complexity projects for our Linear Generator platform... 
    Contract work
    Local area
    Flexible hours

    Mainspring Energy

    Menlo Park, CA
    28 days ago
  •  ...Pragmatic Optimism Excellence without Ego Proactive Collaboration About the Role Mainspring is seeking a Staff Mechanical Design Engineer to own the design of electrical enclosures and power electronics packaging for the Linear Generator. You’ll lead... 
    Local area
    Flexible hours

    Mainspring Energy

    Menlo Park, CA
    28 days ago
  •  ...Pragmatic Optimism Excellence without Ego Proactive Collaboration About the Role Mainspring is seeking a Senior Staff Mechanical Engineer to shape technical direction across multiple teams, systems, or major program areas within our Linear Generator platform.... 
    Local area
    Flexible hours

    Mainspring Energy

    Menlo Park, CA
    28 days ago
  • $251k - $310k

    Staff/Sr. Staff Machine Learning Engineer - World Modeling, Simulation Waymo is an autonomous driving technology company with the mission to be the world...  ...transitions. Optimize for Production & Serving: Scale inference and evaluation frameworks into production simulation... 
    Full time
    Remote work

    Waymo

    Mountain View, CA
    3 days ago
  • $251k - $310k

     ...collaborative group of machine learning (ML) engineers, software engineers, and ML research...  ...this role, you will report to a Senior Staff Engineering Manager You will: Analyze model...  ...identify bottlenecks in training and inference performance (e.g., memory bandwidth, compute... 
    Full time
    Remote work

    Latent Logic

    Mountain View, CA
    3 days ago
  • $250k - $300k

     ...consecutive years. The role / why it matters As Staff Backend Engineer, you will own the distributed systems and reliability behind...  ...for a team handling long recording sessions, compute-heavy inference and unreliable networks. You will set technical direction,... 
    Full time
    Work at office
    Relocation package
    3 days per week

    Raydar

    Mountain View, CA
    11 days ago
  • $230k - $275k

     ...system, and ship production software that thousands of candidates and every Quince hiring team depend on. You will report into Engineering and build internal products to the same bar, with the same tooling and the same peer group, as our consumer-facing systems.... 
    Live in
    Local area

    Quince

    Palo Alto, CA
    3 days ago
  • $182.19k - $231.01k

     ...Staff Engineer Duties: Support the development of new X-ray and Laser science instruments, X-ray and optical beamline systems, and help lead the development of new operational capabilities for the Linac Coherent Light Source facility. Requirements: BS in Mechanical... 
    Full time
    Temporary work
    Part time
    Fixed term contract

    Stanford

    Menlo Park, CA
    13 hours ago
  •  ...simultaneously. Progressing the ring service interval  is one of the most important development programs at Mainspring. As a Staff Mechanical Engineer on this team, you will take ownership of a significant piece of that challenge. What You’ll Do Lead ring and... 
    Local area
    Flexible hours

    Mainspring Energy

    Menlo Park, CA
    3 days ago
  • $175k - $287k

     ...jobs, sales, marketing, content, and trust platforms. As a Staff AI Software Engineer you will own end-to-end machine learning systems that run...  ...drive a meaningful efficiency or quality improvement (i.e. inference/training efficiency, engineer velocity, or tech-debt... 
    For contractors
    Work at office
    Immediate start
    Flexible hours

    Linkedin

    Mountain View, CA
    2 days ago
  • $171k

    Sr. Staff ll, AI Engineer - Search (L7-2)We exist to wow our customers. We know we’re doing the right thing when we hear our customers say,...  ...infrastructure, including vector databases, high-throughput inference, and real-time data pipelines for context enrichment and low... 
    Temporary work
    Flexible hours

    Coupang

    Mountain View, CA
    1 day ago
  •  ...ServiceNow’s leading workflow automation with Moveworks’ Reasoning Engine and natural language capabilities, we deliver the AI platform...  ...variety of responsibilities including distributed training and inference pipeline for large language models(LLM), model evaluation and... 
    Permanent employment
    Work at office
    Remote work
    Flexible hours

    ServiceNow

    Mountain View, CA
    3 days ago
  • $237k - $296k

     ...desire to protect it for future generations. Role Summary We are seeking a high-caliber Sr. Staff Design Verification Engineer to join our ADAS and Inference Silicon team. You will be responsible for the end-to-end functional verification of our next-... 
    Full time
    Contract work
    Temporary work
    Part time
    Local area
    Shift work
    Night shift

    Rivian

    Palo Alto, CA
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Staff Inference Engineer. Be the first to apply!