Staff Inference Engineer
$175k - $218.7kUniphore
Uniphore is the Business AI company. Our sovereign, composable and secure AI platform connects enterprise data, fine-tunes AI models and deploys agentic AI across the enterprise. We empower every worker to boost productivity and help businesses grow faster, operate smarter and reduce costs. Trusted by more than 2,000 businesses globally, and recognized on the Deloitte Fast 500, Uniphore delivers on the promise of AI as a transformative force for business. Job Description:Staff Inference Engineer –Uniphore is building an enterprise AI pla4orm where agents operate across voice, video, text, and structured data in real time. We are looking for a Senior or Staff Inference Engineer to own the inference layer — combining deep model inference expertise with systems engineering to deliver a context-aware runtime serving finetuned SLMs across cloud, on-premises, and widely used GPU platforms.This is a hands-on individual contributor role working closely with Uniphore’s research, model engineering, knowledge systems, agent, and pla4orm teams.What You Will Build Context-aware runtime: Integrate context selection, compression, conversation compaction, and persistent task state with model execution. Adapt context budgets, model selection, and reasoning effort to each task.KV-cache and computation optimization: Build paged KV-cache and prompt-prefix-cache management, cache-aware routing, retention, eviction, and offloading. Preserve correctness as context, permissions, model versions, and adapters change.Inference performance: Implement and tune speculative decoding, attention kernels, continuous batching, quantization, chunked prefill, and multi-GPU parallel execution.SLM and adapter lifecycle: Build model and adapter lifecycle management, multi-adapter serving, zero-downtime hot swapping, and resource scheduling.Production reliability: Implement observability (DCGM, Prometheus, Grafana), tenant isolation, overload protection, failure recovery, and safe rollout and rollback.Cross-environment deployment: Build portable runtime integrations across NVIDIA and AMD GPU pla4orms, cloud and on-premises environments, and single- or multi-GPU configurations.Technical ExpertiseModel inference internals: Autoregressive generation, prefill vs. decoding, attention architectures, compute- vs. memory-bound execution, numerical precision.Speculative decoding: Draft-and-verify execution, acceptance/rejection sampling, adaptive speculation, and throughput tradeoffs.GPU kernels: Flash attention-style kernels, tiling, operator fusion, memory access patterns.CUDA, HIP, Triton, or CUTLASS proficiency.KV and prefix caching: Paged KV storage, fragmentation, cache quantization, offloading, cross request prefix reuse, tenant isolation.Context-window management: Token budgeting, context selection, prompt compression, conversation compaction, sliding window and sparse attention.Serving and scaling: Continuous batching, chunked prefill, admission control, quantization, multi-GPU parallelism, prefill/decode disaggregation.Qualifications Required:• 5+ years engineering, including 3+ years hands-on with GPU systems, distributed inference, or ML serving• Production experience with vLLM, SGLang, or TensorRT-LLM• Strong Python and C++ (C++17/20); ability to read and debug CUDA C++ or HIP kernels• Hands-on GPU profiling and performance optimizationPreferred:• Custom GPU kernel development (CUDA, HIP, Triton, CUTLASS)• PyTorch internals — torch. Compile, custom operators, MoE or disaggregatedprefill/decode• LoRA/SLM adapter serving and zero-downtime hot swapping• NVIDIA and AMD Instinct hardware in production• Telemetry with DCGM, AMD SMI, Prometheus, and GrafanaHiring Pay Range:$175,000 - $218,700Benefits: In addition to competitive base pay, this position also includes an annual incentive opportunity based on target achievement, pre-IPO stock options, benefits including medical, dental, vision, 401(k) with a match, and more, plus generous paid time off, paid holidays, paid day off for your birthday and other paid leave policies to support employees through all phases of life.Location preference:USA - CA - Palo AltoUniphore is an equal opportunity employer committed to diversity in the workplace. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, disability, veteran status, and other protected characteristics.For more information on how Uniphore uses AI to unify—and humanize—every enterprise experience, please visit USA - CA - Palo AltoType: Full time
- About InworldInworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We build first-party... ...projects, open-source contributions to major inference engines, or deep-dive technical write-ups.Full-cycle ownership. You can...SuggestedFull timeWork at officeRelocation package
- Join d-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture that brings compute directly into... ...s unique computational fabric. We are an applied research and engineering team that moves fast, ships real systems, and works directly...Suggested
$175k - $265k
Join d-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture that brings compute directly... ...driven by execution and energized by hard problems.Senior Staff Performance Engineer, Inference CloudLocation: Santa Clara, CA / Remote, USRole...SuggestedPermanent employmentRemote workShift work- ...Role Mission As HAI's LLM Inference Engineer, you will own the serving infrastructure that determines whether our breakthrough healthcare AI reaches patients efficiently and reliably. You'll optimize the systems that translate raw model capability into sub-100ms responses...SuggestedFull timeWork at office
$202.5k - $274k
...to make that possible.Job OverviewIntuit is looking for a Staff Machine Learning Engineer to own the data and platform layer beneath our consumer... ...data path, training and evaluation frameworks, real-time inference serving, and the handoff into our decision engine, across...SuggestedWorldwide$207k - $340k
...review of remote status and approval. We’re hiring a Principal Staff Software Engineer to lead LinkedIn’s GPU-Based Retrieval Platform, a... ...performance tuning.3+ years of experience with NCCL, distributed inference, multi-GPU communication, or optimizing workloads on modern...For contractorsWork at officeRemote workWork from homeFlexible hours- ...workflow automation with Moveworks’ Reasoning Engine and natural language capabilities, we... ...at scale. We’re looking for a hands-on Staff Engineer who can move machine-learning models... ...across infrastructure, data pipelines, inference services, model quality, and product...Full timeWork at officeImmediate startRemote workFlexible hours
$155k - $235k
Join d-Matrix and help build the future of AI inference. We’ve pioneered a memory-centric architecture that brings compute directly... ...are driven by execution and energized by hard problems.Staff Test & Release Engineer, Inference CloudLocation: Santa Clara, Ca / Remote,...Remote workShift work- ...relentless focus on outcomes. ASAPP’s AI Engineering team is seeking an enterprising,... ...agentic systems. We're looking for a Staff Machine Learning Engineer to own and grow... ...experience with LLM-centric services (e.g., inference, orchestration, evaluation, monitoring)...Work at officeImmediate start
$251k - $310k
Staff Machine Learning Engineer - Vision-Language Foundation Models Waymo is an autonomous driving technology company with the mission to be the world... ...the VLM Data Flywheel: Architect the highly scalable inference and evaluation pipelines that leverage these trained Gemini...Full timeRemote work$218.4k - $283.9k
...text, and 3D primitives into interactive worlds. As our Staff Machine Learning Engineer, you will be a core technical leader bringing state-of-the... ...deployment spanning cloud GPUs through to efficient on-device inference where the product requires it. Architecture /& Research...Work at officeWorldwide$227k - $300k
...-grade AI on the Edge. We are looking for a great Senior Staff Machine Learning Engineer to join our seasoned AI team and lead the development of... ...and decent working knowledge of modern C++ (C++14/17 for inference). Deep proficiency with PyTorch or TensorFlow, and experience...Work at officeWorldwideFlexible hoursShift work3 days per week$230k - $345k
Machine Learning Engineer At AI Core, we are scaling the impact of our AI initiatives to become the primary driver of Nu's most critical... ...critical use cases like Credit, RecSys, GenAI, and real-time inference. Address difficult and ambiguous modeling problems that...Work at officeWork from homeRelocation packageFlexible hours$206k - $258k
...member of the ML Compiler team working on software tools to enable inference of deep learning networks hardware on Rivian Hardware... ...autonomy requirements. Qualifications Ph.D. or M.S. in Computer Engineering or a related field. Excellent C/C++ and Python programming skills...Full timeContract workTemporary workPart timeLocal areaShift work$298k - $368k
...fidelity, platform-agnostic Ground Truth (GT) engine that powers the next generation of... ...scale offline data engineering. As a Senior Staff Machine Learning Engineer, you will serve... ..." of high-fidelity 3D perception and ML inference at a petabyte scale. Key Responsibilities...Full timeRemote work- ...Pragmatic Optimism Excellence without Ego Proactive Collaboration Job Overview Mainspring is seeking Staff Level Mechanical Engineers with a focus on designing systems that require thermal and fluids analysis. In this role, you will design system architecture...Local areaFlexible hours
$200k - $221k
...with ones on vehicles that are more affordable, more enjoyable and 10-50x more efficient. ALSO is looking for a Staff Mechanical Design Engineer, Vehicle Integration to join our Vehicle Integration team, owning product architecture and layout from the earliest...Local areaFlexible hours- ...Pragmatic Optimism Excellence without Ego Proactive Collaboration About the Role Mainspring is seeking a Mechanical Engineer to own defined-scope mechanical engineering tasks, workstreams, or moderate-complexity projects for our Linear Generator platform...Contract workLocal areaFlexible hours
- ...Pragmatic Optimism Excellence without Ego Proactive Collaboration About the Role Mainspring is seeking a Staff Mechanical Design Engineer to own the design of electrical enclosures and power electronics packaging for the Linear Generator. You’ll lead...Local areaFlexible hours
- ...Pragmatic Optimism Excellence without Ego Proactive Collaboration About the Role Mainspring is seeking a Senior Staff Mechanical Engineer to shape technical direction across multiple teams, systems, or major program areas within our Linear Generator platform....Local areaFlexible hours
$251k - $310k
Staff/Sr. Staff Machine Learning Engineer - World Modeling, Simulation Waymo is an autonomous driving technology company with the mission to be the world... ...transitions. Optimize for Production & Serving: Scale inference and evaluation frameworks into production simulation...Full timeRemote work$251k - $310k
...collaborative group of machine learning (ML) engineers, software engineers, and ML research... ...this role, you will report to a Senior Staff Engineering Manager You will: Analyze model... ...identify bottlenecks in training and inference performance (e.g., memory bandwidth, compute...Full timeRemote work$250k - $300k
...consecutive years. The role / why it matters As Staff Backend Engineer, you will own the distributed systems and reliability behind... ...for a team handling long recording sessions, compute-heavy inference and unreliable networks. You will set technical direction,...Full timeWork at officeRelocation package3 days per week$230k - $275k
...system, and ship production software that thousands of candidates and every Quince hiring team depend on. You will report into Engineering and build internal products to the same bar, with the same tooling and the same peer group, as our consumer-facing systems....Live inLocal area$182.19k - $231.01k
...Staff Engineer Duties: Support the development of new X-ray and Laser science instruments, X-ray and optical beamline systems, and help lead the development of new operational capabilities for the Linac Coherent Light Source facility. Requirements: BS in Mechanical...Full timeTemporary workPart timeFixed term contract- ...simultaneously. Progressing the ring service interval is one of the most important development programs at Mainspring. As a Staff Mechanical Engineer on this team, you will take ownership of a significant piece of that challenge. What You’ll Do Lead ring and...Local areaFlexible hours
$175k - $287k
...jobs, sales, marketing, content, and trust platforms. As a Staff AI Software Engineer you will own end-to-end machine learning systems that run... ...drive a meaningful efficiency or quality improvement (i.e. inference/training efficiency, engineer velocity, or tech-debt...For contractorsWork at officeImmediate startFlexible hours$171k
Sr. Staff ll, AI Engineer - Search (L7-2)We exist to wow our customers. We know we’re doing the right thing when we hear our customers say,... ...infrastructure, including vector databases, high-throughput inference, and real-time data pipelines for context enrichment and low...Temporary workFlexible hours- ...ServiceNow’s leading workflow automation with Moveworks’ Reasoning Engine and natural language capabilities, we deliver the AI platform... ...variety of responsibilities including distributed training and inference pipeline for large language models(LLM), model evaluation and...Permanent employmentWork at officeRemote workFlexible hours
$237k - $296k
...desire to protect it for future generations. Role Summary We are seeking a high-caliber Sr. Staff Design Verification Engineer to join our ADAS and Inference Silicon team. You will be responsible for the end-to-end functional verification of our next-...Full timeContract workTemporary workPart timeLocal areaShift workNight shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Inference Engineer. Be the first to apply!
- engineering aide Palo Alto, CA
- technology administrator Palo Alto, CA
- staff engineer Palo Alto, CA
- senior staff engineer Palo Alto, CA
- assistant engineer Palo Alto, CA
- senior staff systems engineer Palo Alto, CA
- software engineer staff Palo Alto, CA
- assistant computer programmer
- assistant system engineer
- staff design engineer




