Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Member of Technical Staff, Inference Systems

$230k - $350k

Transparent Search Group

Job Description

Job Description

Member of Technical Staff, Inference Systems

Company: Confidential — founded by Stanford alumni with deep AI infrastructure experience
Location: Palo Alto, CA (on-site 5 days per week)
Compensation: $230,000 - $350,000 + competitive equity
Employment Type: Full-time
Visa Sponsorship: Visa transfers and new sponsorship (H-1B, TN)

About the Company

The client is building a next-generation AI inference platform from the ground up with a relentless focus on performance. It was founded by Stanford alumni with deep AI infrastructure experience, including early work at Together AI, and is engineering the entire inference stack with Rust at its core.

The client has raised a $10M seed from notable investors and is currently in stealth ahead of announcing its fundraise and product.

The Role

The client is hiring Members of Technical Staff (2+ years) to build a high-performance inference platform from scratch. You are a systems engineer who knows inference internals (attention, KV cache, batching, scheduling) and wants to own the whole stack rather than a narrow slice. You will join a small team building a new inference system in Rust, where you shape every design decision.

What You Will Do
  • Build a new inference runtime in Rust, owning batching, scheduling, request routing and the serving stack.
  • Design KV cache management, prefix caching and optimizations that reduce latency and cost per token.
  • Scale serving across GPUs and nodes.
  • Profile, benchmark and ship performance improvements across the inference pipeline.
  • Make core architecture decisions with the founding team.
What You Bring
  • 2+ years of systems engineering experience
  • Deep knowledge of inference internals: attention, KV cache, batching and scheduling
  • Experience inside inference engines such as vLLM, SGLang or TensorRT-LLM
  • Strong systems programming (Rust, C++ or similar)
  • Ability to work on-site in Palo Alto 5 days a week
Tech Stack

Rust, Python, PyTorch, C++, Go, vLLM, SGLang, TensorRT-LLM, CUDA, Triton, NCCL

Before you apply: get a free recruiter-readiness score for your resume at CareerStanding — about a minute, no signup, and your resume stays in your browser.

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Member of Technical Staff, Inference Systems in Palo Alto, CA vacancy
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff - Inference-Multi-Hardware to push the limits of performance for frontier AI systems. Most performance engineering assumes a single vendor's stack. This role assumes none. You'll bring up, optimize,... 
    Suggested
    Full time

    radixark

    Palo Alto, CA
    6 days ago
  • $230k - $350k

     ...Job Description Job Description About the Role Join a small AI systems team building a high-performance inference platform from the ground up. You will shape the runtime architecture and improve how large language models are served across GPUs and nodes. What... 
    Suggested
    Visa sponsorship

    Wintermeyer Ventures

    Palo Alto, CA
    3 days ago
  • $180k - $230k

     ...Job Description Job Description Member of Technical Staff, ML Systems Company: Confidential — venture-backed technology company Location:...  ...About the Company The client builds the training and inference stack for world models. Today's stack was built for language... 
    Suggested
    Full time
    H1b
    Work at office
    Visa sponsorship

    Transparent Search Group

    Menlo Park, CA
    3 days ago
  • $200k - $400k

     ...About the Role RadixArk is looking for a Member of Technical Staff — TPU Systems to build high-performance inference and training systems using JAX, XLA, and Pallas. You'll push model workloads to their limits on TPU hardware, working on SGLang-JAX and other critical... 
    Suggested
    Full time

    radixark

    Palo Alto, CA
    5 days ago
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff — Inference-Multimodal & Diffusion to advance the frontier of generative modeling....  ...work will directly shape next-generation generative AI systems used by researchers, developers, and real-world applications... 
    Suggested
    Full time

    radixark

    Palo Alto, CA
    5 days ago
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff — Inference to push the limits of large-scale AI inference. You will work on the core systems that serve frontier models at scale, optimizing performance, latency, throughput, and cost across thousands... 
    Full time
    Worldwide

    radixark

    Palo Alto, CA
    5 days ago
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff — Kernel / Compiler / Communication...  ...of performance for frontier AI systems. You will work at the lowest layers...  ...is critical to scaling training and inference across thousands of GPUs, where microseconds... 
    Full time

    radixark

    Palo Alto, CA
    5 days ago
  • $180k - $250k

     ...grew out of years of work inside Stanford Medicine, alongside world-class researchers and clinicians, and we now work with health systems and data partners on large longitudinal datasets: claims, health records, imaging and clinical notes. We make that data usable for... 
    Full time

    genmd

    Palo Alto, CA
    6 days ago
  • $200k - $400k

     ...the Role RadixArk is looking for a Member of Technical Staff Cluster Infrastructure to architect...  ...powers frontier-level AI training and inference. You will design and operate highly...  ...scheduling and resource management systems, and push the limits of large-scale distributed... 
    Full time

    radixark

    Palo Alto, CA
    6 days ago
  •  ...adversary that never sleeps. What You'll Do Design Core Systems: Architect the services, pipelines, and storage engines powering...  ...large-scale storage. Infrastructure for AI/ML workloads (inference, model serving, data systems). Enterprise products built for... 
    Full time
    Work at office
    Flexible hours

    armadin

    Palo Alto, CA
    5 days ago
  • $200k - $400k

     ...the Role RadixArk is looking for a Member of Technical Staff — Backend/API Platform Engineer to...  ...implement the REST/gRPC APIs, authentication systems, multi-tenancy isolation, and...  .... This role bridges high-performance inference/training systems with production-grade... 
    Full time

    radixark

    Palo Alto, CA
    6 days ago
  •  ...Job Description Job Description Member of Technical Staff - Biological Engineering, ML, or Lab System About the role Acelegen combines biology, AI, and laboratory automation to accelerate scientific discovery and drug development. Our platform connects protein... 

    Acelegen

    Menlo Park, CA
    8 days ago
  •  ...building the future of analytics with AI at the core. As a Member of Technical Staff , you'll join a team of world-class engineers working on...  ...power our next-generation analytics platform. From distributed systems and high-performance backend services to AI-powered search... 
    Internship
    Work at office
    Local area
    3 days per week

    ThoughtSpot

    Mountain View, CA
    4 days ago
  •  ...the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world...  ...up to make generalist robotics a reality. We're looking for a Staff / Principal ML Training Systems Engineer to own training systems... 
    Full time

    rhoda-ai

    Mountain View, CA
    6 days ago
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff, Developer Technology (DevTech) to make LLM inference and training dramatically faster, cheaper, and more accessible on modern GPU hardware. Our systems sit at the center of how modern AI is served and... 
    Full time

    radixark

    Palo Alto, CA
    6 days ago
  • $500 per month

     ...Come build it with us. About the Role Our recommendation system doesn't exist yet, and it needs to. We have a large and fast-...  ...two minutes and one that lasts two hours. As our RecSys founding member, you'll own this problem end-to-end - set the architecture, build... 
    Full time
    Work at office
    Flexible hours

    astrocade

    Palo Alto, CA
    6 days ago
  • $200k - $400k

     ...About the Role RadixArk is hiring a Member of Technical Staff — CI / Infrastructure to own the...  ...infrastructure that keeps SGLang moving. Our CI system runs 300+ GPU tests across NVIDIA,...  ...the fastest-growing open-source LLM inference engines. When CI is green and fast, 1... 
    Full time
    Night shift

    radixark

    Palo Alto, CA
    5 days ago
  • $200k - $400k

     ...team in building the developer-facing surfaces of our inference and training infrastructure. As a Member of Technical Staff — Product, you will partner closely with our Product Managers, research scientists, and systems engineers to turn cutting-edge AI infrastructure... 
    Full time

    radixark

    Palo Alto, CA
    5 days ago
  • $190k - $210k

     ...Member of Technical Staff, Software Engineering, Mosaic Reporting Cognita Imaging Level: Senior...  ..., you will build the full-stack systems behind Mosaic Reporting, Cognita's AI...  ...and infrastructure teams to integrate inference into production reporting workflows and... 
    Full time
    Remote work

    cognita-imaging

    Palo Alto, CA
    6 days ago
  •  ...Ricursive Intelligence is a frontier AI lab building self-improving systems, starting with chip design. We are reinventing chip development...  .... Shipped RL infrastructure, such as async rollouts and inference engines like vLLM or SGLang. Experience navigating ambiguity... 
    Full time

    Ricursive%20Intelligence

    Palo Alto, CA
    6 days ago
  • $200k - $400k

     ...About the Role As a Member of Technical Staff, Training, you will design, build, and operate the distributed systems behind large-scale model post-training — spanning training, inference, and orchestration, with a focus on the performance, correctness, scalability... 
    Full time
    H1b
    Visa sponsorship
    Work visa

    radixark

    Palo Alto, CA
    5 days ago
  •  ...inefficient abstraction. The Role  As a Member of Technical Staff, AI Training Platform, you will be a...  ...multi-node distributed training systems, implementing elastic sharding and...  ...kernels and performance tuning, LLM inference, CUDA, Triton, NCCL, vLLM, SGLang, VeRL... 
    Work at office
    Flexible hours

    Unconventional, Inc.

    Mountain View, CA
    2 days ago
  • $200k - $400k

     ...About the Role RadixArk is hiring a Member of Technical Staff — Performance in Palo Alto, CA — someone who can push LLM inference and training systems to the limit across real production workloads. You’ll work on the performance-critical path of SGLang, Miles, and... 
    Full time

    radixark

    Palo Alto, CA
    5 days ago
  • $144.36k - $199.39k

     ...that output before it reaches a faculty member or a publication. The successful candidate...  ...an expert who directs and certifies AI systems that do the work. Core Duties Research...  ...guidance and direction to more junior research staff and set the working standards for the lab... 

    Stanford University

    Palo Alto, CA
    2 days ago
  •  ...business value. Mentor and coach junior researchers by providing technical guidance, rigorous code reviews, and career development support...  ...(algebra, analysis, combinatorics) and formal proof systems (Lean, Coq, Isabelle). Demonstrated research acumen: experiment... 

    Axiom Global

    Palo Alto, CA
    5 days ago
  •  ...Platform team, you'll help build the orchestration and reasoning systems that let our security agents operate independently at scale —...  ...in Computer Science, Software Engineering, Math, or a related technical field, with an expected graduation in 2027 or 2028. Comfortable... 
    Full time
    Internship
    Summer internship
    Work at office
    Flexible hours

    armadin

    Palo Alto, CA
    5 days ago
  •  ...APIs) Own projects full lifecycle Design discussions & technical scoping Implementation & testing Post-launch...  ...ideas to intuitive, performant, scalable UX Contribute to system architecture - ensure reliable, maintainable, scalable stack... 
    Work at office
    Remote work
    Visa sponsorship
    Monday to Friday

    Sierra Ventures

    Palo Alto, CA
    5 days ago
  •  ...performance hardware and robot systems to the infrastructure and...  ...levels — from senior/MTS to staff. This is a customer-facing role...  ...model development Communicate technical findings clearly to both...  ...agriculture) Familiarity with inference optimization and runtime... 
    Full time

    rhoda-ai

    Mountain View, CA
    6 days ago
  • $200k - $400k

     ...Developer Advocate to build and engage our technical community around SGLang, Miles, and our...  ...to help them succeed with our systems. This role combines deep technical expertise...  ...by building world-class open systems for inference and training. Our team has optimized kernels... 
    Full time

    radixark

    Palo Alto, CA
    6 days ago
  •  ...architecture patterns, maintain testing standards, and mentor team members. Skills and qualifications: Experience building...  ...AI-native platform turns every camera and sensor into a smart system that enhances safety, efficiency, and awareness. Founded by... 

    Coram AI

    Sunnyvale, CA
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Member of Technical Staff, Inference Systems. Be the first to apply!