Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Member of Technical Staff — Inference-Multi-Hardware

$200k - $400k
Full-time

radixark

About the Role

RadixArk is seeking a Member of Technical Staff - Inference-Multi-Hardware to push the limits of performance for frontier AI systems.

Most performance engineering assumes a single vendor's stack. This role assumes none. You'll bring up, optimize, and maintain SGLang, Miles, and the RadixArk infrastructure stack across NVIDIA and AMD GPUs, Google TPUs, modern server CPUs, and a growing set of emerging AI accelerators. That means porting kernels and runtimes onto unfamiliar hardware, designing the abstractions that keep one codebase fast on all of it. You will be working directly with silicon and our partners, often on pre-release platforms with immature tooling.

This is one of the broadest technical roles at RadixArk. The problem changes shape with every new platform: a memory hierarchy that punishes your last set of assumptions, a compiler that fuses differently, a collective library that doesn't exist yet. We're looking for engineers who find that appealing rather than exhausting, and who can go deep on a new architecture fast without losing the performance instincts they built on the last one.

Requirements

  • 4+ years of experience in systems, performance, or ML infrastructure engineering

  • Deep expertise in at least one accelerator programming model (CUDA, ROCm/HIP, Pallas/XLA, Triton, or a vendor SDK), with demonstrated ability to pick up new ones quickly.

  • Strong understanding of accelerator architecture: memory hierarchy, bandwidth limits, occupancy, and the tradeoffs between them

  • Experience writing or optimizing high-performance kernels for ML workloads

  • Experience with distributed execution and communication libraries (NCCL, RCCL, MPI, or equivalents)

  • Proficiency in C++ and Python

  • Strong debugging and profiling skills at the system level, including on platforms where the tooling is incomplete or unreliable

  • Track record of performance work that shipped into production

Strong Plus

  • Experience bringing up ML workloads on new silicon.

  • Hands-on depth in more than one vendor ecosystem

  • Experience with compiler stacks (XLA, MLIR, TVM, Triton) or building compiler passes and IR transformations

  • Experience designing hardware abstraction layers or portable kernel interfaces

  • Quantization and mixed-precision work across differing numeric formats and hardware support levels

  • Experience with distributed inference systems (SGLang, vLLM) or training/RL frameworks (Miles, Megatron, veRL, TorchTitan)

  • CPU inference optimization (AVX-512/AMX, oneDNN, NUMA-aware execution)

  • Experience optimizing collective communication at scale, or scaling workloads to 1000+ accelerators

  • Contributions to kernel, compiler, or ML systems open source

  • Direct collaboration with silicon vendors or cloud partners on technical evaluations

  • Background in HPC or other performance-critical systems

Responsibilities

  • Bring up RadixArk's inference and training systems on new accelerator platforms and drive them to competitive performance

  • Design hardware abstractions that let a single codebase stay fast across vendors without forking

  • Port and optimize kernels across programming models and memory architectures

  • Build cross-platform benchmarking, profiling, and regression detection so performance claims hold up on every target

  • Debug numerical divergence and correctness gaps between platforms

  • Work with vendor engineering teams on pre-release hardware, compiler and driver issues, and roadmap feedback

  • Partner with kernel, runtime, distributed systems, and product engineers to land performance wins end to end

  • Serve as the internal source of truth on what each platform is actually good at

  • Contribute hardware-specific optimizations, benchmarks, and portability work back to open-source SGLang and Miles

About RadixArk

RadixArk is an infrastructure-first company built by engineers who've shipped production AI systems, created SGLang (30K+ GitHub stars, the fastest open LLM serving engine), and developed Miles (our large-scale RL framework). We're on a mission to democratize frontier-level AI infrastructure by building world-class open systems for inference and training. Our team has optimized kernels serving billions of tokens daily, designed distributed training systems coordinating 10,000+ GPUs, and contributed to infrastructure that powers leading AI companies and research labs. We're backed by well-known infrastructure investors and partner with Nvidia, Google, and frontier AI labs. Join us in building infrastructure that gives real leverage back to the AI community.

Compensation

Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity.

Equal Opportunity

RadixArk is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Member of Technical Staff — Inference-Multi-Hardware in Palo Alto, CA vacancy
  • $200k - $400k

     ...Role RadixArk is looking for a Member of Technical Staff Cluster Infrastructure to...  ...powers frontier-level AI training and inference. You will design and operate highly...  ...~ Experience debugging complex multi-layer issues across hardware, OS, networking, and distributed... 
    Suggested
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff — Inference to push the limits of large-scale AI inference. You will work on the...  ...role for engineers who enjoy working close to the hardware–software boundary and solving performance-critical problems... 
    Suggested
    Full time
    Worldwide

    radixark

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...About the Role RadixArk is looking for a Member of Technical Staff — TPU Systems to build high-performance inference and training systems using JAX, XLA, and Pallas...  ...ll push model workloads to their limits on TPU hardware, working on SGLang-JAX and other critical infrastructure... 
    Suggested
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff — Kernel / Compiler / Communication...  ...is critical to scaling training and inference across thousands of GPUs, where microseconds...  ...engineers who enjoy working close to hardware and solving performance problems that... 
    Suggested
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...Role RadixArk is seeking a Member of Technical Staff, Developer Technology (DevTech) to make LLM inference and training dramatically...  ...more accessible on modern GPU hardware. Our systems sit at the center...  ...from kernels to distributed multi-node systems. Go deep in one... 
    Suggested
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...About the Role RadixArk is seeking a Member of Technical Staff — Inference-Multimodal & Diffusion to advance the frontier of generative modeling. You will work on cutting-edge diffusion and flow-based models for image, video, and multimodal generation, pushing model... 
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...the Role RadixArk is looking for a Member of Technical Staff — Backend/API Platform Engineer to build...  .../gRPC APIs, authentication systems, multi-tenancy isolation, and monitoring infrastructure...  .... This role bridges high-performance inference/training systems with production-grade... 
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...About the Role As a Member of Technical Staff, Training, you will design,...  ...training — spanning training, inference, and orchestration, with a...  ...multimodal post-training across multi-GPU, multi-node...  ...latency, memory efficiency, hardware utilization, and cost. Investigate... 
    Full time
    H1b
    Visa sponsorship
    Work visa

    radixark

    Palo Alto, CA
    3 days ago
  •  ...abstraction. The Role  As a Member of Technical Staff, AI Training Platform, you...  ...neural networks to novel hardware and push the boundaries of...  .... Design and scale multi-node distributed training systems...  ...and performance tuning, LLM inference, CUDA, Triton, NCCL, vLLM,... 
    Work at office
    Flexible hours

    Unconventional, Inc.

    Mountain View, CA
    4 hours ago
  • $200k - $400k

     ...About the Role RadixArk is hiring a Member of Technical Staff — CI / Infrastructure to own the...  ...across NVIDIA, AMD, Intel, and Ascend hardware pools, gating every commit to one of the fastest-growing open-source LLM inference engines. When CI is green and fast, 10... 
    Full time
    Night shift

    radixark

    Palo Alto, CA
    3 days ago
  •  ...reinventing chip development and closing the loop between AI and the hardware that fuels it, recursively accelerating the path to artificial...  .... Shipped RL infrastructure, such as async rollouts and inference engines like vLLM or SGLang. Experience navigating... 
    Full time

    Ricursive%20Intelligence

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...About the Role RadixArk is hiring a Member of Technical Staff — Performance in Palo Alto, CA — someone who can push LLM inference and training systems to the limit across real...  ...performance issues across software, hardware, and infrastructure layers Strong communication... 
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  •  ...Sydney, and Tokyo. About the Role At ThoughtSpot, we're building the future of analytics with AI at the core. As a Member of Technical Staff , you'll join a team of world-class engineers working on technologies that power our next-generation analytics platform.... 
    Internship
    Work at office
    Local area
    3 days per week

    ThoughtSpot

    Mountain View, CA
    1 day ago
  •  ...stack from high-performance hardware and robot systems to the infrastructure...  ...levels — from senior/MTS to staff. This is a customer-facing...  ...development Communicate technical findings clearly to both...  ...agriculture) Familiarity with inference optimization and runtime... 
    Full time

    rhoda-ai

    Mountain View, CA
    3 days ago
  •  ...move faster; from real-time alerts to seamless clip sharing and multi-site visibility. You’ll be joining an ambitious, fast-moving...  ...architecture patterns, maintain testing standards, and mentor team members. Skills and qualifications: Experience building production... 

    Coram AI

    Sunnyvale, CA
    5 days ago
  • $500 per month

     ...someone, and when, is the difference between a session that lasts two minutes and one that lasts two hours. As our RecSys founding member, you'll own this problem end-to-end - set the architecture, build the foundation, and grow it from rule-based systems to deep learning... 
    Full time
    Work at office
    Flexible hours

    astrocade

    Palo Alto, CA
    3 days ago
  •  ...providers, system companies and advanced fabs to tape-out custom hardware co-designed for their next-generation AI workloads. Our...  ...most complex SoCs in the world. What You'll Do As a Member of Technical Staff - Formal Verification Engineer at Architect Labs, you'll... 
    Full time

    architect

    Palo Alto, CA
    3 days ago
  •  ...most complex SoCs in the world. What You'll Do As a Member of the Technical Staff - Formal Methods at Architect Labs, you'll work on the...  ...gives our system the rigor required to take AI-generated hardware artifacts into verified production-grade silicon. This is... 
    Full time

    architect

    Palo Alto, CA
    3 days ago
  • $200k - $400k

     ...experienced product-focused engineers to join our team in building the developer-facing surfaces of our inference and training infrastructure. As a Member of Technical Staff — Product, you will partner closely with our Product Managers, research scientists, and systems... 
    Full time

    radixark

    Palo Alto, CA
    3 days ago
  •  ...most complex SoCs in the world. What You'll Do As a Member of the Technical Staff on the Design Verification & Validation team at Architect...  ...foundries. You'll work at the intersection of rigorous hardware verification and AI-driven design automation. Review AI... 
    Full time
    Night shift

    architect

    Palo Alto, CA
    3 days ago
  • $190k - $210k

     ...Member of Technical Staff, Software Engineering, Mosaic Reporting Cognita Imaging Level: Senior / Staff Type: Full-time Location...  ...training, evaluation, and infrastructure teams to integrate inference into production reporting workflows and improve product... 
    Full time
    Remote work

    cognita-imaging

    Palo Alto, CA
    3 days ago
  •  ...Architect, you will spend 3 months working alongside the founding team to push the boundaries of how AI models explore and optimize hardware designs. This is a high-impact role where your experiments will directly influence our core modeling roadmap. Responsible for... 
    Full time
    Internship

    architect

    Palo Alto, CA
    3 days ago
  •  ...the world. What You'll Do As a Member of the Technical Staff at Architect Labs, you'll sit at the...  ...design and frontier AI translating deep hardware engineering expertise into agentic...  ...Designing and deploying production multi-agent workflows and agents that run autonomously... 
    Full time

    architect

    Palo Alto, CA
    2 days ago
  •  ...most complex SoCs in the world. What You’ll Do As Member of the Technical Staff - Software at Architect, you’ll build the core platform...  ...your fingerprints on every layer of a system redefining how hardware gets built. What We’d Like to See Education: BS/... 
    Full time

    architect

    Palo Alto, CA
    3 days ago
  •  ...architecture, RTL, and our AI system. Lead technical partnerships: Resolve process, library,...  ...in hierarchical implementation, multi-mode multi-corner timing analysis, variation...  ...explain implementation trade-offs to hardware, software, and AI teams. Bonus: Direct... 
    Full time

    architect

    Palo Alto, CA
    3 days ago
  •  ...About the Role We are seeking a Member of Technical Staff, Functional and Performance Modeling...  ...and co-exploring the design space as hardware and software evolve together. This...  ...workload perspective: large language model inference and sharding, KV caching, serving... 
    Full time
    Work at office
    Immediate start
    Visa sponsorship
    Relocation package
    Night shift

    netpreme

    Santa Clara, CA
    3 days ago
  • $148.5k - $223.9k

     ...systems with customers. With your strong technical competence, strategic thinking and...  ...workflows (tool use, planning, memory, multi-step reasoning) Experience building end...  ...debugging model training, evaluation, and inference pipelines Infrastructure & Deployment... 
    Full time

    Salesforce

    Palo Alto, CA
    3 days ago
  •  ...About the Role As a Member of Technical Staff [Research] at NeoCognition , you’ll be part of the core team advancing the frontier of LLM...  ...opportunities. Stay abreast of emerging work in reasoning, multi-agent systems, RLHF, tool use, and LLM fine-tuning — and... 
    Full time

    neocognition

    Palo Alto, CA
    3 days ago
  •  ...most complex SoCs in the world. What You'll Do As a Member of the Technical Staff (Applied AI) at Architect, you'll sit at the intersection of chip design and frontier AI — translating deep hardware engineering expertise into agentic systems that can reason about... 
    Full time

    architect

    Palo Alto, CA
    3 days ago
  • $144.36k - $199.39k

     ...must meet, and auditing that output before it reaches a faculty member or a publication. The successful candidate will have conceived,...  ...Teamwork: Provide guidance and direction to more junior research staff and set the working standards for the lab's hybrid human-and-AI-... 

    Stanford University

    Palo Alto, CA
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Member of Technical Staff — Inference-Multi-Hardware. Be the first to apply!