Staff / Principal ML Systems Engineer - Frontier Model Training Infrastructure

Neolix US is the Silicon Valley Physical AI research team of Neolix, the global leader in L4 autonomous delivery. Neolix operates 30,000+ autonomous logistics vehicles in 300+ cities across 15+ countries, with ~200 million autonomous kilometers driven on public roads. About the role

Build the systems that make ambitious Physical AI experiments possible. Models that learn from video, multimodal driving data and robot interaction demand fast, repeatable and economically disciplined experimentation. As our ML Systems owner, you will build that foundation: a training platform that shortens the full cycle from a legally usable physical-world dataset to a reproducible, validated action model, across compute, storage, networking, failure recovery and evaluation. This is not a generic IT, cluster procurement or DevOps-only role. Success means lower time and cost per validated experiment, not a bigger hardware count. Must-have: hands-on experience at a leading autonomous-driving company

(e.g., Waymo, Tesla Autopilot, Wayve, Momenta, XPeng, Cruise or Nuro). General tech experience alone does not qualify. What you'll own

Training platform architecture: lead selected global training-infrastructure programs and design multi-node training execution, orchestration and operational observability Performance engineering: profile compute, memory, communication, interconnects, I/O, mixed precision and accelerator utilization, and implement measured fixes Reliable experimentation: robust checkpoint/restart, job isolation, model and data provenance, experiment reproducibility and failure diagnosis Multimodal data paths: storage performance, sharding, streaming, cache hierarchy and fast loaders for video and sensor data at training time Responsible distributed execution: support regional access and permitted data-processing boundaries, working with security and legal teams Research partnership: co-design benchmarks with the end-to-end driving, Data Recipe and Data Flywheel owners; support pre-training, post-training and closed-loop training of a video-in, action-out VLA driving model; and lead hard systems debugging Qualifications

Required: experience at a leading autonomous-driving company; general tech experience alone does not qualify Proven ownership of production distributed ML training infrastructure, high-scale training runtimes or demanding model-training performance systems Strong Python and/or C++, with working knowledge of PyTorch/JAX execution, distributed training, data loaders, memory pressure and communications A track record of concrete performance profiling, failure root-cause analysis, resilience engineering and measured throughput or cost improvements Strong foundations in large-scale storage and data movement, scheduling, concurrency and system-level reliability Ability to architect from first principles and personally debug bottlenecks across framework, network, storage and accelerators Preferred

FSDP, DeepSpeed, Megatron, Ray, Slurm/Kubernetes, distributed PyTorch, XLA/JAX or equivalent optimized production training stacks CUDA/Triton kernels, collective communication tuning, FP8/mixed precision, compiler optimization and GPU/TPU/heterogeneous compute High-throughput video and robot datasets, distributed indexes, object/parallel storage, data locality and tiered caches Equity: generous founding-team equity package Benefits: premium health, dental and vision insurance, 401(k), and comprehensive Silicon Valley tech perks You will help define the global training stack for physical action models, making frontier learning reproducible and economical enough to inform real deployment. You will have meaningful autonomy, collaborate with Neolix's global engineering teams, and may pursue approved academic publications or open-source contributions.

#J-18808-Ljbffr