Software Engineer Intern, Inference
rhoda-ai
At Rhoda AI, we’re building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots. Our robots are designed to be generalists capable of operating in complex, real-world environments and handling long-tail edge cases, made possible by our cutting edge research and end-to-end system design. We've raised over $450M and are investing aggressively in model research, infrastructure, hardware development, and manufacturing scale-up to make generalist robotics a reality.
The Role
Our foundation world models have to run fast enough to control robots in the real world, and efficiently enough to support large-scale research. As an Inference Infrastructure Engineer Intern, you'll work with our infra and research engineers to make model inference faster and more efficient across the stack, from data pipelines down to GPU kernels. You'll own a scoped, high-impact project end to end, ship it into our production or research workflow, and present your results to the team.
What You'll Do
Profile and benchmark inference for large video and world models to find bottlenecks in latency, throughput, and memory.
Apply and evaluate inference optimization techniques like quantization and low-precision inference, step distillation and sampling acceleration for diffusion-style models, attention and caching optimizations across frames, model compilation, and graph-level optimizations.
Make data movement and processing more efficient: video decoding and preprocessing, input pipelines, host–device transfer, caching, and storage/IO paths that feed inference and training.
Write and tune GPU kernels (CUDA, Triton, or similar) for performance-critical operators, and integrate them into our inference stack.
Build tooling and benchmarks that let the team track performance regressions and improvements reliably.
Work with researchers to turn promising ideas into production-quality infrastructure.
What We're Looking For
You don't need all of these. Strong depth in any one of the areas below is enough to apply:
ML inference optimization: hands-on experience making deep learning models run faster or cheaper, e.g. quantization (INT8/FP8), pruning/distillation, operator fusion, mixed precision, or compilers and runtimes such as torch.compile , TensorRT, ONNX Runtime, or TVM.
Data efficiency optimization: experience speeding up data loading and preprocessing pipelines (especially video or other high-bandwidth sensor data), cutting IO and memory overhead, or improving data/compute utilization in ML systems.
GPU kernel optimization: experience writing or tuning GPU kernels in CUDA, Triton, CUTLASS, or similar; familiarity with the GPU memory hierarchy, occupancy, and profiling tools (Nsight Systems/Compute, PyTorch profiler).
Basic Qualifications
Currently pursuing a BS, MS, or PhD in Computer Science, Electrical/Computer Engineering, or a related field.
Strong programming skills in Python and/or C++.
Solid understanding of deep learning fundamentals and at least one framework (PyTorch preferred).
Comfortable reading performance data and reasoning about trade-offs between latency, throughput, memory, and accuracy.
Nice to Have
Experience with video model inference is a huge plus , e.g. optimizing video generation, video diffusion, or world models, or real-time video understanding pipelines.
Research experience in systems or ML, e.g. publications or substantial research projects (MLSys, OSDI/SOSP, CVPR/ICCV, NeurIPS/ICML/ICLR, or similar).
Experience with real-time or latency-critical inference, e.g. on robots or edge/embedded accelerators.
Experience with multi-GPU inference (tensor/sequence/pipeline parallelism).
Contributions to open-source ML systems projects.
What You'll Get
A real project with measurable impact on robots operating in the real world.
Mentorship from experienced infra and research engineers.
Access to modern GPU compute for experiments.
- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...ultra high-speed inference.About the RoleWe're hiring a Software Engineer to help contribute to projects on our Inference Platform...Suggested
$160.36k - $240.54k
...components. Develop observability to track ML model lifecycles from data generation to on-road validation.Maintain an in-house ML inference platform to serve large language models efficiently.Maintain an in-house ML compiler platform to compile, deploy, and validate Nuro...SuggestedImmediate startFlexible hours- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud... ...high-speed inference.About the RoleWe’re hiring a Senior Software Engineer for our Inference Cloud team. This team owns the cloud layer...Suggested
$21 - $40 per hour
...networking with industry experts, and developing your skills at the Fortune 1 as you explore our HQ in Northwest Arkansas! Software Engineer Intern: We're looking for Backend, Front end, and Full Stack Software Engineers to join Walmart Global Tech! As a Software...InternshipHourly payMinimum wageSummer workSummer internshipWork at officeRelocationHome office$70 per hour
...and tens of billions in simulation across 15+ U.S. states. Software Engineering builds the brains of Waymo's fully autonomous driving technology... ...better autonomous driving models and systems. Waymo interns partner with leaders in the industry on projects that create...InternshipHourly payFull timeSummer internshipRelocation package$135k - $175k
...with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, INFERENCE (AI DATA ENGINEERING) The application software team is... ...performance AI inference platform that serves the best models internally at SpaceX to accelerate our most ambitious engineering...Temporary workRemote workWorldwideWeekend work- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ....About the TeamThe Core Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure...Remote work
- ...deliver industry-leading training and inference speeds; over 10 times faster than... ...decode on the Cerebras Wafer-Scale Engine.We are hiring a Software Engineer to productionize and optimize... ...Inference Server, or an equivalent internally developed system.Strong understanding...
$70 per hour
...and tens of billions in simulation across 15+ U.S. states. Software Engineering builds the brains of Waymo's fully autonomous driving technology... ...4 autonomous driving, we'd like to meet you. Waymo interns partner with leaders in the industry on projects that create...InternshipHourly payFull timeSummer internship$70 per hour
...and tens of billions in simulation across 15+ U.S. states. Software Engineering builds the brains of Waymo's fully autonomous driving technology... ...to support ML launch evals for the Waymo Driver. Waymo interns partner with leaders in the industry on projects that create...InternshipHourly payFull timeSummer internshipRelocation package$193.3k - $261.5k
...team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into... ...optimize the low-level data-movement software across accelerators, servers,... ...spans the globe and is truly international, you’ll find yourself working side...InternshipLocal areaFlexible hours$165.2k - $223.6k
...team builds.We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into... ...optimize the low-level data-movement software that transfers KV cache and... ...workforce spans the globe and is truly international, you’ll find yourself working...InternshipLocal areaFlexible hours$10,500 per month
...Company Palantir builds the world's leading software for data-driven decisions and operations... ..., and more. The Role Software Engineers at Palantir build software at scale to... ...world use data. As a Software Engineer Intern, you'll have an opportunity to grow more...InternshipWork experience placementWork at officeRemote workWork from home$70 per hour
...and tens of billions in simulation across 15+ U.S. states. Software Engineering builds the brains of Waymo's fully autonomous driving technology... ...) that power the autonomous Waymo Driver. Waymo interns partner with leaders in the industry on projects that create...InternshipHourly payFull timeSummer internshipRelocation package- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ...with ultra-fast decode on the Cerebras Wafer-Scale Engine.We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes...
- ...deliver industry-leading training and inference speeds; over 10 times faster than... ...Infrastructure team builds the software systems that power engineering workflows across Cerebras.Our infrastructure... ....Develop deployment systems, internal tools, and self-service workflows that...
- ...field. You're drawn to AI research and want to contribute to engineering projects at the frontier of enterprise technology. Strong programming... ...mindset - you consider the user's experience when building software Strong communication skills, including the ability to...InternshipPermanent employmentFull timeFlexible hours
$200k - $375k
...scaling, batch scheduling, and global KV caching. Optimize model inference latency, throughput, concurrency, reliability, and tail latency... ...scale. Benchmark, fine-tune, and accelerate inference engines through GPU kernel work and code generation. Develop tools to...Full timeTemporary work$70 per hour
...and tens of billions in simulation across 15+ U.S. states. Software Engineering builds the brains of Waymo's fully autonomous driving technology... ...4 autonomous driving, we'd like to meet you. Waymo interns partner with leaders in the industry on projects that create...InternshipHourly payFull timeSummer internshipRelocation package$33 - $51 per hour
...Program. As a Machine Learning Research Intern on Rivian's Autonomy team, you will... ...task-driven metrics. Scale training and inference pipelines to large real-world datasets... ...related in computer science, electrical engineering, statistics, or a closely related quantitative...InternshipFull timeContract workSummer internshipLocal areaShift work$127.1k - $185k
...Services (AWS) builds the AWS Neuron SDK, the software development kit used to accelerate deep... ...learning accelerators.The Neuron Inference organization is at the forefront of optimizing... ...to the hardware-software boundary, our engineers build systematic infrastructure, develop...InternshipLocal areaFlexible hours- ...industry-leading training and inference speeds; over 10 times faster... ...RoleWe’re hiring a senior backend engineer to build and scale the... ...between customer-facing workflows, internal services, and infrastructure,... ...and operating production software.Education. Bachelor’s or master...
$165.2k - $223.6k
...builds the AWS Neuron SDK, the software development kit used to... ...learning accelerators.The Neuron Inference organization is at the forefront... ...-software boundary, our engineers build systematic infrastructure... ...findings, and learnings with internal developers and external customers...InternshipLocal areaFlexible hours$70 per hour
...and tens of billions in simulation across 15+ U.S. states. Software Engineering builds the brains of Waymo's fully autonomous driving technology... ...4 autonomous driving, we'd like to meet you. Waymo interns partner with leaders in the industry on projects that create...InternshipHourly payFull timeSummer internshipRelocation package$109.65k
...just a tool, it's your playground. Our engineers work on scalable systems, cloud services... ...the world's most beloved brands. Our SDE interns use modern technology to solve complex problems... ...version control systems Knowledge of software development lifecycle, including design,...InternshipFull timeSummer workSeasonal workSummer internshipWork at officeLocal areaWorldwideMonday to Friday$193.3k - $261.5k
We develop AWS Neuron, the complete software stack for Trainium, Amazon's custom cloud-scale machine learning accelerators. Join... ...on the Trainium hardware.As a Sr. Software Development Engineer on the Inference Model Enablement team, you will onboard and optimize state...InternshipLocal areaFlexible hours$10,500 per month
...Company Palantir builds the world's leading software for data-driven decisions and operations... ...The Role Are you passionate about engineering quality, performance, and increasing the... ...around the world use data. As an intern within Palantir's Foundations organization...InternshipWork experience placementWork at officeRemote workWork from home$165.2k - $223.6k
...(AWS) builds AWS Neuron, the software development kit used to accelerate... ...JAX enabling unparalleled ML inference and training performance.The... ...-software boundary, our engineers build systematic infrastructure... ...review, and communicate with internal and external stakeholders....InternshipWork experience placementLocal areaFlexible hours$22.1 - $42.5 per hour
...Control (MACH) division is KLA's software powerhouse, responsible for... ...scientists, software engineers, apps engineers and process engineers... ...Software Engineer (AI/ML) intern may work on a variety of tasks... ...learning model training and inference solutions for KLA products....InternshipHourly payMinimum wage$150.4k - $277.6k
Role Number: 200690248Team: Software and ServicesLocation: Cupertino; SeattlePosted: 2026-10-19SummaryApple is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute, the backbone of Siri AI and Apple Intelligence...Remote workWorldwideRelocation
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer Intern, Inference. Be the first to apply!
- senior software design engineer Mountain View, CA
- software web developer Mountain View, CA
- senior software engineer Mountain View, CA
- software developer fintech Mountain View, CA
- scientific software engineer Mountain View, CA
- new graduate software engineer Mountain View, CA
- senior software engineer ruby on rails Mountain View, CA
- intel software engineer Mountain View, CA
- graduate software developer no experience Mountain View, CA
- software developer entry level Mountain View, CA



