On-Device LLM Inference Engineer — Real-Time, Rust & GPU
Mirai Labs
Mirai Labs in San Francisco seeks engineers to join a senior team building the full on-device stack for real-time local intelligence. You will primarily work on uzu, our inference engine, and focus on supporting new modalities and a variety of features. The ideal candidates should have a deep understanding of how computers and modern language models work, and experience in writing high-performance GPU kernels or Rust systems programming. We welcome applications from talented students and early-career engineers. #J-18808-Ljbffr Mirai Labs
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the On-Device LLM Inference Engineer — Real-Time, Rust & GPU in San Francisco, CA vacancy
$160k - $230k
...efficient and scalable inference for large language models... ...and Optimization Engineer to design, develop, and... ...‑throughput inference, GPU/accelerator optimizations... ...to shape the future of LLM inference infrastructure... ...salary range for this full‑time position is: $160,000 -...SuggestedFull time- ...progress of AI applications out into the real world. With Anyscale, were building the... ...date. About the role As a Distributed LLM Inference Engineer, you will help with systems and... ...consistent. As the market data changes over time, the target salary for this role may be...SuggestedWork at office
- Vast.ai is seeking a systems engineer to scale AI inference and optimize GPU performance at our San Francisco or Los Angeles offices. You will leverage your... ...record in high-performance systems. This role is full-time and on-site, offering equity and a fast-paced startup...SuggestedFull time
$220k
Perplexity is looking for an engineer to join their team in San Francisco. You will work on building and operating the inference engine, supporting new models, migrating GPU kernels, and developing a Rust-based serving runtime. The ideal candidate has 3+ years of experience...Suggested- ...ambiguous, with meaningful real-world constraints and... ...looking for a Core Systems Engineer with a deep mastery of Rust to design, build, and optimize... ...infrastructure and real-time performance engines. This... ...data and compute complex inference logic with ultra-low latency...Suggested
$200k - $400k
...startup in San Francisco, seeks a Senior Engineer to tackle challenging problems in AI. You’... ...design custom database systems, implement LLM token management, and develop Unix OS distributions... ...have strong skills in TypeScript, Rust, or Python and a passion for solving complex...Visa sponsorship- ...shipping excellence. We seek engineers with strong intrinsic... ...to help scale AI inference. You’ll leverage your knowledge... ...systems to optimize GPU performance at the bleeding edge of AI. Full-Time On-site at either our... ...(virtual, 30 minutes) LLM-assisted coding...Full timeWork at office
- ...platforms. You will design and integrate control systems, working on real hardware alongside a small, dedicated team. Applicants should... ...a strong background in robotics with hands-on experience in real-time control system design. The position offers competitive salary, meaningful...Relocation package
- Gravity Engineering Services Pvt Ltd. is looking for a Distributed LLM Inference Engineer to join their team. This critical role focuses on enhancing performance for ML inference, ensuring scalability and efficiency in solutions used by both open-source and corporate clients...
$225k
...RL, ultra‑long context, and inference‑time compute to achieve this goal.... ...About The Role As a Software Engineer on the Inference & RL Systems... ...and sustained throughput under real‑world workloads. You will own... ...bottlenecks across GPU, networking, and storage layers...RelocationVisa sponsorship- Vast.ai Inc. is seeking a systems engineer with HPC or parallel programming experience to help scale AI inference. You will design and optimize GPU kernels and tensor libraries, leveraging CUDA/C++ and related frameworks to push the bleeding edge of AI performance. This...
- ...technology company in San Francisco is seeking a Senior Software Engineer for Backend (Systems / Infrastructure). You will architect and... ...scalability as demand grows. This role involves optimizing APIs, managing GPU workloads, and collaborating with cross-functional teams. Ideal...
- AI Systems Engineer - Codex Core Agents Location... ...Type Full time Department... ...agents operating in real development environments... ...model behavior, inference/runtime issues,... ...across layers: Rust systems code,... ...experience with LLM applications,... ...inference optimization, GPU systems,...Full timeWork at officeLocal areaRelocation packageFlexible hours
- ...is building a new class of robotics-powered products for the supply chain market. The Senior Software Engineer will own firmware development and contribute to real-time embedded systems critical to the robot fleet's performance. You will collaborate across hardware and...
- Research Scientist/Engineer - Post-training, Inference, & Safety and Security Virtue AI Full-time Unknown About Virtue AI Virtue AI sets... ...unifies automated red‑teaming, real‑time multimodal guardrails, and... ..., along with expertise in LLM libraries like PyTorch, HuggingFace...Full time
- ...pioneering technology company in San Francisco is seeking a firmware engineer to deliver high-quality code for embedded systems. Candidates... ...have a strong engineering background and proficiency in C, C++, Rust, and Verilog, with a preference for those with 3+ years of...
- ...where AI agents hang out with real people in group chats, DMs, voice... ...Staff is the title we use for engineers who own hard problems end to... ...messages Serving low-latency LLM responses for group chats with... ...training, fine-tuning, evaluation, inference, or RAG at scale High-...
- ...Building a new kind of platform for real-time generative media, enabling... ...unicorn founders and senior engineers with deep expertise in 3D,... ...for a Founding Engineer, ML Inference with deep expertise in high-performance... ...Working knowledge of GPU hardware (NVIDIA) and the...RelocationVisa sponsorshipRelocation package
$220k - $320k
inference.net, a growing company in San Francisco, seeks an experienced engineer to optimize AI inference performance. The ideal candidate will have over 2 years of experience in ML systems and GPU programming. Key responsibilities include implementing optimization techniques...- Machine Learning Engineer, Inference Want to solve realtime inference problems... ...inference, scheduler design, GPU utilisation, concurrency optimisation... ..., and fast enough for real human interaction. Your work... ...this team. The stack includes Rust, C++, Python, CUDA, TensorRT,...Remote workFlexible hours
- Anyscale is seeking a Distributed LLM Inference Engineer in San Francisco, California. This pivotal role involves pushing the boundaries of performance for ML inference at scale. You'll work closely with product teams to deliver end-to-end solutions while leveraging open...
$220k - $320k
A tech startup specializing in AI inference seeks a skilled professional to optimize their inference stack. Candidates should have over 2... ...in ML systems, fluency in Python, and hands-on experience with LLM frameworks. The role offers competitive compensation of $220,000...Local area$220k
We build and run the inference engine behind every Perplexity query... ...budgets. Our stack is Rust, Python, CUDA, and CuTe... ...work on Examples Of Real Work The Team Does New... ...support in API Gateway. GPU kernels migration to... ...You understand modern LLM architectures and are able...$300k
...model training, or inference. Our client operates... ...high-performance GPU clusters powering... ...into low-latency, real-time inference and custom... ...operate inference engines such as vLLM, SGLang, and TensorRT-LLM across multiple model... ...in Python, Go, Rust, or a comparable language...Permanent employmentWorldwide$168k - $210k
Senior Software Engineer, Financial Crimes Please note this is for San... ..., join us, and build real world value. THE WORK: Ripple... ...our network passes through real-time financial crimes controls: sanctions... ...comparable JVM languages, Go, Rust) Experience with distributed systems...Full timeWork at officeLocal area$180k - $280k
...quietly rethinking the LLM stack from first principles... ...model designed for real-world reliability, decision... ...mid-2024, we've been engineering the foundation for what... ...We're looking for a GPU kernel engineer with deep... ...make our training and inference faster and more efficient...Full timeWork at officeVisa sponsorshipShift work$183k - $276k
Amplitude is seeking a Senior Software Engineer to join their Data Pipeline team in San Francisco. You will tackle complex infrastructure... ...processing. Benefits include comprehensive insurance coverage, flexible time off, and generous stipends for personal development. The salary...Flexible hours- AI Systems Engineer - Codex Core Agents The Codex... ...capability into real‑world action. We own... ...model interaction, inference, sandboxed execution... .../runtime stack, GPU fleet, and product... ...working across layers: Rust systems code,... ...on experience with LLM applications, coding...
- ...repetitive manual work in real environments. Our... ...a Backend Software Engineer at Droyd, you’ll... ...support learning, inference, control, and fleet... ...running on edge devices and GPUs Build and... ...end Nice to have Rust experience, including... ...for embedded or real-time systemsFamiliarity...
$200.8k - $251k
...large language models. Candidates should have system optimization experience and solid software engineering skills, particularly in tools like CUDA and Pytorch. This full-time position offers a competitive salary range of $200,800 - $251,000, along with comprehensive...Full time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to On-Device LLM Inference Engineer — Real-Time, Rust & GPU. Be the first to apply!
