Distributed LLM Inference Engineer - Scale AI at Speed
Anyscale
Anyscale is seeking a Distributed LLM Inference Engineer in San Francisco, California. This pivotal role involves pushing the boundaries of performance for ML inference at scale. You'll work closely with product teams to deliver end-to-end solutions while leveraging open-source technologies and contributing to community projects. Candidates should have a solid understanding of distributed systems and familiarity with deep learning frameworks, ideally with experience in PyTorch and Ray. Anyscale offers competitive compensation and extensive benefits, including healthcare coverage and stock options. #J-18808-Ljbffr Anyscale
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Distributed LLM Inference Engineer - Scale AI at Speed in San Francisco, CA vacancy
- Gravity Engineering Services Pvt Ltd. is looking for a Distributed LLM Inference Engineer to join their team. This critical role focuses on enhancing performance for ML inference, ensuring scalability and efficiency in solutions used by both open-source and corporate clients...Suggested
- ...on a mission to democratize distributed computing and make it accessible... ...accelerate the progress of AI applications out into the... ...developer or data scientist can scale an ML application from their... ...the Role As a Distributed LLM Inference Engineer, you will help systems and...Suggested
- ...San Francisco is seeking a talented engineer to design and implement robust systems... ...that ensure fast and cost-efficient AI inference at global scale. You will be responsible for... ...candidate has a strong background in distributed systems and is eager to engage in complex...Suggested
- ...company is seeking a talented software engineer to join their dynamic Inference team. This role involves designing... ...infrastructure for large-scale multimodal models, focusing on high-... ...product teams to push the boundaries of AI technology, ensuring reliable production...Suggested
$160k - $230k
...the Role At Together.ai, we are building... ...efficient and scalable inference for large language... ...and Optimization Engineer to design, develop, and optimize distributed inference engines that... ...language models at scale. This role will... ...shape the future of LLM inference infrastructure...SuggestedFull time- Vast.ai Inc. is seeking a systems engineer with HPC or parallel programming experience to help scale AI inference. You will design and optimize GPU kernels and tensor libraries, leveraging CUDA/C++ and related frameworks to push the bleeding edge of AI performance. This...
- ...* **Move at Drata Speed (Precision & Velocity... ...as both a central engineering function and an... ...stack to help Drata scale reliably for a... ...SLO tracking, and distributed tracing* Experience... ...with AIOps - using AI/ML-based tooling for... ...services (e.g., LLM inference latency, non-determinism...Work at officeImmediate startWorldwideMonday to FridayFlexible hours
Careers at Drata
San Francisco, CA3 days ago - Research Scientist/Engineer - Post-training, Inference, & Safety and Security Virtue AI Full-time Unknown About Virtue AI Virtue... ...Python, along with expertise in LLM libraries like PyTorch,... ...Experience in conducting large‑scale red‑teaming for LLMs and agents...Full time
$325k
A leading AI research company in San Francisco seeks an engineer to optimize their powerful AI models for high-volume production environments. The ideal candidate... ...with ML architectures, and experience with distributed systems. This role involves collaboration with researchers...$153k - $376k
...designs into code, or iterating with AI. From idea to product, Figma... ...everything we build. As a Software Engineer on our Infrastructure team, you’... ...millions of people worldwide. We’re scaling fast, and we’re looking for experienced distributed systems engineers across a...Full timeRemote workWork from homeWorldwideFigma
San Francisco, CA2 days ago- Health Harbor, located in San Francisco, is seeking experienced engineers to build and scale their Voice AI LLM and orchestration system. The role demands strong problem-solving skills and the ability to work under high pressure, with a commitment of about 70 hours a week...Flexible hours
$166.9k - $225.9k
...operates as both a central engineering function and an... ...native stack to help Drata scale reliably for a rapidly... ..., SLO tracking, and distributed tracing. Experience... ...Experience with AIOps—using AI/ML‑based tooling for... ...services (e.g., LLM inference latency, non‑determinism...Flexible hoursDrata
San Francisco, CA3 days ago- ...experiences with AI. We are primarily... ...As a Software Engineer, Infrastructure at... ...teams to deliver with speed and confidence.... ...our platform and LLM inference serving as we rapidly... ...Architect and operate distributed systems that... ...SaaS services at scale, and agentic architecture...Full timeFlexible hours
Sierra
San Francisco, CA12 days ago $216.2k - $270.25k
Senior Software Engineer, Full-Stack - Scale GP Scale GP (Scale Generative AI Platform) is an enterprise-grade Generative... ...for knowledge retrieval, inference, evaluation, and more. We... ...in Python, working with distributed systems, data pipelines, and ML/LLM components. Integrate...Full time$220k
We build and run the inference engine behind every Perplexity query and deploy... ...of model architectures at scale with tight latency and cost... .... You understand modern LLM architectures and are able to... ...built and operated production distributed systems under real load -...- Acceler8 Talent is looking for a Senior Distributed Systems Engineer with over 7 years of experience in... ...Francisco focuses on building systems for AI-powered clinical environments,... ...TypeScript, combined with experience in scaling systems and working with relational databases...
- ...Shepherd is an AI-native commercial... ...built for this speed: Complex... ...a Software Engineer focused on Applied... ...build, and deploy LLM-powered... ...model serving, inference, evaluation, and... ...scalable APIs and distributed systems Hands... ...AI workloads at scale A product...Work experience placementWork at office
Shepherd Labs Inc.
San Francisco, CA1 day ago - Reflection in San Francisco is seeking an engineer to build and operate web-scale systems for data collection. This role involves working closely with AI researchers to optimize crawling... ...will have a strong background in distributed systems and web data processing. Competitive...
$216k - $270k
Scale GP (Scale Generative AI Platform) is an enterprise-grade Generative... ...knowledge retrieval, inference, evaluation, and... ...strong Senior Full-Stack Engineer to help us build, scale... ...Python, working with distributed systems, data pipelines, and ML/LLM components. *...Full time- Gravity Engineering Services Pvt Ltd. is looking for an Inference Frameworks and Optimization Engineer to enhance the performance of AI infrastructure. This role involves designing distributed inference engines that support multimodal models, optimizing frameworks for...
$190.9k - $232.8k
A leading data and AI company is seeking a Staff Software Engineer for GenAI inference to lead the architecture and optimization of the inference engine. The role... ...requires expertise in CUDA, GPU programming, and distributed systems design. Ideal candidates will have a...- ...infrastructure foundation for AI teams. With instant... ...and serve low-latency inference. Companies like Suno,... ..., including Lovable, Scale AI, Substack, and Suno... ..., and experienced engineering and product leaders with... ...primarily create and distribute technical content that...Work at office
Modal
San Francisco, CA4 days ago $87.95k
...now. We are currently seeking a API LLM Integration / ReactJS Engineer - Hybrid to join our team in Santa... ...Integration Engineer will work with our AI team, software engineers, and... ...unmatched capabilities in enterprise-scale AI, cloud, security, connectivity, data...Full timeTemporary workWork experience placementWork at officeRemote workFlexible hours- Vast.ai is seeking a systems engineer to scale AI inference and optimize GPU performance at our San Francisco or Los Angeles offices. You will leverage your HPC background to push the bleeding edge of AI, working with CUDA/C++ and a modern tech stack. Ideal candidates...Full time
$250k - $285k
...vertically integrated AI infrastructure... ...who believe in the scale of our ambition... ...Product Security Engineer with deep AI/ML security... ..., and distributed AI systems. This is... ...to‑end, including LLM pipelines, vector... ...including MLOps, inference architectures, vector...Temporary workCrusoe
San Francisco, CA5 days ago- ...with is building AI systems to... ...to add a Data Engineer to help build out... ...—designing and scaling pipelines, building... ...exposure to LLM-driven workflows... ...processing and inference pipelines Implement... ...that improve speed and efficiency... ...with distributed systems or large...
Glocomms
San Francisco, CA2 days ago - B Capital is seeking a data engineer to ensure high data quality for training AI models. You will own the upstream data quality for LLM post-training and design automated QA methods in a collaborative environment. Ideal candidates will have strong engineering skills, a...
$149k - $240k
...IQ is HP’s new AI innovation lab.... ...with HP’s global scale, we’re building... ...‑class team—engineers, designers, researchers... ...real‑time AI inference and processing.... ...practices for distributed AI systems.... ...Proficient in LLM integration into... ...Work with the speed and focus of a...Full timeTemporary workLocal areaFlexible hours$228.6k - $314.25k
...world's best data and AI infrastructure... ...business. Founded by engineers — and customer obsessed... ...with data to scaling our services and infrastructure... ...or SRE in highly distributed, multi-cloud... .... Familiarity with LLM infrastructure, training/inference pipelines, or...Full timeWorldwide$192k - $240k
...Security Operations Engineer Brex is the intelligent finance... ...spend effortlessly. Brex's AI-native automation and world-... ...about building systems that scale with speed and intention. Our teams span... ...Experience with securing distributed systems in AWS, cloud and Kubernetes...Work at officeRemote workWork from homeColorwave Inc
San Francisco, CA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Distributed LLM Inference Engineer - Scale AI at Speed. Be the first to apply!
