AI Infra Engineer - Low-Latency Model Serving
Baseten
Baseten in San Francisco seeks an engineer to own the Model API surface for fast, reliable inference. You will optimize runtime performance, profile CUDA kernels, and implement scalable, developer‑friendly serving experiences across multi‑GPU setups. You will collaborate with product and infra teams to ship robust APIs, metering, and auth, contributing to a platform used by leading AI deployments. #J-18808-Ljbffr Baseten
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Infra Engineer - Low-Latency Model Serving in San Francisco, CA vacancy
$275k - $350k
AI Talent Now is seeking an AI Lead in San Francisco, California to integrate AI capabilities into their design tool. Responsibilities include fine-tuning AI models and managing infrastructure for deployment. The ideal candidate will have 5-10 years of experience in machine...Suggested- ...Francisco is seeking a Systems / Platform Engineer to join the tvScientific AI infrastructure team, focusing on sub‑millisecond decisioning, low‑latency data access, and a scalable training/serving stack. You’ll work with SRE, data infra, and software engineering to harden...Suggested
- Nooks in San Francisco is searching for senior engineers dedicated to building real-time Voice AI solutions. Ideal candidates will have over 5 years of engineering... ...expertise, particularly in React, Node.js, and low-latency systems. You'll collaborate directly with clients...Suggested
- ...YC X25) is seeking a Site Reliability Engineer in San Francisco, California, to ensure the reliability of our AI infrastructure platform. Your mission... ...the core systems that power ultra-low-latency serverless computing as we serve billions of requests. With a focus on...Suggested
- Gina’s Tech Jobs - IT Recruiting Agency is seeking talented AI Backend Engineers to develop robust AI systems from home. You will design systems that ensure low latency and high reliability, collaborating with global teams to enhance performance. The ideal candidate has...SuggestedRemote job
- A leading data and AI company in San Francisco is seeking a Staff Engineer to design and implement systems for their AI/ML Model Serving platform. You will collaborate with product, infrastructure, and research teams to ensure high-performance system delivery. The ideal...
- Gravity Engineering Services Pvt Ltd. is seeking a Staff Engineer to design and build scalable systems for Databricks Model Serving. You will optimize performance and lead technical initiatives across CPU and GPU workloads. Ideal candidates have 10+ years of experience...
- ...the web by building AI agents that can... ...from training our own models to generative product... ...: Scale infra for post-training of... ...inference (throughput and latency of perception‑... ...closely with product engineers to translate cutting... ...throughput, latency) Low level systems...Work at officeRelocationVisa sponsorship
Yutori
San Francisco, CA1 day ago $200k - $275k
...world's most dynamic AI companies, like... ...bring cutting-edge models into production.... ...build the platform engineers turn to to ship AI... ...distributed systems, model serving, and developer... ...performance, and infra, helping to define... ...record of owning low‑latency, reliable backend...Flexible hours$227.2k - $417k
About the Role: As a Software Engineer on the ML Infrastructure team, you will collaborate... ...team is developing and maintaining low‑latency ML model serving systems that support Deep Learning,... ...latency, cost, and efficiency of our infra. Lead large scale cross functional...Full timeTemporary workLocal areaFlexible hours- ...the intersection of platform engineering, site reliability, and... ...and operability of Meshy’s AI model serving stack, along with core engineering... ...Continuously optimize throughput, latency, and availability of the... ...as AI infrastructure (AI infra), inference systems, and AI...Work at officeRemote workFlexible hours
MeshyAI
San Francisco, CA2 hours ago $166k - $225k
...running the world's best data and AI infrastructure platform so... ...their business. Databricks’ Model Serving product provides enterprises... ...models. It offers real-time, low-latency inference, governance,... ...cost efficiency. As a Senior Engineer, you’ll play a critical role...Local areaWorldwide- ...is seeking a highly skilled AI Engineer to design, build, and deploy... ...including data pipeline development, model training and evaluation, and... ..., and departmental policies; serve as technical lead for... ...architectures and ensure reliable, low‑latency integration of AI...Work experience placement1 day per week
$216.2k - $270.25k
...technology company is seeking a Software Engineer to join their ML Infrastructure team in San... .... You will design platforms for scalable serving of LLM workloads, work closely with... ...16,200—$270,250 USD, plus equity and comprehensive benefits. #J-18808-Ljbffr Scale AI$192k - $260k
...running the world’s best data and AI infrastructure platform so... ...their business. Foundation Model Serving is the API Product for... ...necessary. We’re looking for engineers who have owned high‑scale operational... ...enable high‑throughput, low‑latency inference on GPU workloads...Local areaWorldwide$140k - $300k
...frameworks, including orchestration engines, tool-use pipelines, memory... ...stability, scalability, and low-latency performance under high-... ...improvement. Optimize inference and serving performance, including... ...supporting SFT, RL, and reward model iteration. Collaborate closely...- ...building the human layer of AI. Our mission is to make... ...in multimodal AI for modeling human-to-human... ...experienced Research Scientist/Engineer with a focus on model... ...benchmark trade-offs across latency, cost, and quality... ...architectures such as low‑rank adapters Strong understanding...Remote workRelocation packageFlexible hours
$135.2k - $306.4k
...The Senior Principal AI Agent / ML Software Engineer is a Senior Staff-level, hands... ...Responsibilities Serve as a senior technical owner... ...execution, inference systems, model serving, AI workflow... ...distributed services optimized for low latency, high throughput, GPU...Temporary workFlexible hoursOracle
San Francisco, CA13 hours ago$250k
...ground truth infrastructure for AI agents - creating the data,... .... As an AI / Infrastructure Engineer focused on LLM systems, you... ...deploying and scaling large language models. This includes building low-latency inference systems, GPU-optimised serving infrastructure, and the...$150k - $300k
Unsiloed AI — Founding Backend Engineer Type: Full-time | On-site | San Francisco, CA Compensation: $15... ...Design and operate high‑throughput, low‑latency services in real production environments... ...mindset Nice to have: experience serving ML / VLM / GPU‑heavy workloads Tech...Full timeH1bWork at officeVisa sponsorship$220k
...build and run the inference engine behind every Perplexity... ...and deploy dozens of model architectures at scale with tight latency and cost budgets. Our stack... ...racks tomorrow. Rust-native serving runtime. Develop our... ...model/tensor parallelism. Low-precision inference: INT8...$225k - $250k
...Cline is the leading open‑source AI coding agent. Trusted by the world's largest engineering organizations and individual developers alike, Cline is model agnostic, provider agnostic, and IDE... ...: Design and operate scalable, low‑latency infrastructure for LLM inference,...$216.2k - $270.25k
As a Software Engineer on the ML Infrastructure team, you will design... ...scalable, reliable, and efficient serving of LLMs. Our platform powers... ...to integrate and optimize models for production and research use... ...mission is to develop reliable AI systems for the world’s most important...Full time$300k - $400k
About the Role Principal AI/ML Engineer - AdTech Drive the development... ...of advanced machine‑learning models and AI‑driven features for... ...build, and deploy scalable, low‑latency ML solutions for campaign optimization... ...integration with ad‑serving architecture. Strategic...- ...Job Description AI Backend Engineer, Work From Home We... ...our AI stack, between models, backend systems, and real users, where latency, correctness, reliability... ...operate backend systems that serve AI-powered insurance... ...reliably at scale with low latency and high...Remote workWork from home
Parallel Partners
San Francisco, CA8 days ago - Gravity Engineering Services Pvt Ltd. is seeking an experienced API Engineer to join their team in San Francisco, California. This role... ...and operating high-performance APIs with a focus on low latency and high throughput. The ideal candidate will have at least 5...
- ...responsible and reliable AI systems, changing... ...science and engineering teams to deliver our... ...to reimagine how we serve our customers and businesses... ...customers. Our AI models and platforms... ...model minutiae or infra plumbing. You will... ...hardware utilization, latency, throughput, and...Work at office
$96.8k - $306.4k
...Description The Senior Principal AI Agent / ML Software Engineer is a Senior Staff-level,... .... Responsibilities Serve as a senior technical owner... ..., inference systems, model serving, AI workflow orchestration... ...services optimized for low latency, high throughput, GPU...Temporary workFlexible hours- About the Role As a Staff Engineer at Databricks, you will play a critical... ...infrastructure of Model Serving. You will design and build systems... ...that enable high-throughput, low-latency inference across CPU and GPU... ...influence Databricks’ broader AI platform strategy. What We...
- Gravity Engineering Services Pvt Ltd. is seeking a Staff Engineer to design and implement core systems and APIs for Foundation Model Serving. The role is crucial for enabling high-throughput, low-latency inference on GPU workloads. Ideal candidates will have over 10 years...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Infra Engineer - Low-Latency Model Serving. Be the first to apply!
Related searches
- ai engineer San Francisco, CA
- ai ml engineer San Francisco, CA
- ai research engineer San Francisco, CA
- senior ai engineer San Francisco, CA
- machine learning ai engineer San Francisco, CA
- ai developer San Francisco, CA
- ai prompt engineer San Francisco, CA
- ai engineer remote San Francisco, CA
- python ai developer
- generative ai engineer