Staff Engineer, LLM Inference & Serving
Mixpeek
Mixpeek in San Francisco seeks an inference researcher to advance end-to-end LLM serving, from kernels to autoscaling. You will own bets on speculative decoding, memory management, and inference optimizations, collaborating with the research lead and customer-facing engineers. You will work with external labs and push innovations into practical products, shaping the research agenda and impacting cost per token and tail latency. In-person work in our SF office is expected. #J-18808-Ljbffr Mixpeek
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Staff Engineer, LLM Inference & Serving in San Francisco, CA vacancy
$150k - $300k
Prime Intellect is looking for a skilled ML Systems Engineer to build and optimize LLM serving infrastructure and inference systems. This hybrid role involves contributing to the scalability of their reinforcement learning training. Successful candidates will have over...SuggestedRelocation package$190.9k - $232.8k
A leading data and AI company is seeking a Staff Software Engineer for GenAI inference to lead the architecture and optimization of the inference engine. The role requires expertise in CUDA, GPU programming, and distributed systems design. Ideal candidates will have a strong...Suggested$150k - $300k
...to build and optimize machine learning systems as part of a hybrid team. This role focuses on developing efficient architecture for serving LLMs and optimizing performance using modern infrastructure tools. Ideal candidates will have significant experience with ML...SuggestedRemote job- ...San Francisco is seeking a Member of Technical Staff for their infrastructure team. In this role, you will own the cloud systems that serve our compression API and build global low-latency, high-throughput GPU ML inference infrastructure. The ideal candidate will have solid...SuggestedVisa sponsorship
$220k
We build and run the inference engine behind every Perplexity query and deploy dozens of model architectures... ...Vera Rubin racks tomorrow. Rust-native serving runtime. Develop our internal Rust-based... ...is a plus. You understand modern LLM architectures and are able to bring them...Suggested- Gravity Engineering Services Pvt Ltd. is seeking a Staff Engineer to design and build scalable systems for Databricks Model Serving. You will optimize performance and lead technical initiatives... ...background in model serving and inference systems. This role offers opportunities...
- ...Talent is looking for a Member of Technical Staff, specializing in ML Infrastructure and Inference, focused on building and optimizing... ...candidate will possess strong software engineering fundamentals, experience with model serving, and proficiency in Python and C++....
- Gravity Engineering Services Pvt Ltd. is seeking a Staff Engineer to lead technical efforts for their Inference Runtime. This role requires a deep understanding of systems engineering... ...high-performance distributed systems serving millions of users. The ideal candidate...
- ...business. Foundation Model Serving is the API Product for... ...frontier AI model inference for open source models like... ...necessary. We’re looking for engineers who have owned high... ...in getting deep building LLM APIs and runtimes at scale. As a Staff Engineer, you’ll play a...
$192k - $260k
A leading data and AI company is seeking a Staff Engineer to design and implement core systems for Foundation Model Serving. The ideal candidate will have over 10 years of experience in building large-scale distributed systems and will collaborate closely across teams...$180k - $250k
A tech company in San Francisco is looking for a Staff Software Engineer to enhance model performance for generative media models. The ideal candidate will design advanced model serving architectures, develop performance tools, and collaborate with the Applied ML team....$181.1k - $318.4k
Staff/Sr. Machine Learning Engineer, Foundation Models - AI, Search & Knowledge Platforms... ...exciting Apple products serving millions of queries every... ...Research team to optimize inference for cutting edge model architectures... ...with Nvidia TensorRT-LLM, vLLM, DeepSpeed, Nvidia...Relocation$215k - $265k
...is seeking a highly experienced Senior Staff Engineer specializing in AI Data Path & Storage... ...systems with next-generation AI inference pipelines. This role involves coding,... ...understanding of AI inference systems, LLM serving architectures, and KV cache optimization...Local area- Jaide Health is seeking an engineer for their Model Efficiency team in San Francisco. The role focuses on building reliable ML systems... ..., plus strong skills in C++ or Python and insights into the LLM inference ecosystem. A commitment to diversity and inclusive work...Remote job
$298k - $368k
...from a diverse set of sensors, enabling engineers like you to (1) develop methods for efficiently... ...offboard hardware. You will Design VLM/LLM model architecture and drive strong... ...Proven expertise in low-latency on-device inference techniques and a deep understanding of hardware...Full timeRemote work- ...seeking an experienced AI Engineer with deep expertise in... ...our team as a Senior Staff Architect. In this role... ...reward shaping, and model serving. Research &... ...techniques , including inference‑time search, chain‑of‑thought... ...large language model (LLM) training and fine‑tuning...Flexible hours
$350k
Mirendil in San Francisco is searching for an engineer to develop and optimize inference systems for cutting-edge AI models. You will handle the complete inference stack, enhancing performance and reliability. The role involves partnering with teams to deploy new architectures...$160k - $250k
...startup building AI agents for hardware engineers. You\'ll own the core agent intelligence... ...This role reports directly to the CTO and serves as technical lead for a small team of AI... ..., with at least 2 years building agentic LLM-based systems — agents that call tools, manage...For contractorsRemote work- ...Company in San Francisco, USA, is hiring a Member of Technical Staff, Infrastructure to own the end-to-end cloud infrastructure for... ...-performance compression API. Build global, low-latency GPU ML inference systems that sit in the critical path of customer traffic, and...Visa sponsorship
$253k - $308k
Staff Engineer, Engineering Productivity & AI Quality Harper is an AI-native commercial insurance... ...is existential. Our agents write code, serve customers, assemble submissions, and... ...Quality (eval suite design, golden datasets, LLM‑as‑judge graders, production trajectory...Part timeWork at officeRelocation$168.91k - $273.37k
...that's the point. As a Forward Deployed AI Engineer, you're part of a single team that both... ...data products. Design and ship LLM‑ and agent‑powered systems on that data —... ...usable by LLMs, the agent frameworks, and the serving infrastructure. Build the evals, tracing...Full timeLive inWork at officeLocal areaRemote workWork from homeHome officeFlexible hours2 days per week- B Capital is seeking a skilled engineer for GPU infrastructure in San Francisco. This role involves designing and operating high-performance systems for model inference, synthetic data generation, and reinforcement learning. The ideal candidate has strong GPU systems experience...
$200k - $400k
Inferact is looking for a Developer Relations Engineer in San Francisco, California, to help developers utilize vLLM for AI inference. This unique role involves teaching technical concepts, creating educational content, and engaging with the AI infrastructure community....Remote work- ...generation of enterprise AI. As a Staff AI Infrastructure Engineer, you will design, build,... ...large‑scale training, serving, evaluation, and... ...infrastructure, high‑performance inference, and enterprise AI platforms... ...as vLLM, SGLang, TensorRT‑LLM, Triton Inference Server,...Work experience placementFlexible hours
$207k - $290k
.... SAIGroup's portfolio serves 2,000+ global enterprise... ...an experienced AI Engineer with deep expertise in... ...join our team as a Senior Staff Architect. In this role... ...techniques , including inference-time search, chain-of-thought... ...large language model (LLM) training and fine-...WorldwideFlexible hoursJazzX AI
San Francisco, CA25 days ago- ...the barrier to creating engineered threats and AI‑... ...a Member of Technical Staff, ML Product Engineer, this... ...and services that make inference fast, reliable, and cheap... ...behind Omnii: real‑time serving plus large batch scoring... ...internals: vLLM, TensorRT-LLM, or Triton, with hands‑...Local area
$200k - $400k
A leading AI technology company located in San Francisco is seeking an infrastructure engineer to build distributed systems for their AI inference engine. The role involves designing systems that ensure minimal latency and maximum reliability. Candidates should have a...Visa sponsorship$232k - $290k
# Senior Staff Security Engineer, AIRipple·San Francisco, CA·$232k - $290kfull-timeleadPosted 3 hours... ...define and operationalize guardrails for LLM and agentic AI adoption, and build AI-... ...operate within Ripple's auditable perimeter.Serve as Ripple's go-to technical resource on...- Gravity Engineering Services Pvt Ltd. is seeking a Staff Engineer to design and implement core systems and APIs for Foundation Model Serving. The role is crucial for enabling high-throughput, low-latency inference on GPU workloads. Ideal candidates will have over 10 years...
- About the Role As a Staff Engineer at Databricks, you will play a critical role in shaping both... ...the foundational infrastructure of Model Serving. You will design and build systems that enable high-throughput, low-latency inference across CPU and GPU workloads, influence...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Engineer, LLM Inference & Serving. Be the first to apply!
Related searches
- assistant chief engineer San Francisco, CA
- technology administrator San Francisco, CA
- project engineer assistant project manager San Francisco, CA
- staff security engineer San Francisco, CA
- engineering aide San Francisco, CA
- senior staff systems engineer San Francisco, CA
- staff design engineer San Francisco, CA
- research assistant engineering San Francisco, CA
- staff data engineer San Francisco, CA
- assistant engineering manager San Francisco, CA