Senior LLM Serving & Inference Engineer
NEAR AI
NEAR AI in San Francisco is seeking an expert in high-performance LLM serving systems and inference optimization. In this role, you will push the boundaries of how large language models are served. You will architect and maintain production, high-traffic LLM serving systems and optimize throughput, latency, and cost for leading open-source models. This role requires deep hands-on experience with inference engines such as SGLang, vLLM or TensorRT, GPU architectures, and software stacks using #J-18808-Ljbffr NEAR AI
Vacancy posted 17 hours ago
Similar jobs that could be interesting for youBased on the Senior LLM Serving & Inference Engineer in San Francisco, CA vacancy
$167.2k - $209k
...generation of AI-driven applications. We are seeking a Senior Engineer 2 to join our AI Inference Data Plane team. In this role, you will be a key... ...: Familiarity with distributed inference serving frameworks such as llm‑d, NVIDIA Dynamo, or Ray Serve. Hardware & Interconnects...SeniorLocal areaRemote workWorldwideFlexible hours- ...We are specifically seeking an expert in high‑performance LLM serving systems and inference optimization. In this role, you will push the boundaries... ...with expertise debugging and optimizing major inference engines such as SGLang, vLLM, or TensorRT. Deep knowledge of state...Suggested
NEAR.AI
San Francisco, CA4 days ago - Senior Software Engineer - AI Inference Systems AI needs far more compute than exists today, but simply adding more... ...or operating ML inference or model serving systems. A solid understanding of... ...development experience. Exposure to TensorRT-LLM, vLLM or similar inference runtimes...Senior
- ...enable efficient and scalable inference for large language models (... ...Frameworks and Optimization Engineer to design, develop, and optimize... ...to shape the future of LLM inference infrastructure, ensuring... ...parallelism for high-performance serving. Apply CUDA graph...Suggested
- Gravity Engineering Services Pvt Ltd. is seeking a talented individual in San Francisco to architect and implement robust, scalable inference systems for AI models. This in-person role focuses on optimizing model serving infrastructures for high throughput and low latency...Suggested
$200k
Kaon (prev. FlowGPT) is seeking a Platform Engineer in San Francisco, CA. This role involves optimizing large-scale LLM serving frameworks and building a scalable AI platform. Responsibilities include customizing scheduler logic, developing CUDA kernels, and maintaining...Senior$220k - $320k
A tech startup specializing in AI inference seeks a skilled professional to optimize their inference stack. Candidates should have over 2... ...in ML systems, fluency in Python, and hands-on experience with LLM frameworks. The role offers competitive compensation of $220,000...SeniorLocal area$300 per month
...At Crusoe, our Production Engineering team ensures the reliability... ...platform. We’re looking for a Senior Production Engineer with a... ...services with a focus on serving and scaling LLM workloads Build automation... ...distributed AI pipelines and inference services Define, measure,...SeniorTemporary workCrusoe
San Francisco, CA4 days ago- ...to be backed by Andreessen Horowitz, NEA, and Addition with $250+ million raised to date. About the role As a Distributed LLM Inference Engineer, you will help with systems and optimizations that push the boundaries of performance for inference at large scale. This is...Work at office
Anyscale
San Francisco, CA1 day ago - Anyscale is seeking a Distributed LLM Inference Engineer in San Francisco, California. This pivotal role involves pushing the boundaries of performance for ML inference at scale. You'll work closely with product teams to deliver end-to-end solutions while leveraging open...
- Gravity Engineering Services Pvt Ltd. is looking for a Distributed LLM Inference Engineer to join their team. This critical role focuses on enhancing performance for ML inference, ensuring scalability and efficiency in solutions used by both open-source and corporate clients...
$250.8k - $286.2k
...world-class applied science and engineering teams to deliver our industry... ...to reimagine how we serve our customers and businesses... ...training, large language model inference, similarity search, guardrails... ...and introduce state-of-the-art LLM optimization techniques to improve...SeniorFull timePart timeLocal areaCapital One
San Francisco, CA1 day ago$166.9k - $225.9k
...Customer-Obsessed keeps the people we serve at the center of our work. Competitive... ...'s SRE team operates as both a central engineering function and an embedded reliability practice... ...of AI/ML-backed services (e.g., LLM inference latency, non-determinism, prompt pipeline...SeniorWork at officeImmediate startWorldwideMonday to FridayFlexible hoursDrata Inc
San Francisco, CA17 hours ago- ...point As a Forward Deployed AI Engineer, you’re part of a single team... ...things you fix at the root This is senior, hands‑on, 0‑to‑1 work in a... ...data products Design and ship LLM‑ and agent‑powered systems on... ...the agent frameworks, and the serving infrastructure Build the evals...SeniorLive inRemote workWork from homeFlexible hours
Afresh Technologies
San Francisco, CA1 day ago $220k - $320k
inference.net, a growing company in San Francisco, seeks an experienced engineer to optimize AI inference performance. The ideal candidate will have over 2 years of experience in ML systems and GPU programming. Key responsibilities include implementing optimization techniques...Senior$156.06k - $231.14k
...point. As a Forward Deployed AI Engineer, you’re part of a single team... ...you fix at the root. This is senior, hands‑on, 0‑to‑1 work in a space... ...products. Design and ship LLM‑ and agent‑powered systems on... ...the agent frameworks, and the serving infrastructure. Build the evals...SeniorFull timeLive inWork at officeLocal areaRemote workWork from homeHome officeFlexible hours2 days per weekAfresh
San Francisco, CA3 days ago- Harrison Clarke seeks an exceptional LLM Infrastructure Engineer in San Francisco to design and scale systems for autonomous AI agents. You'll develop scalable agent execution runtimes, work on orchestration systems for multi-agent workflows, and optimize LLM deployments...Senior
- TRM Labs, an AI-powered intelligence company, seeks a Senior or Staff ML Systems Engineer - LLM to scale AI infrastructure and production-grade ML tooling. You will build modular stacks, embed models into real-time workflows, and evaluate state-of-the-art AI tools to ensure...Senior
- MakerMaker is seeking a talented Senior ML Systems Engineer to build and maintain production inference systems in San Francisco. This role requires at least 3 years of experience in production-grade infrastructure for large-scale models, along with strong analytical skills...Senior
- ...is seeking a Member of Technical Staff to design and optimize inference systems. The role involves managing KV cache allocation and improving... ...components. Ideal candidates should have strong software engineering skills and experience with ML inference systems, particularly...Senior
$286.2k - $326.7k
...Overview Senior Distinguished Engineer, AI Compute (Remote Eligible) At Capital... ...to reimagine how we serve our customers and businesses... .../ DL model training, model inference and feature generation pipelines... ...accelerate diverse workloads from LLM pre-training and...SeniorFull timePart timeLocal areaRemote work- Acceler8 Talent is seeking a Senior Software Engineer to design and optimize AI inference systems for production workloads. You’ll work across runtime behavior, scheduling, memory management and system performance to deliver faster, more scalable AI inference. You’ll collaborate...Senior
- Femtech Insider Ltd. is seeking a Senior AI Engineer to design and implement AI-driven health guidance systems. The ideal candidate will have... ...collaborating closely with product and data science teams to create LLM-backed features and ensure the quality and safety of AI...Senior
- ...Senior Compliance Engineer, AI Governance True Anomaly seeks those with the talent and ambition to... ...mission systems, and products, this role serves as the organizational authority on how... ...and governance gates within LLM pipelines, evaluate AI vendors and platforms...SeniorPermanent employment
True Anomaly
San Francisco, CA3 days ago - Arcadia is looking for a Senior Software Engineer (AI Agents) in San Francisco. In this pivotal role, you will build core systems for an intelligent... ...responsibilities will include designing production-grade LLM systems, collaborating closely with the founding team, and making...Senior
- Inception is seeking engineers and scientists to design, optimize, and scale the diffusion LLM serving systems powering production inference. Your work will help make inference faster, more cost-efficient, and more reliable. You will extend orchestration frameworks (Kubernetes...
- ...architectures and shaping how frontier AI models serve production traffic at enterprise scale,... ...Role: We are looking for a Core Systems Engineer with a deep mastery of Rust to design,... ...petabyte-scale data and compute complex inference logic with ultra-low latency. You will...Senior
- ...next-generation code review tool powered by large language models. We are seeking a Senior Full-Stack Engineer to lead the development of a proof-of-concept prototype that integrates LLM-based analysis into existing CI/CD workflows. Responsibilities: - Design and...SeniorFull timeContract workRemote work
Fuku
San Francisco, CA5 days ago $220k
...Perplexity is looking for an engineer to join their team in San Francisco. You will work on building and operating the inference engine, supporting new models, migrating GPU kernels, and developing a Rust-based serving runtime. The ideal candidate has 3+ years of experience...SeniorPerplexity
San Francisco, CA6 hours ago$190k - $250k
...models and a proprietary, high‑efficiency serving platform. Backed by multi‑million‑dollar... ...from AMD with hands‑on support from AMD engineers the team is scaling rapidly to build the... ...highly skilled Distributed Training and Inference Engineer to build, optimize, and...Flexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior LLM Serving & Inference Engineer. Be the first to apply!
Related searches
- senior trade analyst San Francisco, CA
- senior app developer San Francisco, CA
- senior customer service advisor San Francisco, CA
- senior international account manager San Francisco, CA
- senior product manager mobile San Francisco, CA
- senior magento developer San Francisco, CA
- senior quantitative risk analyst San Francisco, CA
- senior business development director San Francisco, CA
- sr marketing manager San Francisco, CA
- sr technical product manager San Francisco, CA