Staff GenAI Inference Engineer: Optimize LLM Serving Latency
$190.9k - $232.8kMenlo Ventures
A leading data and AI company is seeking a Staff Software Engineer for GenAI inference to lead the architecture and optimization of the inference engine. The role requires expertise in CUDA, GPU programming, and distributed systems design. Ideal candidates will have a strong software engineering background and a proven ability to collaborate with researchers and drive architectural decisions. Competitive compensation is offered, with a salary range of $190,900 to $232,800 USD. #J-18808-Ljbffr Menlo Ventures
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Staff GenAI Inference Engineer: Optimize LLM Serving Latency in San Francisco, CA vacancy
- ...fast-growing team of engineers in San Francisco... ...models Low-latency OCR models for targeted... ..., high-throughput inference for OCR and multimodal... ...and caching Optimize kernels, tokenization... ...Evaluate vLLM, TensorRT LLM, and Triton... ...profiling and model serving Nice to have...SuggestedWork at officeVisa sponsorshipRelocation package
- Crusoe in San Francisco is seeking a Staff Technical Program Manager to lead the Managed Inference platform team. You will ensure end-to-end program delivery for LLM workloads, driving innovation in... ...a strong focus on AI and model serving. Join us at Crusoe and make a...Suggested
$181.1k - $318.4k
Staff/Sr. Machine Learning Engineer, Foundation Models - AI, Search... ...Apple products serving millions of... ...incredible low latencies, drawing every... ...chance to work on optimizing billions of parameter... ...to optimize inference for cutting... ...Nvidia TensorRT-LLM, vLLM, DeepSpeed...SuggestedRelocation$220k
We build and run the inference engine behind every Perplexity... ...architectures at scale with tight latency and cost budgets. Our... .... Rust-native serving runtime. Develop our... ...You understand modern LLM architectures and are... ...and inference optimization techniques (e.g. quantization...Suggested$166k - $225k
...available to all. Job Description As a research engineer on the Scaling team, you will be... ...performance improvements through advanced optimization techniques including kernel fusion,... ...production training workflows and knowledge of LLM training dynamics including...SuggestedWorldwide- ...data and AI company in San Francisco is seeking a Senior Engineer to enhance their Model Serving platform. This role requires expertise in building... ...distributed systems and collaboration across teams to optimize performance and reliability. Ideal candidates will have...
$130k - $175k
...your interest in Uncountable Engineering! Description Uncountable is seeking... ...through Generative AI. As an LLM Applications Engineer, you... ...: Implement and optimize RAG (Retrieval-Augmented Generation... ...Specifically optimizing queries that serve as the context for LLM...Uncountable Inc
San Francisco, CA3 days ago$263.9k - $301.2k
...applied science and engineering teams to deliver our... ...to reimagine how we serve our customers and businesses... ...large language model inference, similarity search,... ...state-of-the-art LLM optimization techniques to improve... ...performance—scalability, cost, latency, throughput—of large-...Full timePart timeLocal areaCapital One
San Francisco, CA3 days ago$314.8k - $359.3k
...Sr. Distinguished AI Engineer At Capital One, we are... ...capabilities to reimagine how we serve our customers and... ...large language model inference, similarity search,... ...state‑of‑the‑art LLM optimization techniques to improve... ...— scalability, cost, latency, throughput — of large...Local areaCapital One National Association
San Francisco, CA4 days ago$220k - $320k
Senior Software Engineer - Model Performance Help us make inference blazingly fast. If you... ..., and turning optimization techniques into production... ...with the goal of serving models faster and... ...performance: latency, throughput, cost... ...SGLang, TensorRT-LLM) and underlying libraries...Work at office$225.4k - $257.2k
...applied science and engineering teams to deliver our... ...to reimagine how we serve our customers and businesses... ...large language model inference, similarity search,... ...state‑of‑the‑art LLM optimization techniques to improve... ...— scalability, cost, latency, throughput — of large...Full timePart timeLocal areaCapital One National Association
San Francisco, CA3 days ago$229.9k - $262.4k
...Overview Senior Lead AI Engineer At Capital One,... ...to reimagine how we serve our customers and businesses... ...large language model inference, similarity search,... ...state-of-the-art LLM optimization techniques to improve... ...— scalability, cost, latency, throughput — of large...Full timePart timeLocal areaCapital One Financial Corp
San Francisco, CA1 day ago$192k - $260k
...Foundation Model Serving is the API... ...frontier AI model inference for open source... ...re looking for engineers who have owned... ...deep building LLM APIs and runtimes... ...at scale. As a Staff Engineer, you’... ...throughput, low‑latency inference on... ...trade‑offs to optimize performance, throughput...Local areaWorldwide$225.4k - $257.2k
...applied science and engineering teams to deliver our... ...to reimagine how we serve our customers and businesses... ...large language model inference, similarity search,... ...state‑of‑the‑art LLM optimization techniques to improve... ...— scalability, cost, latency, throughput — of large...Full timePart timeLocal area- ...-class talent in AI, engineering, and product as we redefine... ..., building, and optimizing scalable, high-... ...infrastructure, Kubernetes, and LLM and ML deployment... ...and reducing latency for large-scale AI systems... ...data pipelines, model serving, and inference optimizations. Ensure...
- ...Work together with engineers, scientists, operators... ..., evaluation, and inference across Handshake.... ...training, and model serving. Develop and scale our LLM platform, including... ...controls for cost, latency, and reliability.... ...data infrastructure. Optimize inference...Full timeWork at officeRemote workFlexible hours
$250k
...frameworks that abstract across platforms, memory managers that optimize dynamically, observability stacks that surface what's... ...no visibility into what those models are doing in the field. Inference latency, memory pressure, thermal headroom: none of it is legible. When...- ...company in San Francisco seeks a Staff Software Engineer to lead kernel-level performance engineering for GenAI workloads. The role involves designing and optimizing high-performance GPU kernels,... ...focused on pushing the frontier of inference performance. #J-18808-Ljbffr...
$120k - $160k
Description Uncountable Engineering is seeking driven... ...: Building specific LLM powered features end-... ...specific tasks (prompting, optimizing latency, tuning/testing... ...in‑house LLM stack to serve the evolving needs of... ...for fine‑tuning and inference, etc) Uncountable has...- A leading technology firm in San Francisco is looking for an AI Security Engineer to ensure the safety and integrity of their AI systems. You'll serve as a technical authority, leading secure AI solutions and MLOps governance. The position requires expertise in Generative...
- ...our memory platform powering LLM apps. You’ll own features end... ...Build for scale & speed: Optimize latency, caching, and query patterns;... ...Familiarity with vLLM , model serving, or lightweight ML infra integrations... ...comes from a founder or an engineer who joined yesterday. We let...Work at officeRemote work
Mem0
San Francisco, CA2 days ago $197.3k - $225.1k
...Overview Lead AI Engineer (MLX, Agentic AI, Gen AI... ...capabilities to reimagine how we serve our customers and... ...large language model inference, similarity search,... ...state-of-the-art LLM optimization techniques to improve... ...— scalability, cost, latency, throughput — of large...Full timePart timeLocal areaCapital One
San Francisco, CA2 days ago- ...Machine Learning Engineer Build the data infrastructure for robots operating in the real world.... ...Key Responsibilities: Building and owning inference infrastructure — model serving, scaling, latency/cost optimization (think TorchServe, vLLM, Triton) Selecting...InternshipRemote work
Foxglove
San Francisco, CA1 day ago $220k - $270k
...Staff Software Engineer page is loaded## Staff Software Engineerremote... ...Team, you'll serve as a forward-deployed... ...performance, and recommend optimizations* Design and expand... ...extracting composable LLM workflows (drafting,... ...evaluation design, cost and latency optimization, prompt...Temporary workRemote workJones Lang LaSalle Incorporated
San Francisco, CA4 days ago$170k - $215k
...future of the climate. As a Staff Storage Administrator on... ...Roadmap Influence: Serve as the primary technical... ...You will sit with their engineering teams to provide feedback... ...ensuring all systems are optimized for AI training and inference I/O patterns. Enterprise...Temporary work- ...the Investigational Drug Services (IDS) Staff Pharmacist II provides pharmaceutical care... ...facilitate a change in the therapeutic plan, optimize medication therapies, and ensure... ...Training and Educational Activities Serve as a preceptor for pharmacy learners (e.g...Work experience placementWorldwideShift work
University of California , San Francisco
San Francisco, CA2 days ago - Inference.net is seeking a Senior Software Engineer to maximize inference performance. You will optimize the end-to-end stack from CUDA kernels to serving frameworks, focusing on latency, throughput, and cost efficiency for large language models. You will work across the...
$160k - $250k
...Together AI is building the Inference Platform that brings the most... ...If you get a thrill from optimizing latency down to the last millisecond... ...across data centers and model engine pods. Develop auto-scaling... ...latency and throughput to serve diverse workloads efficiently...Full timeLocal area- ...Pharmaceutical Services, the Outpatient Staff Pharmacist II provides pharmaceutical care... ...facilitate a change in the therapeutic plan, optimize medication therapies, and ensure... ...Training and Educational Activities Serve as a preceptor for pharmacy learners (e.g...Work experience placementWorldwideShift work
University of California , San Francisco
San Francisco, CA4 days ago $93.36k
...and taskings actions. Your typical duties will include: Serving as staff assistant to an executive senior leader performing analysis and... ...artificial intelligence (AI) such as a large language model (LLM) like ChatGPT or Copilot. To preview the questionnaire please...Permanent employmentFull timePart timeFor contractorsInternshipWork at officeVisa sponsorshipCustoms and Border Protection
San Francisco, CA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff GenAI Inference Engineer: Optimize LLM Serving Latency. Be the first to apply!
Related searches
- staff engineer San Francisco, CA
- senior staff systems engineer San Francisco, CA
- assistant chief engineer San Francisco, CA
- staff design engineer San Francisco, CA
- assistant engineer San Francisco, CA
- research assistant engineering San Francisco, CA
- senior staff engineer San Francisco, CA
- staff security engineer San Francisco, CA
- software engineer staff San Francisco, CA
- assistant engineering manager San Francisco, CA