Inference Performance Engineer — Turbocharge LLM Inference
Jobleads-US
Baseten is seeking an Inference Performance Engineer to accelerate AI workloads across the stack—from the inference engine to scheduling, serving, and routing. You will apply techniques like prefill/disaggregation, speculative decoding, and KV-cache management to shrink latency and cut costs, impacting how fast customers run models.
Join a fast-paced startup that values hands-on impact, cross-team collaboration, and contributions to open-source inference engines while enabling production-grade
#J-18808-Ljbffr Jobleads-USVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Inference Performance Engineer — Turbocharge LLM Inference in San Francisco, CA vacancy
$170k - $245k
...NEA, and Addition with $250+ million raised to date.About the roleAs a Distributed LLM Inference Engineer, you will help systems and optimizations that push the boundaries of performance for inference at large scale. This is an incredibly critical role to Anyscale as it...PerformanceWork at officeAnyscale
San Francisco, CA1 day ago- ...The role You'll own the cost and performance of our inference stack. Your work will determine how... ...change. You'll work closely with the engineers operating the serving fleet while... ...engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when...PerformanceFlexible hours
- ...Baseten is hiring distributed systems engineers to build the runtime that powers large-scale LLM inference. You will deploy and operate Model APIs, routing, autoscaling... ...through deployment, with emphasis on performance, observability, and developer experience. This...Performance
- ...Adaption Labs, Inc. is seeking an ML systems engineer to own the cost and performance of the inference stack. You will optimize caching, batching, quantization, decoding, and kernel-level tuning to improve throughput and latency while preserving model quality. You will...Performance
$204k - $216k
...scale. You build and run the inference infrastructure that turns the... ...applied AI, research, and platform engineering, and you make the difference... ...and change. 2. Latency and performance Drive down latency so... ...-on experience serving ML or LLM models in production at scale...PerformanceSapience AI Corporation
San Francisco, CA2 days ago- ...Modal is seeking a systems engineer to advance the container runtime stack used for inference workloads, working close to Linux, containers, storage, GPU drivers... ...infrastructure in Rust and Go, diagnose tough Linux and performance issues, and influence the architecture of Modal...Performance
- ...is seeking a Senior or Staff Software Engineer with deep expertise in building and operating... ...systems. You will work on Hedra’s inference infrastructure, designing and operating... .... The role emphasizes reliability, performance, and cost efficiency, requiring you to...Performance
- ...Sierra is the leading platform for customer-facing AI agents, based in San Francisco. We are seeking a Software Engineer on the Inference team to define architectures for self-hosted and provider-based inference, focusing on latency, reliability, and cost at scale....
$300k
...GPU Optimisation Engineer — Real-Time Inference Want to push GPU performance to its limits — not in theory, but in production systems handling real-time speech and... ...with attention kernels, decoding paths, or LLM‑style runtimes Comfort profiling with low‑level...PerformanceRelocationVisa sponsorshipFree visaVincere Ltd
San Francisco, CA3 days ago- ...driven and detail‑oriented AI/ML Engineer to join our growing applied... ..., and regularisation. LLM & NLP experience — Must Have... ...TorchServe, BentoML, vLLM, or Triton Inference Server. Nice to Have... ...data, reviewed bi‑annually. ~ Performance bonus (up to 20% of base) tied...PerformanceFlexible hours
Qubeaxis
San Francisco, CA4 days ago - ...small, fast-growing team of engineers in San Francisco powering Fortune... ...low-latency, high-throughput inference for OCR and multimodal models... ...Evaluate vLLM, TensorRT LLM, and Triton tradeoffs Implement... ...with clear SLOs Own performance dashboards and capacity planning...PerformanceVisa sponsorshipRelocation package
PULSE
San Francisco, CA2 days ago $300 per month
...strategies, and be part of a high-performing team that believes in each... ...:At Crusoe, our Production Engineering team ensures the reliability... ...focus on serving and scaling LLM workloadsBuild automation and... ...distributed AI pipelines and inference servicesDefine, measure, and...PerformanceTemporary workCrusoe
San Francisco, CA4 days ago- ...with hands-on support from AMD engineers the team is scaling rapidly... ...about pushing the limits of performance on modern accelerators. In this... ...for large-scale training and inference. This role is ideal for... ...with a focus on large-scale LLM training and inference. Integrate...PerformanceFlexible hours
- ...Tubi is seeking a Staff Software Engineer for the ML Infrastructure team to build world-class ML inference platforms powering recommendations, search, and content understanding. You will advance low-latency model serving, feature/vector stores, and experimentation engines...
- ...Responsibilities As a ML System Engineer on the AI & ML Platform’s Inference team, you will design and optimize... ...(vLLM, SGLang, Triton, TensorRT-LLM, etc.).It would be great, but not... ...required if you have:1+ years of system performance optimization experienceLow-level...PerformanceWork at officeLocal area
Atlassian
San Francisco, CA4 days ago $190k - $265k
...improve their business. Founded by engineers — and customer-obsessed — we... ...of AI.The Foundation Model Inference team is the backbone of... ...enterprise-grade reliability and performance. Our Foundation Model APIs... ...The impact you will have:Build LLM infrastructure powering large...PerformanceLocal areaWorldwideDataBricks
San Francisco, CA3 days ago$250k - $300k
...strategies, and be part of a high-performing team that believes in each... ....About the Role:As a Senior Engineering Manager, you will lead an... ...technical and hands-on with the inference stack end to end: profiling performance... ....Comfort with modern LLM serving frameworks such as...PerformanceTemporary workCrusoe
San Francisco, CA2 days ago- ...About This RoleAs a Senior ML System Engineer on the AI & ML Platform’s Inference team, you will design and optimize... ...experience, 2+ years of system performance optimization experienceDeep low-level... ...(vLLM, SGLang, Triton, TensorRT-LLM, etc.).Strong background in system...PerformanceWork at officeLocal area
Atlassian
San Francisco, CA2 days ago $190k - $215k
...strategies, and be part of a high-performing team that believes in each... ...running production AI inference workloads and understands the... ...Inference, model serving platforms, LLM deployments, AI agents, or... ...Coordinate with the product and engineering teams to define customer...PerformanceFull timeTemporary workCrusoe
San Francisco, CA21 hours ago- ...Baseten powers mission-critical inference for the world's most dynamic... ...and help build the platform engineers turn to ship AI products.... ...to lead part of our Inference Performance team. This team makes the world... ...PyTorch, TensorRT or TensorRT-LLM. A track record of driving...PerformanceFlexible hours
$189.6k - $237k
...language model training and inference. The platform has been powering... ...training and evaluation of LLM's, as well as evaluation of data... ...ML systemsStrong software engineering skills, proficient in frameworks... ..., qualifications, interview performance, and relevant education or...PerformanceFull timeScale AI
San Francisco, CA3 days ago- ...Baseten powers mission-critical inference for the world's most dynamic... ...and help build the platform engineers turn to ship AI products.... ...runtime that powers large-scale LLM inference on Baseten. Our inference... ...models with industry-leading performance, scalability, and reliability...PerformanceFlexible hours
- ...Baseten powers mission-critical inference for the world's most dynamic... ...and help build the platform engineers turn to ship AI products.... ...We're looking for inference performance engineers who want to make the... ...contributions to the exciting field of LLM inference. EXAMPLE...PerformanceFlexible hours
$200k - $250k
...We're looking for a Senior Systems Engineer who owns reliability the way a founding... ...security and compliance posture, database performance, AI inference infrastructure: you can cover these... ...and disaster recovery. Own AI and LLM inference infrastructure end-to-end: Vertex...PerformanceRemote jobWork at officeWork from home- ...own the strategy and evolution of its AI inference platform. You will define how the... ...scales workloads, and delivers gains in performance, reliability, and cost. This hands-on... ...infrastructure and strong collaboration with engineering and research. You will drive end-to-end...Performance
- ...Baseten is seeking an Engineering Manager to lead part of our Inference Performance team in San Francisco. You will mentor engineers, set direction for GPU optimization across the inference engine and runtime, and partner with cross-functional groups to ship high-performance...Performance
$250k - $350k
...bring AI into production that performs when it matters most,... ...Staff Machine Learning Research Engineer, you will operate across the... ...needed — training/fine-tuning, inference, memory and retrieval, evaluation... ...in agent training methods, LLM alignment, or applied MLExperience...PerformanceFull timeScale AI
San Francisco, CA3 days ago$190k - $235k
San Francisco, CaliforniaSoftware Engineering /Full time Exempt /HybridRETHINK MANUFACTURING... ...using deep learningOptimize model inference for GPU deployment, leveraging CUDA, TensorRT... ..., CoaXPress)C/C++ experience for performance-critical componentsExperience with MLOps...PerformanceFull timeBright Machines
San Francisco, CA21 hours ago- ...Nscale is seeking a Principal AI Engineer to lead the inference and post-training pillar of our AI systems engineering organization in the United... ...guiding 20–50+ engineers and defining API contracts and performance benchmarks. You will own cost, latency, and quality...Performance
$220k - $260k
...that, including what we sense, what we infer, which problems are best addressed through... ...in the inference graph, and where the performance ceiling really is.Owning perception quality... ....GPU inference in practice: TensorRT engines, batching, fp16, and reasoning about where...PerformanceLocal areaNight shiftAtomic
San Francisco, CA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Inference Performance Engineer — Turbocharge LLM Inference. Be the first to apply!
Related searches
- sports performance coach San Francisco, CA
- performance food service San Francisco, CA
- performance food group San Francisco, CA
- performance shop San Francisco, CA
- performance specialist San Francisco, CA
- performance windows San Francisco, CA
- performance nutrition San Francisco, CA
- high performance computing engineer San Francisco, CA
- sports performance San Francisco, CA
- performance improvement consultant San Francisco, CA