LLM Infra Backend Engineer: Streaming & Guardrails
CloudAct Inc.
CloudAct Inc. is looking for a backend engineer to enhance the FastAPI proxy. You will handle streaming, retries, and ensure the system operates under production loads with efficiency. The ideal candidate should have a minimum of 5 years of backend Python experience and be proficient with asyncio, especially in high-concurrency environments. Additional advantages include exposure to LLM APIs or streaming proxies. #J-18808-Ljbffr CloudAct Inc.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the LLM Infra Backend Engineer: Streaming & Guardrails in Sunnyvale, CA vacancy
- ...provider. That means guardrails, prompt management, A/... ...attribution, rate limits, and streaming. You will own the... ...Extend the Nemo Backend FastAPI proxy with new... ...to have Prior work on LLM APIs, model gateways,... ...Experience with LLM routing engines or model gateways...Suggested
- ...and help millions of people put food on the table by finding the perfect job. Hiring.Cafe is building a 100 times better job search engine — fast, comprehensive, honest, and actually useful. We index millions of jobs, remove noise, rank what matters, and help people...SuggestedFlexible hours
- Nutanix is seeking a software engineer based in Santa Clara, California, to develop AI-powered product features. The role requires building backend services and APIs, implementing LLM workflows, and collaborating with product managers and designers. The ideal candidate...Suggested
- About the Role We are seeking a Senior Backend & Infra Engineer to help shape the core infrastructure... ...Core AI Capabilities: Optimize LLM integrations, multi-provider model routing... ...window management, cost optimization, and streaming responses. Build Memory & Retrieval...SuggestedFull timeWork at office3 days per week
- HiringCafe, located in Cupertino, California, is on a mission to create a superior job search engine. We prioritize real opportunities and aim to transform how job seekers find their next employment. We are looking for talented individuals with strong programming fundamentals...SuggestedFlexible hours
- A leading healthcare technology company in Mountain View is seeking a talented backend engineer to enhance their Ambient AI solutions. The role involves contributing to a cutting-edge healthcare platform and scaling infrastructure for large health systems. Ideal candidates...
$140k - $200k
A healthcare technology company is looking for a Backend Engineer to join the Ambient AI team in Mountain View, CA. The role involves developing advanced AI solutions and scaling infrastructure for large health systems. Candidates should have strong programming skills...$170k - $240k
...team is building the autonomous backend that turns healthcare's most... ...cycle operations, with LLM-powered agents and copilots that... ...document extraction with real guardrails and compliance built in. You'... ...looking for a Senior Backend Engineer who takes ownership end-to-end...Full timeWork at officeImmediate startCommure
Mountain View, CA4 days ago$180k - $225k
...Overview FlexAI is looking for a Senior Backend Engineer (Infrastructure & AI Platform) with... ...Partner with Product, Runtime, and Infra leadership to translate requirements into... ...Data: SQL + NoSQL databases, caching, streaming systems Observability: Prometheus,...Work at officeFlexAI
Santa Clara, CA3 days ago$166k - $220k
A leading technology company in Mountain View is seeking an experienced backend or infrastructure engineer to join their ML/AI Environments team. The ideal candidate will have 5+ years of experience and strong programming skills in Python, Scala, or Java. You will be responsible...- ...Senior Backend Engineer At Commure, we're building the AI Operating System for healthcare,... ...into core revenue cycle operations, with LLM-powered agents and copilots that automate... ..., and document extraction with real guardrails and compliance built in. You'll own the...Full timeImmediate start
Commure
Mountain View, CA8 days ago $212.8k
...Security group, the mission of the Online Infra team is to ensure responsible data... ...in an event-driven manner. As a software engineer, you will be engaged in designing solutions... ...privacy by design and default principles in backend systems. • Experience of privacy awareness...Temporary workLocal areaWorldwideTik Tok
San Jose, CA4 days ago- ...centers from cost centers into engines of customer intelligence,... ...facing agents, agent-assist, and backend automation—along with deep... ...observability, monitoring, safety, and guardrails for AI systems Contribute to... ...for real-time processing, streaming, or event-driven...Remote jobFlexible hours
$180k
...highly motivated, and focused on engineering excellence. This organization... ...is built. Most of our backend infrastructure is written in... ...EXPERIENCE: Experience with LLM inference engines and serving... ...knowledge of gRPC (unary, response streaming, bi-directional streaming,...Temporary workWorldwidexAI
Palo Alto, CA29 days ago- ...RESTful web APIs. ~ Experience in kafka services or any event streaming services ~ Experience with Maven tool to build and deploy... ...product features. Required Skills : Professional Engineer Basic Qualification : Additional Skills : Engineer...
Procyon TS
Sunnyvale, CA4 days ago - ...curation. About the Role We're looking for an Infrastructure Engineer to own the execution layer beneath our RL environments: the systems... ...to detect failure modes early in a rollout (reward hacks, infra faults, fairness issues) and to revert to a known‑good state, patch...
- A leading AI infrastructure company in California is seeking a Senior Backend Engineer. This role involves architecting and developing core backend systems for their AI PaaS platform, with a strong emphasis on Golang expertise. The role requires knowledge of distributed...
- Qualcomm is seeking a skilled Backend Engineer to join their backend team in Santa Clara, California. You will design and implement scalable backend systems for the AI-driven Video Surveillance as a Service platform. The ideal candidate will have a Bachelor's degree along...
$151.9k - $227.9k
...Technologies, Inc. Job Area: Engineering Group, Engineering Group... ...seeking a skilled and motivated Backend Engineer to join our backend team... ...especially in computer vision or LLM agents. • Familiarity with video streaming technologies, media processing, or...Work experience placementWork from homeQualcomm
Santa Clara, CA2 days ago$196.5k - $219.3k
...implement functions to monitor and filter LLM requests/responses in real time,... ...into platform features. Mentor junior engineers on secure backend development and best practices in an Agile... ...vector databases/embedding stores, or streaming data pipelines for ML – especially as...Full timeWorldwideEdelman
Sunnyvale, CA4 days ago- A tech company in Palo Alto is actively seeking a Backend Engineer to develop systems that manage GPU clusters for AI workloads. The role demands 5+ years of backend experience, with strong skills in Go or Python. You will design APIs for GPU orchestration and manage resources...
$160k - $200k
...Senior Frontend Engineer We’re a cybersecurity company... ...handle high-volume, streaming data with intuitive... ...contracts, contribute to backend endpoints as needed,... ...responses with citations, guardrails/redaction, and human-... ...) Understanding of LLM streaming patterns,...Full timeFlexible hoursEdelman
Sunnyvale, CA4 days ago$160k - $200k
...time position focused on developing and maintaining media-related backend services for real-time communication. Candidates should be... ...in C++ and/or Go, and have a strong understanding of real-time streaming protocols and WebRTC. The role includes responsibilities for optimizing...Full time$204k - $259k
...Analysis Infra SWE Waymo is an autonomous driving technology... ...Design and build low-latency streaming data processing platform, enabling... ...Computer Science, Computer Engineering, or equivalent ~5+ years of professional experience in backend engineering, databases, and...Full timeInternshipRemote workWaymo
Mountain View, CA1 day ago- ...place for you. We are seeking a senior software engineer with experience in backend API development, Kubernetes-based microservices,... ...'s security, safety, and privacy of GenAI and LLM-based systems with advanced guardrail capabilities. Integrate GenAI systems...Local area
Protegrity
Menlo Park, CA3 days ago - Backend Engineer (Senior+) TLDR Build a beautiful and intuitive next generation data tool. Tech stack: TypeScript, React, Python. Work autonomously... ...data from 3rd party sources to your internal destinations. Streaming pipelines: Ingest and transform real-time data. DBT: Build,...Full timeRemote work
$213k - $263k
...Senior Backend Engineer, Fleet Infrastructure Waymo is an autonomous driving technology company with the mission to be the world's most... ...Science or equivalent practical experience ~7+ years of backend/infra experience ~ Strong knowledge in building software...Full timeTemporary workRemote workWaymo
Mountain View, CA3 days ago$213k - $263k
...Senior Backend Engineer, Security Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver... ...or equivalent practical experience ~7+ years of backend/infra experience ~ Strong knowledge with security concepts (encryption...Full timeRemote workWaymo
Mountain View, CA3 days ago$244.8k
...Backend Software Engineer, TikTok Live Recommendation Infrastructure Location: San Jose Employment... ...About the Team Live-stream is a new and rapidly growing business... ...audiences. Our Live-stream Recommendation Infra team is responsible for building up and...Temporary workLocal areaTik Tok
San Jose, CA4 days ago$150k - $175k
...generation of enterprise AI. What you will do As part of the engineering team, you'll work alongside industry-leading cryptographers, AI... ...in working with AI Agents (LangGraph, CrewAI, AutoGen), LLM APIs (OpenAI, Anthropic), and the Model Context Protocol (MCP)....Flexible hoursDigiCert
Sunnyvale, CA20 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to LLM Infra Backend Engineer: Streaming & Guardrails. Be the first to apply!
Related searches