Lead AI Agent Optimization & Evaluation Engineer
Reddit, Inc.
Reddit, Inc. is seeking a seasoned engineer to lead the development of its agent optimization and evaluation platform. You will oversee evaluation environments, benchmarking, offline replay, and synthetic task generation while driving production feedback loops to improve system reliability and efficiency. You will design model routing and orchestration, define productivity metrics, and examine agent behavior in collaboration with platform, infrastructure, and developer productivity teams. #J-18808-Ljbffr Reddit, Inc.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Lead AI Agent Optimization & Evaluation Engineer in San Francisco, CA vacancy
$180k - $225k
About Scale AIScale AI is the data... ...with the world's leading enterprises and government... ...production AI agents that automate complex... ...of the hardest engineering challenges.As a Frontier... ...to design, evaluate, and deploy intelligent... ...models, or model optimization.Familiarity with...SuggestedFull timeScale AI
San Francisco, CA2 days ago$300k
...Research Engineer, Agent Systems One of the most mission-driven organizations in AI is building the infrastructure that makes intelligent... ...and production Develop evaluation infrastructure that measures... ...loops, and system-level optimization — not just features You’re...SuggestedVisa sponsorshipAionia Group
San Francisco, CA1 day ago- ...About Us Salient builds AI agents for regulated financial services... ...The Role We're hiring the Lead Product Manager for the agent... ...teams, writing specs an engineer respects, and defining what "... ...hallucination rates — build the evaluation loop, and move the numbers....SuggestedFull timeWork at office
Salient
San Francisco, CA1 day ago $217.57k - $271k
...embrace the thoughtful use of AI tools in our daily work... ...The Role This Staff Engineer role sits at the... ...experience. You will define and lead the discipline of testing AI agents, evaluating LLM behavior, and... ...retrieval data quality gates to optimize vector search accuracy,...SuggestedFull timeTemporary workWork at officeLocal areaRemote workFlexible hoursID.me
San Francisco, CA1 day ago- Blee, an AI-first marketing content compliance platform... ...is seeking a Staff AI Engineer to architect and... ...frameworks where multiple agents—powered by custom and... ...You will also build and optimize applied AI/ML pipelines... ...tuning, prompt engineering, evaluation frameworks, and...Suggested
- ...Cognition AI in San Francisco is seeking a senior Safety/Alignment lead to build the safety function for Devin. You will shape the research agenda, evaluation, red-teaming, and deployment policies, working... ...with researchers and engineers. This hands-on role requires...
- ...Our client is building AI coworkers for IT teams: a security-focused product where an agent registers as a governed identity in... ...- and systems-heavy founding engineering role where \"full-stack\" means... ..., sandboxes, simulation/evaluation infrastructure, or developer...Full timeH1bWork at officeVisa sponsorshipMonday to Friday
Thomas Talent Network, LLC
San Francisco, CA4 days ago $200k - $300k
...People Hive is supporting an early-stage AI infrastructure client in San Francisco... ...Senior/Staff Founding Full-Stack Engineer – Agent Systems. The successful candidate will... ...Experience with agent infrastructure and evaluation systems Experience with sandboxes or...Full timeH1bWork at officeVisa sponsorshipMonday to FridayPeople Hive HR Consultancies LLC
San Francisco, CA1 day ago- Matter Intelligence seeks an Agent Infrastructure Engineer in San Francisco to build production systems... ...physics-informed and general-purpose AI into secure, evidence-backed intelligence... ..., platform, product, design, and evaluation teams. This role focuses on durable execution...
$200k - $275k
...Secure Every Identity, from AI to Human Identity is the key to... ...let’s talk. About Okta for AI Agents Okta secures access for 20,000... ...identity, platform, and security engineering teams, write production code... ...their architects and governance leads, and convert them into...Local areaWorldwideShift workSegment (Twilio)
San Francisco, CA1 day ago- Renaissance Geek seeks a senior backend engineer to own the review pipeline and the production backend. You will... ..., integrations, data observability, and the evaluations that determine review quality. You will optimize asynchronous workflows, improve prompts and model...Remote job
- ...Technical Program Manager to scope, organize, and direct the work of our technical staff across external engagements, including on-site evaluations and rapid-response analysis. You’ll own each engagement end to end from scoping to dissemination. You will work with our...
- ...Overview Deepgram is the leading platform underpinning... ...-dollar Voice AI economy, providing real... ...production-grade voice agents at scale. More than 2... ...a Senior Software Engineer - Model Evaluation & AI Systems to join... ...that inform release and optimization decisions, and directly...
- Turing is seeking US-based academic dermatologists to lead clinical reviews for AI model evaluation. You will curate complex dermatology cases, contribute to evaluation frameworks, and advise client teams on AI outputs. This is designed for dermatology faculty wanting rigorous...Remote jobContract workFlexible hours
- ...execution across post-training, enterprise post-training, fellowships, evaluation releases, and partner research, turning open-ended goals into concrete plans. You will align researchers, engineers, SPLs and external partners, surface blockers, and decide on priorities...
$197.3k - $225.1k
...AI Engineer 4 (Gen AI Platform Services: Agentic AI, Agent Guardrails, Agent Evaluation, Agent Memory) At Capital One, we are creating responsible... ...to deliver our industry leading capabilities with... ...-the-art foundation model optimization techniques to improve the...Full timePart timeLocal areaCapital One
San Francisco, CA1 day ago- ...San Francisco, CA seeks a highly technical Forward Deployed AI Engineer - Agent Systems to build and deploy advanced agent systems for... ...customers, building data pipelines, post-training workflows, evaluation infrastructure, and robust AI deployments for production use...
- Blee, an AI-first marketing content compliance platform, seeks an experienced AI Engineer to architect multi-agent systems and scalable ML pipelines. Work across fine-tuning, prompts, and evaluation, with emphasis on explainability and governance. Hybrid role with offices...Remote work
- ...On-site We are seeking a highly technical Forward Deployed AI Engineer - Agent Systems to build and deploy advanced agent systems for large... ...systems, data pipelines, post-training workflows, and evaluation infrastructure required for production deployments. Key Responsibilities...
- ...Software Engineer, Agent Evaluation and Quality Engineering · Full-time · San Francisco; New York Our mission is to automate coding. The first... ...You'll Work On Designing and building best-in-class AI evaluation system: curated datasets, offline replay, scorers...Full timeWork at office
Anysphere
San Francisco, CA4 days ago - About Decagon Decagon is the leading conversational AI platform empowering every brand to deliver concierge... ..., and Hunter Douglas to deploy AI agents that power personalized, deeply... ...functionally with Sales, Product, and Engineering to translate those needs into successful...Full timeWork at officeLocal area
- Pluto is building an AI-native professional network. We’re looking for a Founding Engineer to help build this product from the ground up, working directly with the founders to create intelligent agents that search, reason, remember, and act on behalf of users. You’ll tackle...
- ...Governance, Risk, and Compliance leader to align policy with engineering and product teams. You will own agent-related data governance, least-privilege access, and... ..., and driving audit readiness and certifications like ISO 27001 and EU AI Act. #J-18808-Ljbffr EngineersOfAI
- Artificial Intelligence Underwriting Company is seeking a Standards Lead for AI agents to guide the AIUC-1 framework. You’ll read incident reports, form strong opinions within 24 hours, and determine implications for controls. You’ll translate new regulations into actionable...
- Valerie Health, a fast-growing AI-native healthtech startup in San Francisco, seeks a Senior Agent Operations Associate to turn rapidly evolving AI models into reliable... ...with Customer Strategy, Operations, and Engineering to build, launch, and improve #J-18808-Ljbffr...
- Sphere seeks an engineer to lead our internal AI agent enablement efforts in San Francisco, working to safely scale AI agents across engineering and operations. You’ll build Go/Python-backed systems that let agents accelerate workflows with robust safety and human review...
- Pond AI is seeking a hands-on Head of Engineering (founding-level) in San Francisco to own the platform end-to-end and ship AI agents in production. You will lead a small, ambitious engineering team and drive architecture, tooling, and delivery for scalable agent-based...
- Sierra is seeking an Agent Strategist in San Francisco to own how our AI agents come to life for customers, from pilot to production handling thousands of conversations daily. You’ll work at the intersection of product, agents, and customer-facing work, translating business...
- Harbor is creating a category and seeks a leader to make AI-agent risk legible for executives. You will own Harbor's external narrative... ...will build the agent-platform partner program, design a demand engine for enterprise sales, and translate underwriting insights into...
- Kastle is hiring for a strategic customer-facing role to lead end-to-end deployments of AI agents in consumer lending. You will guide deployments, drive... ...across complex workflows. You will collaborate with Engineering and Sales to surface needs, shape product outcomes,...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead AI Agent Optimization & Evaluation Engineer. Be the first to apply!
Related searches
- work from home chat agent San Francisco, CA
- gate agent San Francisco, CA
- executive protection agent San Francisco, CA
- commissioning agent San Francisco, CA
- registered agent San Francisco, CA
- sourcing agent San Francisco, CA
- booking agent San Francisco, CA
- remote chat agent San Francisco, CA
- right of way agent San Francisco, CA
- showing agent San Francisco, CA