Staff Technical Product Manager, Evals
$200k - $350kHercules
What are we looking for? Hercules is looking for a senior or staff level product manager to own evals for our AI coding agent What will you do? Your #1 and only goal is to make Hercules' AI coding agent cheaper, faster, and more accurate (our pareto frontier) You will own Hercules Bench, our offline eval set You will own Hercules online evals You will leverage our 500k+ users, 1M+ code repos, and millions of chat threads to guide your eval creation You will anonymize data to protect our customer's privacy What skills do we expect you to have? We expect candidates to be incredibly strong across all of the following areas: Software fundamentals : You are building an AI coding agent. We expect you to be technical and able to code. We expect you to have a decent understanding of system design, architecture, data modeling, and API design. In saying this, you will work with an engineering counterpart to work on complex implementation Evals . You have explicitly created and/or managed technical evals before. You should be able to work across a variety of different eval methodologies and deeply understand optimal reward modeling, Product sense : strong intuition for the goals customers are trying to achieve and novel solutions to help them achieve them Design sense : Track record of building simple, delightful UX/UI that makes it easy for customers to achieve their goals AI context and harness engineering : strong experience with the most up to date context and harness engineering best practices like summarization/compression, prompting, prompt caching, subagents, memory, context management, tools, async tools, deferred tools, skills, etc Yes, we know this industry is evolving rapidly and the \"best practices\" change every week, but yes, we are still actively looking for someone who has wrestled with these challenges at scale and has strong opinions about what works and what doesn't Our core stack : TypeScript, React, (some sort of agent framework like AI SDK), (some sort of eval platform like LangSmith) AI : able to truly 10x your workflow by being on frontier of AI coding What’s Hercules’ stack? Technologies: Typescript, React, AI SDK, Postgres, Redis, Clickhouse, Tailwind, IaC (CDK, Terraform), Docker, Otel, Hono, Drizzle ORM, Temporal Clouds: Cloudflare, AWS AI Models: OpenAI, Anthropic, Google What are Hercules’ operating principles? Hercules’ operating principles guide how we make decisions every day. We take them very seriously. Most of the principles can be learned on the job, however, we do specifically filter for the following principles in our hiring process: We ship . We ship fast, iterate, and correct mistakes quickly. Our bias for action speeds up our learning and delights our customers. We strive for excellence. Doing things well is in our DNA. We have good taste. We get the details right. Our outputs are exceptionally simple and self-serve. We’re proud of our work. We are hungry . We are willing to work hard and break through walls to achieve the outcomes we want. We are willing to operate at any level to get things done. Is Hercules in-office or remote? Hercules founding team works in-office in San Francisco (Kearny and Bush). We have a very strong preference for in office. We are open to remote for exceptional candidates who are willing to 1) work in US time zones and 2) travel to the office regularly. What are Hercules’ benefits? Unlimited AI token budget Take what you need vacation policy Healthcare, vision, and dental Daily lunches, dinners and snacks in the office We guarantee you will do the best work of your career here What’s the compensation? $200k-$350k + equity (variable based on seniority, equity, and location) #J-18808-Ljbffr Hercules
- ...Job Description Job Description Job Description Senior Technical Product Manager - AI Agents, Evals & Reliability, Artificial Intelligence (AI) Required, Work From Home We are looking for a Senior Technical Product Manager. This is a deeply technical,...SuggestedRemote workWork from home
- Software Guidance & Assistance, Inc., (SGA), is searching for an Technical AI Product Manager for a Remote Full Time assignment with one of our premier Technology clients. Responsibilities : Define and document requirements for AI-powered products and systems...SuggestedFull timeRemote work
- ...Product ManagerWe're hiring a Product Manager to help decide what Pump builds next as we grow from cloud savings into AI cost optimization. You'll own... ...fast-growing B2B SaaS startupTrack record of shipping technical products that customers adopted and paid forTechnical...Suggested
- ...running on software built 20-40 years ago. The product sits between legacy ERPs and modern... ...across ERP Sync, AP Automation, Expense Management, and Vendor Management. About the Role We’re hiring a hands-on, technical product leader to build new products on top...SuggestedFull timeFor contractorsInternshipImmediate startRemote work
- ...Join to apply for the Technical Product Manager role at Novel Technology Services . Before continuing to the job description, please note a few important details: The client requires candidates to possess five years of experience as a Product Manager/Engineer. Internships...SuggestedInternship
- ...enabling them to build the future without being constrained by infrastructure complexity. About the Role Smarsh is seeking a Technical AI Product Manager to lead the strategy, development, and commercialization of AI-powered solutions within our compliance and data archival...
- ...not solved before. Other work expands a product that an existing customer already uses.... ...product that shipped 3+ years in product management, including helping take a product from 0... ...in conversation Hands-on experience with technical products. You can write and review SQL,...Home office
- ...Technical Product ManagerSan Francisco | 5 Days OnsiteLocation: Onsite in San FranciscoCompensation: Competitive Salary + EquityUniversalAGI... ...About the RoleUniversalAGI is hiring our first Technical Product Manager to build and own the execution of our core platform from 0 to...For contractorsWork at officeImmediate startFlexible hours1 day per week
$143k - $175k
...their use, and combat underage usage of our products. We have the opportunity to address one... ..., scientists, designers, product managers, operations experts, and customer service... ...and requirements to audiences of varying technical backgrounds, including business stakeholders...- ...Job title: Senior Technical Product Manager Number of Positions: 2 Domain: Order-to-Cash (O2C) We are seeking Senior Technical Product Managers / Product Architects who possess a strong blend of business, functional, and technical expertise within the Order-to-Cash (O2...
- ...Senior Technical Product ManagerLocation: San Francisco, CA Company Stage of Funding: Series B AI Enterprise Software (91M Raised) Office Type... ..., and workflow automation. As a Senior Technical Product Manager, you'll own core product areas that blend AI, data, and enterprise...Work at officeRelocation package
$160k - $180k
...Bobyard Construction Product RoleBobyard Construction is a multi-trillion industry running on PDFs, rulers, and 1990s software. The biggest choke point is the cost estimate. Takeoffs are manual, slow, and one mistake blows millions. We use computer vision and NLP to read...- ...David AI Technical Product ManagerDavid AI is the first audio data research company. We bring an R&D approach to data–developing datasets with... ...to push the frontier of audio AI.Our Technical Product Manager team builds the Data Factory at David AI. We set the strategy...Work experience placementWork at office
- ...teams to prioritize patient care and access to healthcare. Technical Product Manager You own product vision, roadmap, and momentum for our agentic... ...too early. You work with health care executives and ground staff all the same. You sit at the intersection of customers,...Immediate start
- ...looking for bold, ambitious people to help redefine how businesses manage and experience travel. Working here, you can expect... ...ordinary and build the future of travel with us. Your Mission As a Technical Product Manager at Engine, you'll own key parts of a platform at the...
- ...Technical Product ManagerSensor Tower's digital intelligence platform provides unparalleled insight into market trends, advertising, competitor... ...industry leaders have found success. As a technical product manager focused on our mobile intelligence offering, you will help...Contract workWorldwide
$104.9k - $199.07k
...important healthcare questions. Through our products, education, and services, MedInsight is... ...solutions for healthcare cost and care management.MedInsight is a subsidiary of Milliman;... ...Summary: MedInsight is seeking a Technical Product Manager with a platform engineering...Full timeWork experience placementRemote workFlexible hours$190.8k - $267.1k
Senior Technical Product Manager, Ads ML Platform Reddit Senior Technical Product Manager, Ads ML Platform Team Description: The Ads Marketplace team is a strategic growth engine for Reddit Ads. Our mission is to democratize high-performance advertising by enabling brands...For contractorsWork experience placementWork at officeRemote workHome officeFlexible hours$180k - $220k
...challenging work, with the prime directive of "Keeping Our Heroes Connected"? THE OPPORTUNITY Silvus is seeking a highly motivated Technical Product Manager (TPM) to drive requirements, product backlog, market adoption, and revenue growth for the Electronic Warfare (EW) &...Permanent employmentFor contractorsWork at officeRemote workRelocation- Motorola Solutions’ Silvus Technologies, Irvine, CA, seeks a Technical Product Manager (Spectrum Dominance) to translate DoD requirements into structured software features and own the end-to-end backlog. The role bridges R&D and go-to-market for advanced spectrum dominance...Remote job
$100k - $150k
...of our customers delivering life-changing medicines and treatments to patients more efficient and productive. We are looking for an experienced technical product manager to design and deliver high-value AI Agent solutions for Vault Medical. In this role, you will be the...Work at officeLocal areaWork from home$101k - $152k
...any employer, at the date of hire. This position is ineligible for employment Visa sponsorship. Overall Purpose The Technical Product Manager is a key role on our Data & AI team charged with defining and delivering the next generation of enterprise data platform capabilities...Hourly payTemporary workWork at officeImmediate startVisa sponsorshipWork visaFlexible hours- ...Technical Product ManagerSan Francisco, CAAbout UsMetriport is an open-source data intelligence platform that helps healthcare organizations... ...product specifications for the engineering team.Execution: Managing the product lifecycle from ideation through development,...Work at officeWork from homeRelocation
- Anthropic Limited is seeking a Technical Program Manager for Safeguards Infrastructure and Evals in San Francisco. You will own operational health, drive reliability, incident response, and post-mortem processes while coordinating platform investments and cross-team dependencies...
$178k - $258k
...experience in corporate/commercial strategy, management consulting, business architecture, or Go-To-Market strategy. Experience in product or program management. Preferred... ...responsible for all support and media and technical services for customers big and small across...Worldwide$225k
...scheduling, and dedicated local property management, so owners get the joy of a true... ...to purchase and make our growth product-led. We're hiring a principal-level technical product manager to own that... ...how to prove they're ready. Evals and guardrails go in before launch...Full timeLocal areaRemote workHome office$220k - $240k
...AI and high-performance computing. Our Technical Sourcing team sits at the center of that... ...IREN is looking for a Technical Sourcing Manager, Server & Storage Platforms to take full... ...’s speed and margins. Track OEM and ODM product roadmaps closely and shape sourcing strategy...Daily paidContract workTemporary workLive inLive outRelocation package- ...information, this is the place for you. Evals at Exa The Evals Team sits at the... .... That’s where you come in. As a Product Manager on the Evals team, you will partner... ...Strong numerical literacy and technical background. CS/EE degree, production engineering...Full timeH1b
$228k - $314k
...NBCUniversal, and Square—build better products and digital experiences. With... .... As Director of Product Management for AI & Growth, you’ll lead... ...” looks like for AI. Set up evals, quality benchmarks,... ...adoption and revenue.Enough technical depth to engage credibly with...Home office$385k
...to build beneficial AI systems. About the role As a Product Manager, Technical, you will own and be responsible for building a world class... ...Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However...Work at officeLocal areaVisa sponsorshipFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Technical Product Manager, Evals. Be the first to apply!
- hr manager San Francisco, CA
- finance hr manager San Francisco, CA
- director of people operations San Francisco, CA
- human resources hr manager San Francisco, CA
- director of people & culture San Francisco, CA
- staff manager San Francisco, CA
- hr data manager San Francisco, CA
- people manager San Francisco, CA
- hr executive San Francisco, CA
- staff program manager San Francisco, CA



