Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Product Manager, Agent Harness

The Consulting Solutions

About the Role As a Product Manager for the Agent Harness, you will own this framework. Agent quality is improving rapidly—we shipped Composer2, our own frontier coding model, and are training agents through real‑time RL on user data. Your job is to turn those research advances into product that developers can feel. This is not a role where you write specs and hand them off. You’ll be reading agent traces, analyzing failure modes, designing evaluation frameworks, and making judgment calls about what an agent should and shouldn’t attempt. You’ll work at the boundary between research and product, where the roadmap is shaped by empirical results as much as customer feedback. Responsibilities Owning the agent planning and execution framework: how agents decompose tasks, decide what tools to use, and recover when a step fails. Balancing autonomy with predictability. Designing how developers observe and steer agents: real‑time progress, guardrails, the ability to redirect mid‑task. The experience should build trust without requiring micromanagement. Building evaluation and benchmarking systems: defining what "good" means for agent quality—task completion rate, error recovery, hallucination frequency—and building the harnesses to measure it. These measurements drive engineering and research priorities. Analyzing agent traces at scale: identifying where agents get stuck, loop, hallucinate, or take unproductive paths, and turning those patterns into concrete improvements. Defining the primitives for agent extensibility: how agents use tools, access codebase context, call external services via MCPs and plugins on the Cursor Marketplace, and how developers customize agent behavior through rules and constraints. Improving the default Cursor agent experience (the “Auto” model setting): making smart model choices based on user needs, model capabilities, and cost appetite. Shaping multi‑agent coordination: how subagents share context and avoid conflicts when executing in parallel across files and systems. This matters more as developers spin up fleets of agents simultaneously. You may be a fit if You have built or evaluated AI agents, LLM applications, or ML‑powered developer tools. You're deeply technical. You're comfortable reading code, analyzing traces, and reasoning about system behavior at a low level. You have strong intuition for evaluation and measurement. You know how to define metrics that capture quality, not just activity. You can move between the big picture and the details—from "what should agents be capable of in six months?" to "why did this agent fail on this specific task?" You're comfortable in a research‑adjacent environment where the roadmap is shaped by empirical results, not just customer requests. You have experience with reinforcement learning, agent frameworks, or AI evaluation—either as a practitioner or working closely with researchers. You thrive in ambiguous, fast‑moving environments and enjoy making hard tradeoffs with incomplete information. #J-18808-Ljbffr The Consulting Solutions

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Product Manager, Agent Harness in San Francisco, CA vacancy
  • The Consulting Solutions is seeking a Product Manager for the Agent Harness. In this role, you will own the framework and translate advancements into a product developers can use. Responsibilities include analyzing agent performance, defining evaluation metrics, and ensuring... 
    Suggested

    The Consulting Solutions

    San Francisco, CA
    2 days ago
  •  ...specific agentic AI. Assort’s omnichannel AI agents seamlessly integrate with EHR/PMS and...  .... Since launching in 2023, Assort has managed over 65M+ patient interactions , slashing...  ...in healthcare operations—designing production-grade AI that clinicians and patients rely... 
    Suggested
    Work at office

    Assort Health Inc.

    San Francisco, CA
    2 days ago
  • $100k - $250k

     ...for? Hercules is looking for a senior or staff level product manager to own our AI coding agent. What will you do? Your #1 and only goal is to make...  ...pareto frontier) You will own Hercules' entire context/harness system design (which includes system prompt, tools, memory... 
    Suggested
    Work at office
    Remote work

    Hercules

    San Francisco, CA
    2 days ago
  • $350k

     ...example areas you might work on: Build and innovate on the agent harness: agent loop architecture, tool integrations, prompt scaffolding...  ...systems for horizontal scaling of agents: memory, state management, multi‑agent coordination, and task decomposition Build guardrails... 
    Suggested

    Mirendil

    San Francisco, CA
    2 days ago
  • crewAI, Inc. is seeking a Product Manager to enhance the agent experience with their no-code Builder. You'll shape the product direction by collaborating closely with engineering, design, and the VP of Product. The successful candidate will manage the roadmap, engage with... 
    Suggested

    crewAI, Inc.

    San Francisco, CA
    2 days ago
  • $149k - $190k

    Prosper is seeking a Product Manager for the Agent Platform in San Francisco to enhance in-house tools for agents. You will partner with Operations leaders and engage frontline agents to optimize workflows. This hybrid role allows two days a week at the office and offers... 
    Work at office
    2 days per week

    Prosper

    San Francisco, CA
    5 days ago
  •  ...and CTO of Facebook. Bret was also one of Google’s earliest product managers and co‑creator of Google Maps. Before founding Sierra, Clay spent...  ...Google Workspace. What you’ll do Bring Enterprise‑grade AI Agents to the World: We're on a mission to revolutionize the way... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    4 days ago
  • The role We're hiring a Product Manager to drive our agent experience — the surface where users build, deploy, and interact with agents. This centers on our no-code Studio builder, which is at the heart of our product: easier to build with than the alternatives, while... 

    crewAI, Inc.

    San Francisco, CA
    2 days ago
  • Decagon in San Francisco is searching for a Product Manager for Duet, their AI agent product. This role focuses on owning the Duet product roadmap while ensuring effective customer relationships and product validation. The ideal candidate will have over 5 years of experience... 

    Decagon

    San Francisco, CA
    3 days ago
  •  ...compliance operations using AI-driven voice agents. Backed by Y Combinator, Commerce...  ...40x in the last 6 months and our Agent Product and Success team is a high impact group...  ...environments. This role blends relationship management with hands on execution and is ideal for... 
    Contract work
    Relocation package

    Kastle

    San Francisco, CA
    2 days ago
  •  ...physical economy, where the problems are hard, the data is complex, and the impact is immediate and tangible. We’re looking for Agent Product Managers to help design, launch, and scale the next generation of AI agents that power our customers’ businesses. What You’ll Do:... 
    Immediate start
    Sleeping nights

    Netic

    San Francisco, CA
    5 days ago
  • A leading AI solutions company in San Francisco is seeking an experienced Product Manager to lead the development of enterprise-grade AI agents. This role involves collaborating closely with cross-functional teams to innovate and execute product strategies that enhance... 

    Sierra

    San Francisco, CA
    4 days ago
  • About the Role We’re looking for a Product Manager to lead a growing team responsible for building the agents that evolve the Axion platform. You’ll work in close collaboration with our Engineering, Design, Solutions, and Operations teams. The ideal candidate has a deep... 
    Temporary work
    Local area
    Flexible hours

    Axion

    San Francisco, CA
    2 days ago
  • Glean Technologies, Inc. is seeking a Product Manager for Agent Security and Governance in San Francisco. You will play a vital role in shaping the product and team, developing new technologies and features while understanding customer needs. Candidates should have over... 

    Glean Technologies, Inc.

    San Francisco, CA
    5 days ago
  •  ...like with Conduit. At properties using the product, guests could ask a question whenever...  ...away, and get routed to a human when the agent needed help. The result felt faster, calmer...  ...Conduit helps operators build, deploy, and manage agents across the systems they already... 
    Work at office

    Conduit AI

    San Francisco, CA
    2 days ago
  • Valdera in San Francisco is looking for a Product Manager to lead the strategy and execution of AI-native products. This role requires a leader with a strong background in technology, design, and business strategy, focusing on creating AI-powered systems. Key responsibilities... 

    Valdera

    San Francisco, CA
    5 days ago
  • Key Responsibilities Define and own the product roadmap for AI agents that automate and enhance hospitality guest interactions. Collaborate with engineering...  ...concepts and iterate quickly. Prioritize features and manage the product backlog using Agile methodologies, ensuring... 

    Gravity Engineering Services Pvt Ltd.

    San Francisco, CA
    2 days ago
  • A leading AI platform company in San Francisco is seeking a Product Manager for Agent Studio, the core environment for developing AI agents. The role involves defining user experiences, owning the development life cycle, and integrating AI tools. Ideal candidates will possess... 

    Sierra

    San Francisco, CA
    3 days ago
  • GoTo Meeting is expanding its team in San Francisco with a Product Manager for the Agent Platform. This pivotal role involves shaping the tools for frontline agents by collaborating with operations and utilizing AI to enhance productivity. The ideal candidate should have... 

    GoTo Meeting

    San Francisco, CA
    5 days ago
  • B Capital is seeking a Product Manager for Salesforce Data 360 in San Francisco. The ideal candidate will lead the development of AI Agents and enterprise search capabilities. This role requires strong technical skills, the ability to prototype ideas, and experience in... 

    B Capital

    San Francisco, CA
    1 day ago
  • 慨正橡扯 in San Francisco seeks a dedicated Product Owner to lead AI agents for strategic clients. The role demands deep engagement with product ownership, requiring a mix of technical savvy and customer interaction. Ideal candidates will boast 5-8 years' experience in technical... 

    慨正橡扯

    San Francisco, CA
    4 days ago
  • Assort Health is looking for a Product Manager for AI Agents to lead the charge in developing high-performance AI systems that improve healthcare access. You will collaborate closely with engineering and design teams and work directly with customers to refine AI agents... 

    Assort Health

    San Francisco, CA
    4 days ago
  •  ...prioritizing in person work where possible. The Role Compa is building AI agents for our customer compensation teams to enable smarter, faster pay decision making with greater confidence. As a Product Manager, you’ll own key areas of our AI agent platform, turning emerging AI... 

    Compa

    San Francisco, CA
    3 days ago
  • Traba, based in San Francisco, is seeking an experienced Product Manager to join our Agents team. This unique role focuses on building a cutting-edge agentic platform for supply chain workflows, integrating deeply with customer operations. The ideal candidate will embody... 

    Traba

    San Francisco, CA
    3 days ago
  • LiveKit is seeking a Product Manager for their Agent Observability product. In this role, you will develop a vision and roadmap crucial for understanding how voice agents perform in production environments. You'll design user-friendly interfaces for both technical and non... 

    LiveKit

    San Francisco, CA
    5 days ago
  • Sierra is looking for an experienced Product Manager to build enterprise-grade AI agents and engage with customers to understand their needs. You will work closely with engineering teams to shape the product roadmap and ensure successful implementations. The ideal candidate... 

    Sierra

    San Francisco, CA
    3 days ago
  • A logistics technology company is seeking a Product Manager to build innovative AI-driven solutions that enhance efficiency in logistics. The ideal candidate will have over 6 years of experience in product management, particularly in developing complex software products... 

    Augment

    San Francisco, CA
    5 days ago
  • Netic is looking for an Agent Product Manager to join our innovative team in San Francisco. You will design, launch, and scale AI agents that enhance customer operations and deliver measurable outcomes. The ideal candidate has experience in product management, technical... 

    Netic

    San Francisco, CA
    4 days ago
  • $200k - $285k

    A leading conversational AI company in San Francisco is seeking a Product Manager to define the product foundation for AI agents. You will own the roadmap for core capabilities and collaborate with customers and teams to create powerful, flexible solutions. Ideal candidates... 
    Flexible hours

    Decagon

    San Francisco, CA
    2 days ago
  • $100k - $250k

    A technology company in San Francisco is looking for a senior or staff level product manager to lead the development of their AI coding agent. The ideal candidate will own the design and implementation of the product while leveraging large user data for enhancements. Candidates... 
    Work at office
    Remote work

    Hercules

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Product Manager, Agent Harness. Be the first to apply!