AI Product Engineer

About Replay

At its core, Replay was founded to help founders. We started by supporting startups through shutting down, but we have since expanded into unlocking a new revenue stream for all types of businesses.

In 2025, we had a unique insight: the data every company generates each day through collaboration, communication, and building is some of the most valuable training data in the world. Public and synthetic data can only get frontier models so far, so the next generation of model progress depends on real, proprietary data grounded in how actual businesses operate. We are a primary source of it, partnering directly with the frontier AI labs building what comes next.
Why Join Replay Now

• We have scaled from $0 to a multi-eight-figure run rate in a matter of months
• We have raised from top-tier investors, including Floodgate, Afore, Ludlow, and Hustle Fund
• We are small enough that you will carry outsized responsibility and grow as quickly as the company does
• You will partner with and build for some of the fastest and most important companies in the world
• You will help build a massive, category-defining business from the ground floor

The Role

This is a production product-engineering role for someone who has shipped and operated LLM-backed software-not a prompt-engineering, research, or company-wide AI strategy position.

As our Senior AI Product Engineer, you will turn our early dissolution support agent into a trustworthy product that resolves well-bounded customer needs and establishes the right operating model for more complex workflows. You will move between customer experience, application code, retrieval, tool contracts, model behavior, evaluation, permissions, observability, rollout, and production learning. The goal is correct resolution and customer trust-not maximum deflection or autonomy.
What You'll Do

• Own AI-assisted dissolution support from the customer problem through production behavior, measurement, and iteration
• Design retrieval, context, structured outputs, tool contracts, orchestration, and deterministic boundaries for grounded, inspectable behavior
• Build clear answer, status, no-action, and human-handoff experiences that remain useful when evidence or authority is incomplete
• Create representative, versioned evaluations for routing, grounding, usefulness, safety, stability, and real customer outcomes
• Ship with explicit permissions, tenant boundaries, privacy controls, auditability, canaries, rollback, and recovery
• Instrument runtime and tool reliability, latency, cost, repeat contact, support effort, and serious failure modes
• Turn production failures into durable product, evaluation, and system improvements
• Determine whether complex workflows should be automated, AI-assisted, structured for a human, or deliberately remain human-owned, then build the approved product approach
• Simplify, replace, or remove agentic components when deterministic software or a clearer product experience would work better
• Work closely with Product, Support, Security, domain experts, and full-stack engineers who own the surrounding Dissolution product

What Success Looks Like

• Customers get correct, useful resolution for a meaningful set of dissolution needs-not merely fewer human replies
• Unsupported claims and unsafe actions remain inside explicit launch guardrails, with sensitive failures treated as stop-ship issues
• Human handoffs are timely, accurate, and carry enough context to help the customer rather than restart the conversation
• New intents move from evidence design through safe release using repeatable evaluation, tool, observability, and rollout infrastructure
• Quality, runtime reliability, latency, cost, privacy, and customer effort remain visible as the product grows
• At least one valuable multi-step workflow has an evidence-backed operating model-automated, AI-assisted, or deliberately human-owned-with clear authority, auditability, and recovery

You Might Thrive Here If

• You have personally owned a production software product, including an LLM-backed capability beyond a prototype
• You are a strong product engineer who can build across customer experience, application code, backend systems, AI behavior, and production operations
• You understand retrieval, context selection, structured outputs, tool use, orchestration, evaluation, and observability-and know when simpler, deterministic software is the better tool
• You can turn ambiguous user needs into explicit evidence, state, authority, and failure boundaries
• You have designed for unsupported claims, uncertainty, permissions, privacy, human escalation, rollback, and recovery
• You use representative evidence to make ship, revise, or stop decisions rather than optimizing demos or one aggregate score
• You enjoy learning a consequential domain and working directly with Product, Support, Security, and engineering partners
• You use modern AI development tools fluently and verify their output with the same rigor you apply to product behavior

This Role May Not Be for You If

• You want to focus primarily on model research, prompt iteration, or AI infrastructure without owning the complete customer and production outcome
• You believe more autonomy, more model calls, or a more sophisticated agent framework is inherently better
• You prefer to hand off evaluation, security, observability, or production operation after a prototype works
• You want a company-wide AI charter rather than focused ownership of the Dissolution product

Bonus

• Experience building customer-support, operations, or multi-step workflow agents
• Experience with LangGraph, LangChain, or comparable orchestration approaches
• Experience with typed tool protocols, retrieval systems, golden datasets, offline evaluation, shadow deployments, or model-based judges
• Experience with privacy-sensitive, multi-tenant, audited, legal, financial, or other high-trust products
• Strong Python plus TypeScript, React, Node.js, or comparable full-stack experience