Software Engineer, Agent Platform

Responsibilities

• Own AI-powered feature behavior across product surfaces, including quality, safety, variance, and failure modes.
• Design prompting, retrieval, routing, and tool-use strategies for non-deterministic systems.
• Build and maintain evaluation systems using statistical signals, distributions, trends, golden datasets, regression detection, and human feedback.
• Detect and resolve hallucinations, partial correctness, instruction drift, and context sensitivity.
• Implement guardrails, fallbacks, and degradation paths for unexpected model behavior.
• Partner with product and infrastructure teams on quality, cost, latency, robustness, and shipping decisions.
• Influence model selection, model behavior, and tool design for real user workflows.
• Set quality standards and mentor engineers on non-deterministic system design and evaluation.

Requirements

• 6+ years of professional engineering experience with ownership of complex production systems.
• Experience owning AI/LLM behavior beyond basic integration, including variance and failure-mode mitigation.
• Ability to reason about probabilistic systems and quality, cost, recall, precision, speed, and robustness tradeoffs.
• Experience designing or maintaining evaluation frameworks, golden datasets, regression detection, or human-in-the-loop feedback loops.
• Strong product intuition and ability to define quality for non-deterministic outputs.
• Ability to work independently in ambiguous problem spaces and establish standards others rely on.
• Bonus: production experience with RAG, agentic systems, or tool-using models.
• Bonus: familiarity with vector databases, embeddings, or retrieval pipelines.
• Bonus: exposure to fine-tuning, model routing, or post-training techniques.
• Bonus: experience building shared AI infrastructure or mentoring engineers.