Software Engineer, Agents (Internal Audit)

Responsibilities

• Run error analysis on real testing data and turn findings into fixes that make agent judgment repeatable.
• Build structured-output pipelines that convert model output into audit artifacts.
• Make quality, latency, cost, model-selection, prompting, and orchestration tradeoffs across long multi-phase runs.
• Turn ambiguous problems into plans, shipped features, and clear decisions about scope.
• Work with subject-matter experts and design-partner firms to improve agents and expand coverage into new controls and internal-audit areas.
• At higher levels, own major agent areas, establish evaluation and reliability practices, mentor engineers, and influence technical strategy.

Requirements

• Must have shipped LLM-backed product features to production for real users.
• Must have applied AI experience with evaluations, error analysis, and model-selection decisions.
• Must be comfortable working full-stack with sufficient backend depth for agent orchestration.
• Must be able to work autonomously from ambiguous specifications and collaborate across PM, design, and domain experts.
• Preferred experience includes Python, TypeScript, React, Postgres, Hasura, GraphQL, Temporal or comparable workflow orchestration, and evaluation tools such as Langfuse, Braintrust, LangSmith, or Arize Phoenix.
• Preferred qualifications include structured-output and artifact-generation experience, startup or 0-to-1 experience, direct customer collaboration, regulated-domain knowledge such as internal audit or SOX, and document processing involving PDF and Excel.

Benefits

• Competitive compensation with equity.
• Comprehensive health and wellness benefits, including wellness benefits beginning on the first day.
• Flexible time off and work schedules.
• Technology reimbursements.
• 401(k) plan.
• Twice-yearly in-person offsites across the United States.