Senior Software Test Engineer

Explicitly requires vibe coding skills — the team uses AI coding tools (Claude Code, Cursor) as the primary development surface and relies on AI for evals and graders. About the Role Senior Software Test Engineer to build eval infrastructure and do hands-on exploratory testing for AI agents, enabling rapid, high-quality delivery by combining automation, production forensics, and close partnership with business stakeholders. Job Description Role Curative is hiring a Senior Software Test Engineer to drive quality and measurement for AI agents and production services. This role combines building eval and observability infrastructure with hands-on exploratory testing, production forensics, and close collaboration with business teams. Key Responsibilities Design, build, and maintain the eval platform: harnesses, datasets, graders, and CI integration to measure agent behavior (task success, tool-use correctness, drift, latency, cost). Perform hands-on exploratory testing across member, operations, and provider workflows, including unusual and edge paths. Validate that the delivered product matches stakeholder needs by working directly with business teams and extracting real workflows. Develop production signal, dashboards, and failure taxonomies; reproduce incidents, diagnose root causes, and fix or hand off precise remediation steps. Make pragmatic product and prioritization decisions under ambiguity; act as a de facto product/quality owner when needed. Requirements 5+ years in software quality, testing, or engineering with experience in both automation and exploratory testing. Strong fluency with LLM evaluation: golden datasets, programmatic graders, LLM-as-judge approaches, and regression suites for prompts and agent loops. Hands-on experience with AI agents and common failure modes (silent drift, tool misuse, compounding errors). Deep debugging skills: logs, traces, queries, and production forensics, with ability to fix issues in Python or TypeScript. Strong stakeholder communication and the ability to extract requirements from non-technical partners. Comfort working without mature processes, making fast trade-offs, and reviewing AI-generated code diffs. Excellent written communication and pragmatic quality judgment. Preferred Experience building eval or observability infrastructure from scratch. Familiarity with LLM observability tooling (LangSmith, Braintrust, Arize) or equivalent homegrown solutions. Experience defining agent guardrails and permissible unsupervised behaviors. Why this role You will be close to the business and own both the testing infrastructure and the product outcomes. The role offers wide responsibility across eval infrastructure, hands-on testing, production stakes, and direct collaboration with product and operations. Curative Health Plan (100% employer-covered medical premiums for you and 50% coverage for dependents on the base plan) $0 copays and $0 deductibles (with completion of Baseline Visit) Preventive and primary care, mental health support, one-on-one care navigation, chronic condition programs Maternity and family planning support Employer-provided life and disability coverage with supplemental options Flexible spending accounts Generous PTO policy and 11 paid company holidays 401(k) for full-time employees Up to 8–12 weeks paid parental leave (role eligibility-dependent) Python TypeScript Claude Code Cursor LangSmith Braintrust Arize Skills Test Automation Exploratory Testing Debugging Production Forensics Observability CI/CD Integration Regression Testing Stakeholder Communication Prioritization Product Judgment Written Communication Domain Expertise Experience Level Senior Employment Type Full-time 100% employer-covered medical premiums for employee (50% for dependents on base plan) $0 copays and $0 deductibles (with completion of Baseline Visit) Preventive and primary care Chronic condition programs (diabetes, weight, hypertension) Maternity and family planning support Employer-provided life and disability coverage (with supplemental options) Flexible spending accounts 11 paid annual company holidays 401K for full-time employees Up to 8–12 weeks paid parental leave (based on role eligibility) #J-18808-Ljbffr Vibehackers