Member of Technical Staff

Member of Technical Staff (QA Engineer - Agentic Systems)

Join a fast-growing AI startup transforming how pharmaceutical companies create and launch marketing campaigns. You'll be the first dedicated QA hire, owning quality across a platform powered by LLMs, agentic systems, and AI workflows. This is a highly technical role where you'll work directly with leadership and engineering to build the testing, evaluation, and quality infrastructure that keeps the platform reliable, compliant, and production-ready.

Responsibilities

• Build and own evaluation frameworks for LLM and agentic systems
• Create automated testing infrastructure across AI, backend, and frontend systems
• Design tests that identify agent failures, regressions, and compliance risks
• Own end-to-end testing using Playwright and manual exploratory QA
• Implement CI/CD quality gates, test automation, and release sign-off processes
• Partner with Engineers and leadership to define the company's quality standards

Requirements

• Strong experience with Python and building testing infrastructure
• Hands-on expertise with Playwright or modern end-to-end automation frameworks
• Experience testing LLM, ML, or non-deterministic systems
• Knowledge of evaluation methodologies including golden datasets, LLM-as-judge, and regression testing
• Strong understanding of quality metrics, statistical analysis, and error investigation
• Ability to operate independently and build processes from 0 → 1
• Agentic AI, LangChain, LangGraph, or LLM tooling experience strongly preferred

Why apply?

• First QA hire with ownership over the entire quality function
• Work on the cutting edge of AI and agentic systems
• Partner closely with an elite team backed by top healthcare and AI operators
• Competitive compensation with meaningful equity upside
• Health, dental, vision, 401(k), relocation support, and unlimited PTO
• Help shape a platform already trusted by 50+ pharmaceutical brands