Research Engineer, QC Automation

Job Description

Job Description

Location: San Francisco Bay Area, CA, US (On-site)
Schedule: Full-time
Salary: $100K–$200K

Company:
Our client is a Y Combinator-backed company building infrastructure to create reinforcement-learning training data and evaluations for frontier AI agents, along with a marketplace connecting that work to frontier labs. Its platform is used by frontier labs, Fortune 500 companies, and startups.

Description:
Our client is seeking a Research Engineer, QC Automation to automate quality control for training data created by companies using its infrastructure. This role will build systems that scale quality as the company meets continued strong demand, combining technical execution with strong judgment about data quality and genuine curiosity about unfamiliar domains.

Key Responsibilities:

• Create QC systems grounded in true understanding and human judgment without relying heavily on LLMs
• Define and enforce quality standards for training data
• Design experiments and metrics to grade agent outputs
• Partner with data vendors to diagnose agent failure modes, debug quality issues, and improve data-generation processes
• Build systems for auditing supplier datasets, including sampling strategies, rule-based and model-assisted validation pipelines, and feedback loops
• Integrate QC learnings into infrastructure tools and the data-vendor portal to reduce anomalies, inconsistencies, and edge cases

Qualifications:

Required:

• Proficiency in Python, Docker, and Linux environments
• Strong judgment about what good data means and how to measure it
• Genuine curiosity about unfamiliar domains and skill at asking the questions needed to understand them
• Experience building scalable data-validation pipelines or automated QA/QC systems without a prescribed roadmap
• Experience with benchmarks and evaluations
• Early-stage startup experience and independent execution

Technical aptitude and learning potential matter more than years of experience.

Preferred:

The following are considered strong signals:

• Knowledge of statistics
• Strong written and verbal communication
• Comfort designing metrics, experiments, and QA/QC processes
• Ability to construct tasks in new evaluations
• Comfort in unstructured problem spaces

Why Join Them?

• Build quality-control systems for reinforcement-learning training data and evaluations for frontier AI agents
• Help scale data quality as the company meets continued strong demand
• Work on infrastructure used by frontier labs, Fortune 500 companies, and startups
• Relocation and visa support are available for strong candidates