Research Engineer

The Role

Raydar is recruiting for this role on behalf of our client. Design and build the measurement systems used to judge the quality of machine learning outputs. You will define the scoring approach, assemble labeled reference data and run evaluation pipelines, then feed the findings back into model and product decisions.

What You'll Do

Define quality measures that capture what a good result looks like across a wide variety of inputs and edge cases.

Build evaluation pipelines and harnesses that score output quality reliably and at scale.

Create and curate labeled reference data so that evaluation results can be trusted.

Connect evaluation findings back to model and product decisions.

Present results in a way that gives the whole team clear, actionable answers.

What We're Looking For

Proven experience building evaluations for LLM-based data extraction or data quality at scale.

Experience creating the scoring approach and the evaluation system yourself, rather than only running an existing benchmark.

Experience assembling human-labeled or verified reference data to compare model outputs against.

Evaluation work that clearly influenced model or product decisions, visible in your work history.

Strong Python skills.

A solid grounding in statistics for choosing and interpreting quality measures.

Confidence working on open-ended problems where there is no established way to measure success.

A bias toward shipping, measuring and iterating quickly.

Willingness to work on-site five days a week, now or after relocating before the start date.

Benefits

Health, dental and vision coverage

Life and disability insurance

Paid time off

Paid parental leave

Location And Work Model

San Francisco, CA, United States or Toronto, ON, Canada

On-site, five days a week

Full-time

Compensation, As The Posting Describes It

Equity

Retirement plan