Evals Specialist

This contractor will support the Frontier Evals team in executing a high-priority evaluation campaign by monitoring the end-to-end data collection process, performing quality control on incoming data, and helping maintain throughput without compromising standards. The role requires fluency in one of Consulting, Finance, Data Science, Software Engineering, or Legal so the contractor can reliably judge whether tasks, deliverables, and rubrics reflect expectations for high-quality work in those fields. Experience in multiple domains is a plus! A passion for professional artifacts quality (docs, spreadsheets, decks, and clear and compelling writing) is a must.

Is your CV ready If so, and you are confident this is the role for you, make sure to apply asap.

Scope Of Responsibilities

Incoming Data Quality Control (QC)

• Review incoming tasks, deliverables, and rubrics to confirm they meet campaign standards.
• Identify common failure modes
• Provide actionable feedback to contributors and reviewers, and track campaign progress.

Data Collection Process Monitoring

• Monitor campaign health and throughput, including submission volume, rework rates, bottlenecks, turnaround time, and recurring quality issues.
• Surface systematic issues to Frontier Evals leads and propose solutions
• Maintain lightweight QC logs, issue trackers, or audit trails to support retrospectives, analysis, and process improvement.

Contributor Support and Coordination (as needed)

• Answer contributor questions to improve quality
• Coordinate with reviewers and subject matter experts on edge cases and escalate ambiguous decisions
• Help calibrate contributors on what "good" looks like for tasks and rubrics in the target domain

Technical Work (light to moderate)

• Use Codex, spreadsheets, light SQL, or simple Python scripts
• Assist with basic data hygiene and organization of uploaded materials, naming conventions, and record completeness.

Additional Tasks (as Needed)

• Draft or revise campaign documentation, examples, QC checklists, and "good vs. bad" examples for the target domain
• Participate in targeted xhyhwjd QA sweeps

Qualifications / Profile

• 3+ years of professional experience in the target domain
• Passion for docs, spreadsheets, decks, and clear and compelling writing
• Strong judgment about what realistic, high-quality work looks like in that field
• Excellent attention to detail and comfort applying standards and rubrics consistently
• Able to write clear, direct feedback that improves quality quickly
• Comfortable working in spreadsheets; familiarity with basic SQL or Python is helpful but not required
• Able to work in ambiguous, fast-moving environments and make pragmatic judgment calls

Pay Rate: $105-$120/hour