AI Benchmark and Task Designer [Remote]

AI Benchmark and Task Designer is a remote review track for evaluating AI outputs across ai benchmark and task designer creative review workflows. Reviewers grade craft, constraints, and tooling adherence; flag production-readiness issues; and document the corrected approach so the modeling team can train on it.

Why this role matters

AI Benchmark and Task Designer creative review AI is only useful if a working professional would actually ship the output. AuraOne uses portfolio-credentialed reviewers to grade outputs the way a senior creative would — with craft, constraints, and tooling all in scope.

Responsibilities

• Review AI outputs against current ai benchmark and task designer creative review craft and tooling standards for AI Benchmark and Task Designer assignments.
• Grade composition, constraint adherence, and production-readiness on a structured rubric.
• Flag tooling-specific failures (file format, scale, naming, layer hygiene) with severity tags.
• Capture the corrected approach or reference so the modeling team can train on it.
• Adjudicate disputed creative judgments against published references or firm style guides.
• Maintain reviewer-quality scores in inter-rater calibration cycles.

Qualifications

• Portfolio or production experience in ai benchmark and task designer creative review or an adjacent creative discipline for AI Benchmark and Task Designer work.
• Hands-on familiarity with the standard tooling for the discipline.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the craft issue and the standard being applied.
• Reliable async availability for at least 10 hours per week.

Example tasks

• Grade a model's ai benchmark and task designer creative review output against the brief and rate craft, constraints, and production-readiness.
• Flag a tooling-specific failure (incorrect file format, unsupported feature, naming convention) with severity.
• Adjudicate a disputed creative judgment between two reviewers using a firm style guide.
• Audit a 25-asset batch for rubric consistency and report drift to the program lead.

Nice to have

• Studio, agency, or in-house team experience shipping production work.
• Prior experience reviewing AI-assisted creative tooling and its failure modes.
• Familiarity with AAA or feature-grade pipelines and asset standards.

Skills

• Creative review
• Production-grade craft
• Tooling fluency
• Asset evaluation
• AI Benchmark and Task Designer creative review
• Cross-cutting evaluation roles
• AI evaluation
• Rubric writing
• Expert review

Work model

Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

Hourly rate confirmed after the interview process.

Application process

Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.