English (New Zealand) Audio Evaluator for AI Research [Remote]

Join a high-impact audio AI research project as an English (New Zealand) Audio Generalist Evaluator Expert. This role involves transcription, annotation, and evaluation tasks that are essential for training and benchmarking advanced language models. It is a structured, short-term engagement suited for individuals with strong academic or analytical backgrounds who are fluent in New Zealand English and excel at translating complex audio and visual information into precise, well-structured text. Key Responsibilities

• Transcribe and Optimise Audio & Video
  • Listen to, analyse, and transcribe audio and video content in English (New Zealand), adhering to detailed constraints and instructions.
  • Produce high-quality written outputs in English, with supporting work in English when required.
  • Ensure clarity, accuracy, and strict adherence to formatting and stylistic guidelines.
  • Capture nuances such as tone, intent, formal vs. informal register, regional expressions, Māori loanwords, and contemporary New Zealand English usage where relevant.
• Define and Document Evaluation Standards
  • Establish clear expectations for correct and high-quality responses in general consumer audio contexts.
  • Develop detailed evaluation rubrics and grading guidelines in English.
  • Document standards to ensure consistency across reviewers and model evaluations.
  • Identify linguistic nuances, grammatical patterns, colloquialisms, cultural references, and edge cases specific to New Zealand English.
• Conduct Model Testing and Grading
  • Run prompts through language models and assess generated outputs.
  • Evaluate responses against predefined criteria for accuracy, completeness, fluency, and instructional clarity.
  • Provide structured feedback to improve model performance in New Zealand English audio tasks.
• Support Benchmarking and Quality Assurance
  • Participate in QA and review cycles to ensure tasks, rubrics, and outputs meet quality standards.
  • Maintain consistency and reliability before datasets are integrated into official benchmarks.
  • Collaborate with project leads to resolve ambiguities and improve task design.

• Minimum Qualifications
  • Strong writing, editing, and critical thinking skills.
  • Ability to work independently, manage time effectively, and meet deadlines.
  • Native or near-native fluency in English (New Zealand) (spoken and written).
  • Strong familiarity with New Zealand English, regional vocabulary, accents, Māori influences, and contemporary language usage.
  • Ability to accurately transcribe and analyse English audio content across general consumer contexts.
  • Available to commit 10, 20 hours per week.
• Preferred Qualifications
  • College students or recent graduates.
  • Background in linguistics, humanities, social sciences, journalism, translation/localization, or technical disciplines.
  • Prior experience with transcription, annotation, localisation, evaluation, or research workflows in English.
  • Familiarity with differences between New Zealand English, Australian English, and other major English varieties.
  • Interest in AI, language models, or applied research environments.

Hourly engagement, remote work.

$50 per hour.

Must be fluent in New Zealand English and available for 10, 20 hours per week.