Senior AI Trainer

Help improve next-generation AI systems by evaluating model behavior, creating challenging prompts, and producing precise training feedback. This remote contract role focuses on structured evaluation of AI chat and search tools, with an emphasis on accuracy, consistency, and high-quality annotation. Key Responsibilities

• Evaluate AI chat and search tools under standardized, repeatable testing conditions.
• Apply detailed grading guidelines to evaluate, rank, and annotate model responses.
• Provide structured feedback designed to improve model performance.
• Write, refine, and stress-test prompts that explore model capabilities across diverse scenarios, edge cases, and user intents.
• Verify the consistency and accuracy of evaluations and correct discrepancies between assigned ratings and applicable rubrics or guidelines.
• Use AI training and evaluation tools to record findings and submit evaluation datasets according to project standards and milestones.
• Collaborate with trainers and other contributors to resolve ambiguities, refine evaluation guidelines, and maintain consistency across the project.
• Participate in ongoing quality reviews and incorporate feedback to maintain rigorous annotation and training standards.

• Exceptional attention to detail and accuracy when reviewing and evaluating AI-generated content.
• Strong written and verbal communication skills for collaboration, reporting, and timely updates regarding progress and challenges.
• Demonstrated time management and self-organization skills for independent remote project work.
• Analytical and problem-solving abilities, including a proactive approach to resolving evaluation challenges.
• Relevant skills may include video annotation, data labeling, use of video annotation tools, attention to detail, and time management.

• Experience with AI model training, reinforcement learning from human feedback, data labeling, or similar data-centric evaluation projects.
• Familiarity with AI products and tools, including prompt engineering and an understanding of model behaviors and limitations.
• Experience creating evaluation rubrics, grading model outputs, or reviewing AI responses for accuracy, helpfulness, and safety.
• Background in AI training, machine learning data preparation, or projects focused on large language models is strongly preferred.

• Remote independent contractor engagement.
• Role is available to candidates located in North America and Germany.

Hourly compensation ranges from $14 to $36.

Candidates apply through the online application process. Continuing with an application requires acknowledgment of the candidate terms and privacy policy.