Data Scientist for AI Model Evaluation

Apply deep domain expertise to train and evaluate next-generation AI systems by producing, refining, and validating high-quality, domain-informed data. This part-time contractor role focuses on improving model outputs through careful content review, prompt refinement, rubric-based evaluation, and clear written reporting. No prior AI employment is required, domain knowledge and rigorous analytical skills are the priority.

• Review, edit, and refine AI-generated content and data for accuracy, clarity, and domain relevance against project-specific rubrics.
• Develop and optimize prompts to guide model behavior, using professional writing and technical documentation skills.
• Perform rubric-based evaluations of AI model outputs, provide structured feedback, and recommend improvements.
• Annotate data, fact-check outputs, and contribute to quality assurance to ensure adherence to analytic standards.
• Conduct independent research to validate facts and improve data quality for model training and evaluation.
• Interpret complex datasets or findings and summarize them into clear, actionable reports and technical summaries.
• Collaborate asynchronously with project leads and fellow domain experts to share insights and best practices.

• Required skills: critical thinking, analytical reasoning, attention to detail, quality assurance, written communication, technical documentation, prompt authoring and refinement, AI output evaluation, professional writing, report writing, business communication, problem solving, content review, data interpretation, logical reasoning, professional editing, independent research, data annotation, rubric-based evaluation, content evaluation, AI model evaluation, fact checking, and technical editing.
• Preferred experience: 3+ years in Data Science, Machine Learning, Applied AI, Statistics, Quantitative Analytics, or Data Analytics.
• Proven record of producing or reviewing research papers, analytical reports, experiment summaries, notebooks, or technical documentation.
• Experience with data annotation, content review, or rubric-based evaluation is highly desirable.
• Background in prompt engineering, AI output evaluation, fact checking, or RLHF is advantageous but not required.
• Advanced degrees such as a Master’s, JD, MBA, or PhD are preferred; contributors from highly selective universities or leading technology companies are valued.

• Engagement type: Independent contractor, part-time.
• Location: Remote.
• Work is project-based and focused on a customer initiative to advance AI technology; tasks are completed asynchronously and in collaboration with project leads and other experts.

• Pay range: $100 to $200 per hour.

• No prior AI employment required, domain expertise and strong analytical and writing skills are the primary criteria.
• Candidates will be screened and vetted through the platform’s talent selection process, which uses an AI-driven recruiter to assess fit for specific projects.
• Selection for assignments depends on project needs and demonstrated fit to the rubric and domain requirements.