Psychologist for AI Model Evaluation

Psychology experts apply clinical and research knowledge to evaluate and improve large language models (LLMs) in psychological domains, designing prompts, assessing responses for accuracy and nuance, and contributing research-informed feedback that shapes future AI behavior.

• Create and refine domain-specific prompts and test tasks to probe LLM performance.
• Evaluate LLM outputs for factual accuracy, conceptual clarity, ethical considerations, and appropriate clinical framing.
• Conduct independent literature or topic research, using AI tools as an aid, to support task design and evaluation.
• Document findings and provide structured feedback to research teams to help improve model behavior.
• Participate asynchronously in project work, collaborating with research teams and a small cohort of domain experts when assigned to a project.

• Master''s degree from a US institution, or current master’s student in Psychology at a US institution.
• Subject-matter expertise in psychology strong enough to critically evaluate and often outperform current AI systems in explaining key concepts.
• Comfort working primarily in asynchronous formats and coordinating with research teams remotely.
• Selection for individual projects is competitive, with cohorts chosen for disciplinary and research expertise.

• Employment type: Part-time.
• Work is remote and asynchronous, allowing independent scheduling from any location.
• Flexible hours with no minimum required weekly commitment.
• Placement into specific projects depends on available projects in the relevant domain.

• Pay rate: Up to $55/hr (depending on the project).

• F-1 students may be eligible if they have CPT or OPT authorization; consult your Designated School Official to confirm eligibility.
• If your school requires CPT to be tied to a course, this program may not meet that requirement.
• STEM OPT is not supported for this program.

The program operates year-round, however project openings vary by domain and time. Assignment to projects is contingent on project availability and the fit between your expertise and project needs.

• Create a candidate account and submit your profile materials.
• Complete identity verification as instructed.
• Apply to or join relevant project openings and complete any project-specific onboarding when selected.
• Begin assigned work and receive compensation according to the project terms.