Linguist for AI Model Evaluation

Apply your linguistics expertise to evaluate large language model outputs and help train next-generation AI systems. You will analyze model-human conversations for linguistic precision, assess adherence to instructions, and deliver clear, actionable feedback that guides model improvement.

• Carefully review and analyze conversations between humans and large language models for accuracy, clarity, and completeness.
• Assess LLM responses against defined instruction-following guidelines and rubrics.
• Provide detailed, constructive written feedback and, when requested, verbal insights about model output quality.
• Identify subtle errors and inconsistencies in grammar, usage, and context to ensure linguistic precision.
• Deliver concise, actionable recommendations that emphasize accuracy over creativity.
• Document findings and suggested improvements in clear, organized reports.
• Collaborate with project coordinators to clarify guidelines and resolve ambiguities in evaluation tasks.

• Required skills: linguistics analysis, grammar and usage evaluation, strong written communication, attention to detail, and experience with guideline-based evaluation.
• Native proficiency in U.S. English is required for this engagement.
• Advanced academic training or extensive professional experience in linguistics, language analysis, or a closely related field is preferred.
• Proven ability to produce nuanced linguistic assessments and clear written reports.
• Ability to provide professional written and verbal feedback in English.
• Experience with rubric- or guideline-based evaluation is preferred but not required.
• Availability to begin contributing immediately upon selection.

• Engagement type: Contractor, independent contractor relationship.
• Location: Remote, applicants must be located in the United States or Western Europe.
• Work focuses on language model evaluation tasks; no prior AI experience is required.

• Pay: $40.00 to $50.00 per hour.

• Must be a native U.S. English speaker for this project.
• Must be located in the United States or Western Europe.
• Must be available to start immediately if selected.