Bengali Language AI Response Evaluator

Evaluate AI-generated responses in Bengali, identify factual errors and areas for improvement, and produce clear English-language analyses that will help shape higher-quality model outputs. This role combines fact checking, qualitative judgment, and written feedback to improve conversational AI behavior.

• Perform fact checking using trusted public sources and external tools.
• Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
• Assess reasoning quality, clarity, tone, and completeness of model responses.
• Ensure model outputs align with expected conversational behavior and system guidelines.
• Write evaluation artifacts and feedback in English, with precise, reproducible comments.

• Bachelor''s degree.
• Native fluency in Bengali, strong proficiency in English, with excellent English writing skills to articulate nuanced feedback.
• Significant experience using large language models, with an understanding of how and why people use them.
• Strong attention to detail, able to notice subtle issues others may miss.
• Background or experience in domains that require structured analytical thinking, for example research, policy, analytics, linguistics, or engineering.

• Prior experience with RLHF, model evaluation, or data annotation work.
• Experience writing or editing high-quality written content.
• Experience comparing multiple outputs and making fine-grained qualitative judgments.

• Consistently identify factual inaccuracies, reasoning errors, and communication gaps in model responses.
• Produce clear, consistent, and reproducible evaluation artifacts.
• Provide feedback that leads to measurable improvements in response quality and user experience.

• Location, remote, global.
• Contract work, hourly engagement.

• Pay rate: 15 - 20 hourly.

• Fluent language skills required: Bengali, native fluency, and English, strong proficiency.
• To be considered for this role you must take the Bilingual Competency interview in Bengali, this is a required step in the selection process.