AI Safety Red Teamer, English and Bengali

Role Overview

Help make conversational AI systems safer by testing them with adversarial inputs, identifying vulnerabilities, and creating high-quality red-team data. This text-based remote role involves evaluating AI outputs, including outputs related to sensitive topics such as bias, misinformation, and harmful behavior. Participation in higher-sensitivity projects is optional; topics are communicated in advance, with clear guidelines and wellness resources available.

Key Responsibilities

• Test conversational AI models and agents for jailbreaks, prompt injections, misuse cases, bias exploitation, and multi-turn manipulation.
• Review AI responses for accuracy, completeness, appropriateness, and potential safety issues.
• Annotate failures, classify vulnerabilities, and flag systemic risks.
• Follow testing taxonomies, benchmarks, and playbooks to ensure consistent evaluation.
• Create reproducible reports, datasets, and attack cases that can be used to improve AI systems.

Qualifications

• Native fluency in both English and Bengali is required.
• Strong judgment about language and content, including the ability to explain why an AI response is accurate, incomplete, or inappropriate.
• Careful attention to subtle errors, inconsistencies, and gaps.
• Ability to work consistently within guidelines and quality standards.
• Clear written communication for technical and non-technical audiences.
• Comfort adapting across projects, task types, and customer needs.

Preferred Backgrounds

• Adversarial machine learning, including jailbreak datasets, prompt injection, RLHF or DPO attacks, or model extraction.
• Cybersecurity, including penetration testing, exploit development, or reverse engineering.
• Socio-technical risk evaluation, including harassment or misinformation probing, abuse analysis, or conversational AI testing.
• Psychology, acting, or writing that supports unconventional adversarial thinking.

Work Terms

• Remote, hourly engagement.
• All work is text-based.

Compensation

• $16 to $22 per hour.