AI Safety Specialist - Evaluation Expert
$60 - $70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Safety Practitioner
Type: Contract
Compensation: $60–$70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
- Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
- Apply and refine evaluation rubrics for RLHF , SFT , and AI safety benchmarking .
- Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
- Provide structured feedback to improve model alignment and safety performance.
- Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.
Qualifications
Must-Have
- Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline.
- 5+ years of professional experience in AI Safety , Trust & Safety, journalism, public policy, scientific research, security, or a related field.
- Excellent written English , critical thinking, and analytical reasoning skills.
- Ability to consistently evaluate nuanced and policy-sensitive scenarios.
Preferred
- Experience with AI Safety , RLHF , SFT , Trust & Safety, or AI evaluation.
- Familiarity with safety policies, content moderation, or evaluation rubric development.
- Experience reviewing complex, high-risk, or ambiguous content.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: BI dashboards / performance reporting Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedContract workSummer workWork at officeRemote work$90 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...genuinely correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use of...SuggestedContract workSummer workLocal areaRemote work- ...What you'll do Test AI agents using privacy-focused scenarios... ...Apply privacy principles to evaluate complex or ambiguous cases.... ...AI agents and AI privacy or safety evaluations. Application process... ...projects. We work with specialists across fields such as engineering...SuggestedPart time
- Remote is seeking a Neuroscience Specialist - AI Trainer to help shape the future of AI by guiding neural models with rigorous neuroscience... ...document failures, and propose improvements to prompts and evaluation metrics. This contract role requires a master’s or PhD in...SuggestedRemote jobContract work
$24 per hour
...Use your accounting and QuickBooks experience to help improve AI responses to real small-business and financial questions. In this remote freelance project, you'll evaluate accuracy, identify gaps and help build reliable standards for accounting-related AI tasks. A part...SuggestedPart timeFreelanceRemote work$8 - $65 per hour
Are you an Arabic (Yemeni) language expert eager to shape the future of AI? Large-scale language models are... ...looking for Yemeni Arabic language specialists who live and breathe phonetics,... ...improvements to our prompt engineering and evaluation metrics. A Master's degree in...Hourly payContract workFor contractorsRemote work$8 - $65 per hour
...Caribbean/Belize/Guyana) language expert eager to shape the future of AI? Large-scale language models are evolving... ...Caribbean/Belize/Guyana) language specialists who live and breathe regional... ...to our prompt engineering and evaluation metrics. A Master's degree in Linguistics...Hourly payContract workFor contractorsRemote work- Logoslabs is seeking a Yemeni Arabic language expert to guide AI training and evaluation. You will interact with the model in Yemeni Arabic, assess linguistic accuracy and cultural relevance, and document error traces to improve model reasoning. The role emphasizes transparency...Remote jobHourly payFor contractors
- Remote is seeking a Mongolian Language Specialist to help train AI models. You will review and annotate Mongolian content, evaluate outputs for accuracy, and refine prompts and linguistic guidelines. Fluency in Mongolian and experience in translation, linguistics, language...Remote jobHourly payContract work
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: Software Engineer Expert Type: Contract Compensation: $... ...Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...Contract workSummer workWork at officeRemote work$130k - $180k
...professional to serve as our Senior Safety Domain Expert for Veeva Falcon. In this... ...shaping next-generation AI and automation tools. You... ...Verification: Actively review, evaluate, and verify AI-driven... ...with PV leads and processing specialists at early adopter organizations...Work at officeLocal areaWork from homeShift work$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Fab Safety and Emergency Response Expert Type: Contract Compensation... ..., and safety documentation. Evaluate the accuracy and depth of safety protocols...Contract workSummer workRemote work$60 per hour
Prolific seeks Chemistry Experts and Chemical Engineers to join our Expert Network and evaluate AI models using your chemical expertise. Successful candidates will be invited to assess and review AI-generated chemistry tasks and ensure their accuracy. Compensation can reach...Remote jobHourly payFlexible hours$84 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: AI Safety Red Teamer Type: Contract Compensation... ...hallucinations, and policy failures. Evaluate model robustness across misinformation, cyber...Remote jobContract workSummer work$37 per unit
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: AI Safety Specialist Type: Contract Compensation:... ...: Remote Role Responsibilities Evaluate the accuracy and depth of AI-generated...Contract workSummer workRemote work$80 - $120 per hour
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Role...Contract workSummer workWork at officeRemote work- Mercor is seeking a Biology / environmental science Evaluator for a remote contract role. You will assess AI-generated artifacts using domain-specific rubrics, identify errors, and provide structured feedback to improve content quality. Candidates should have 5+ years...Remote jobContract workWork at office
$150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: Machine Learning Expert Type: Contract... ...of 8 attempts. Focus on inference evaluation and systems work rather than heavy model...Full timeContract workSummer workRemote work- Synthires is seeking a detail-oriented Generalist - Research & AI Evaluation for remote, part-time contract. You will develop research questions and evaluation materials to benchmark advanced AI models, and you will gather and summarize information across diverse subjects...Remote jobContract workPart timeImmediate start
$293.5k
General Information Job Title Expert Senior Manager, AI Engineering Job ID 104335 Work... ...research, model experimentation, and evaluation design to production system... ...continual improvementBalance performance, safety, responsible AI principles, and cost...Permanent employmentFull timeApprenticeshipWork at officeLocal areaWork from homeHome office3 days per week- We Are:Accenture is helping companies use generative AI and semantic layer to reinvent their enterprise and optimize business functions... ..., and scenarios).As needed by the specific problem, you design, evaluate, and maintain ontologies.As a significant part of this role, you...Work experience placementLive inWork at officeLocal area
- ...Senior AI Engineer New York City or San Francisco | Hybrid... ...continuously improving clinical safety and accuracy. That means advancing... ..., retrieval algorithms, and evaluation infrastructure that allow... ...ground truth is incomplete, and experts disagree, and where the right...Full time
$8 - $65 per hour
...Back to jobs Guaraní Language Expert - Freelance AI Trainer Project World Wide - Remote... ...We’re looking for Guaraní language specialists who live and breathe regional dialects... ...improvements to our prompt engineering and evaluation metrics. A Master's degree in...Hourly payContract workPart timeFor contractorsFreelanceImmediate startRemote work$8 - $65 per hour
...Application for Arabic (Iraqi) Language Expert - Freelance AI Trainer Project at Meridial Back... ...looking for Arabic (Iraqi) language specialists who live and breathe phonetics, morphology... ...to our prompt engineering and evaluation metrics. A Master's degree in Linguistics...Hourly payContract workPart timeFor contractorsFreelanceImmediate startRemote work$90 - $140 per hour
...technical talent with leading AI research labs.... ...Responsible AI / Security Specialist Type:... ...responsible AI problems. Evaluate security and responsible... ...rubrics to assess the safety accuracy and harm... ...other subject matter experts to ensure consistency...Full timeContract workSummer workRemote workWeekday work$84 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Inorganic Chemistry & Synthesis Expert Type: Contract Compensation: $... ...0 hours/week Role Responsibilities Evaluate technically complex problems involving inorganic...Contract workSummer workRemote work$8 - $65 per hour
...Dutch (Suriname/Caribbean) Language Expert - Freelance AI Trainer Project at Meridial Back to... ...Dutch (Suriname/Caribbean) language specialists who live and breathe grammar, orthography... ...to our prompt engineering and evaluation metrics. A Master's degree in Linguistics...Hourly payContract workPart timeFor contractorsFreelanceImmediate startRemote work$30 per hour
...to $30/hour ## About SME Careers As AI systems become more sophisticated, they rely... ...captures the full picture of how expert work actually gets done, the sequence of... ...diverse subjects. Optimize AI Performance: Evaluate and rank AI responses to enhance the model...Hourly payFull timeContract workFreelanceLocal areaRemote workFlexible hours$48 - $62 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Swedish Type: Contract Compensation: $48–...Contract workSummer workRemote work$60 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...-truth outputs and objective rubrics. Evaluate AI models on visual document understanding... ...content . Collaborate with subject matter experts to ensure task relevance and accuracy....Contract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Safety Specialist - Evaluation Expert. Be the first to apply!
- ai data scientist San Francisco, CA
- ai scientist San Francisco, CA
- technology expert San Francisco, CA
- subject matter expert San Francisco, CA
- guest service support expert San Francisco, CA
- fulfillment expert San Francisco, CA
- safety internship San Francisco, CA
- patient safety specialist San Francisco, CA
- safety associate San Francisco, CA
- safety attendant San Francisco, CA








