AI Safety Specialist - Evaluation Expert
$60 - $70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Safety Practitioner
Type: Contract
Compensation: $60–$70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
- Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
- Apply and refine evaluation rubrics for RLHF , SFT , and AI safety benchmarking .
- Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
- Provide structured feedback to improve model alignment and safety performance.
- Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.
Qualifications
Must-Have
- Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline.
- 5+ years of professional experience in AI Safety , Trust & Safety, journalism, public policy, scientific research, security, or a related field.
- Excellent written English , critical thinking, and analytical reasoning skills.
- Ability to consistently evaluate nuanced and policy-sensitive scenarios.
Preferred
- Experience with AI Safety , RLHF , SFT , Trust & Safety, or AI evaluation.
- Familiarity with safety policies, content moderation, or evaluation rubric development.
- Experience reviewing complex, high-risk, or ambiguous content.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
- Invisible Technologies Inc. seeks an AI QA trainer to harden model reasoning and reliability... ...failure modes to raise standards for safety, bias checks, and prompt robustness. As a... ...with researchers and engineers to evaluate LLM outputs, identify edge cases, and propose...SuggestedRemote jobFor contractorsFlexible hours
$37 per unit
...technical talent with leading AI research labs. Headquartered... ...Jack Dorsey . Position: AI Safety Specialist Type: Contract... ...Role Responsibilities Evaluate the accuracy and depth of AI... ...Collaborate with subject matter experts to ensure consistency, relevance...SuggestedContract workSummer workRemote work$35 - $42 per hour
...project availability Help Shape the Future of AI in Finance, Audit & Risk Volga Partners... ...join our growing network ofon-call AI Evaluation Specialistssupporting the development, training... ...’ on-call network of subject matter experts. As client projects become available,...SuggestedHourly payExtra incomePermanent employmentContract workTemporary workRemote workFlexible hours$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location...SuggestedContract workSummer workWork at officeRemote work- ...Position Title: Senior Computer Programmer – AI Evaluation Specialist Location: Remote (Continental U.S.) Company: JR Software Solutions... ...an AI Evaluation Specialist to determine the effectiveness, safety, usability, and readiness of enterprise AI tools for adoption...SuggestedPermanent employmentFull timeRemote work
$68 per hour
Registered Nurse AI Evaluation Expert (English) Use your nursing judgment to review AI patient education, triage guidance, clinical documentation... ...train and assess AI systems. Your work focuses on accuracy, safety, clarity, and alignment with nursing practice. Review AI-...Hourly payContract workPart timeFor contractorsRemote workFlexible hours$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...and Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work- ...education and research sector is seeking a Social Sciences PhD Expert for a remote part-time role, requiring strong analytical... ...The position involves creating historically relevant prompts, evaluating AI outputs, and contributing to research initiatives. Ideal candidates...Part timeRemote workFlexible hours
$8 - $65 per hour
...Are you an Arabic (Levantine) language expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into powerful... ...and suggest improvements to our prompt engineering and evaluation metrics. A Master's degree in Linguistics, Middle...Hourly payContract workFor contractorsFreelanceRemote work$8 - $65 per hour
...Arabic (Levantine) Language Expert - Freelance AI Trainer Project World Wide - Remote Are you an... ...looking for Arabic (Levantine) language specialists who live and breathe phonetics,... ...improvements to our prompt engineering and evaluation metrics. A Master’s degree in...Hourly payContract workFor contractorsFreelanceImmediate startRemote work$55 per hour
...Freelance Biology Expert with Python - AI Trainer 5 days ago – Be among the first 25 applicants... ...Do The Mindrift platform connects specialists with AI projects from major tech innovators... ...comprehensive scoring criteria to evaluate the accuracy of the AI's answers....Part timeFreelanceRemote work- ...we do at UNICEF. For every child, the right to safety How can you make a difference? The UNICEF Evaluation Office (EO) is commissioning a global evaluation... ...effectively in the future. The EO is seeking a Senior Expert in Organizational Decision-making for this...Remote jobWork experience placementWork at office
$30 per hour
AI Safety & Evaluation Specialist Role type: Contractor Location: United States Commitment: Part-time, project-based Duration: 6 weeks Compensation: $30 $60 USD/hour Required Skills privacy risk assessment data protection ai evaluation...Full timePart timeFor contractors- ...Job Description: Computational Biology & Bioinformatics Expert (PhD) Role Type: Contractor Location: Remote (Worldwide... ...their scientific expertise to the development and evaluation of next-generation AI systems. In this role, you will apply your knowledge of...Remote jobFor contractorsWorldwide
$8 - $65 per hour
Antibiotic Resistance Specialist - Freelance AI Trainer Project United States of America Are you a master of antibiotic resistance eager to shape... ..., and suggest improvements to our prompt engineering and evaluation metrics. A PhD in microbiology, molecular biology, or a...Remote jobHourly payContract workFor contractorsFreelanceImmediate start$55 per hour
Help evaluate AI responses and create biology prompts as a remote Biology Expert AI Trainer. The role is open to US candidates with graduate-level biology expertise and pays up to $55 per hour. Remote Hourly · $55/hr $55/hr Compensation Contract, part-time Engagement...Hourly payContract workPart timeRemote workFlexible hours$8 - $65 per hour
Agency seeks an Arabic (Levantine) language expert to shape AI training data and evaluate model performance. You will interact with the model in Levantine Arabic, document failures, and help improve prompt engineering and evaluation metrics. A Master's degree is ideal;...Remote jobHourly payContract workFor contractors$40 per hour
Prolific is seeking Neuroscience Experts to act as evaluators for a high-level AI verification project. You will review dense scientific literature to judge the accuracy and structural integrity of AI-generated graphical abstracts, ensuring that summaries faithfully reflect...Remote job- Agency is seeking a highly skilled Māori language specialist to help train large-scale AI models. You will review, annotate, and refine... ...you’ll work remotely from the United States, evaluating model performance, providing expert feedback, and helping shape linguistic...Remote jobFor contractors
$40 per hour
...create high-quality STEM training data for frontier AI models. Our data is used directly in training and evaluation pipelines at leading AI labs to improve model reasoning in technical domains. We are looking for experts in Physics to design rigorous, deterministic...Contract workPart timeRemote work$55 per hour
...collective intelligence to ethically shape the future of AI. What We Do The Mindrift platform connects specialists with AI projects from major tech innovators. Our... ...AI. Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers. Correct the...Part timeFreelanceRemote work$8 - $65 per hour
Agency seeks a Maltese Language Specialist - AI Trainer for a remote contract role. You will annotate Maltese content, evaluate AI outputs for accuracy, and provide linguistic guidance to improve model performance. Responsibilities include identifying error patterns, refining...Remote jobHourly payContract work$24 per hour
...Use your accounting and QuickBooks experience to help improve AI responses to real small-business and financial questions. In this remote freelance project, you'll evaluate accuracy, identify gaps and help build reliable standards for accounting-related AI tasks. A part...Part timeFreelanceRemote work$110 - $140 per hour
...Bring senior chemistry expertise to AI evaluation projects focused on reaction mechanisms, analytical methods, lab safety, and chemical evidence. Review model responses and... ...evidence. Pay $110–$140/hr Alpheva is a new expert network (launched September 2026); payout methods...Hourly payContract workRemote work- Handshake is looking for skilled QGIS users to support AI research through flexible, hourly contract work. You'll draw on hands-on GIS experience to evaluate AI-generated content and provide feedback that helps AI better understand GIS tasks and spatial data workflows....Remote jobHourly payContract workFlexible hours
- ...A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities...Immediate start
$65 - $70 per hour
...Biology Expert (PhD) — AI Safety is a remote red-team track for stress-testing AI systems against... ...team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before... ...evaluation Application boundary Creating a specialist profile records your experience and...For contractorsRemote work10 hours per week- ...Biology Subject Matter Expert (AI Training) About the Role We're looking for biology experts to help evaluate and improve AI systems trained on advanced scientific content. Your deep knowledge of biology, biochemistry, or biotechnology will directly shape how...Hourly payOngoing contractContract workFreelanceRemote workFlexible hours
$8 - $65 per hour
...Prolific is hiring Mental Health Professionals in New York to train and evaluate AI models. As a Domain Expert, you will be responsible for reviewing AI-generated responses, completing tasks related to psychology, and improving AI models based on your expertise. Pay rates...Hourly payRemote workWork from homeFlexible hours$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Fab Safety and Emergency Response Expert Type: Contract Compensation... ..., and safety documentation. Evaluate the accuracy and depth of safety protocols...Contract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Safety Specialist - Evaluation Expert. Be the first to apply!
- ai data scientist New York, NY
- ai scientist New York, NY
- technology expert New York, NY
- subject matter expert New York, NY
- guest service support expert New York, NY
- sql expert New York, NY
- fulfillment expert New York, NY
- safety internship New York, NY
- patient safety specialist New York, NY
- safety associate New York, NY



