AI Safety Specialist - Evaluation Expert
$60 - $70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Safety Practitioner
Type: Contract
Compensation: $60–$70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
- Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
- Apply and refine evaluation rubrics for RLHF , SFT , and AI safety benchmarking .
- Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
- Provide structured feedback to improve model alignment and safety performance.
- Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.
Qualifications
Must-Have
- Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline.
- 5+ years of professional experience in AI Safety , Trust & Safety, journalism, public policy, scientific research, security, or a related field.
- Excellent written English , critical thinking, and analytical reasoning skills.
- Ability to consistently evaluate nuanced and policy-sensitive scenarios.
Preferred
- Experience with AI Safety , RLHF , SFT , Trust & Safety, or AI evaluation.
- Familiarity with safety policies, content moderation, or evaluation rubric development.
- Experience reviewing complex, high-risk, or ambiguous content.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on us.fitly.work
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the AI Safety Specialist - Evaluation Expert in New York, NY vacancy
$37 per unit
...technical talent with leading AI research labs. Headquartered... ...Jack Dorsey . Position: AI Safety Specialist Type: Contract... ...Role Responsibilities Evaluate the accuracy and depth of AI... ...Collaborate with subject matter experts to ensure consistency, relevance...SuggestedContract workSummer workRemote workMercor
New York, NY4 days ago$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location...SuggestedContract workSummer workWork at officeRemote work- ...education and research sector is seeking a Social Sciences PhD Expert for a remote part-time role, requiring strong analytical... ...The position involves creating historically relevant prompts, evaluating AI outputs, and contributing to research initiatives. Ideal candidates...SuggestedPart timeRemote workFlexible hours
Crossing Hurdles
New York, NY1 day ago $40 - $60 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Healthcare Administration Expert Type: Contract Compensation... ...deterministic rubrics for agent performance evaluation. Develop scenarios in patient access...SuggestedHourly payWeekly payFull timeContract workFor contractorsSummer workRemote workMercor
New York, NYmore than 2 months ago$73 per hour
...ethically shape the future of AI. What We Do The Mindrift platform connects specialists with AI projects from major tech... ...modern AI systems are tested and evaluated? This is a flexible, project... ...with less hand‑holding. Real expert complexity only. You're improving...SuggestedPermanent employmentPart timeFreelanceRemote workFlexible hoursMind Rift
New York, NY3 days ago$70 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Computational Particle & Nuclear Physics Expert Type: Contract Compensation: $... ...tools. Familiarity with benchmark or evaluation design. Background in scientific teaching...Contract workSummer workRemote workMercor
New York, NY4 days ago$400 per month
Obsidian is seeking contributors for a Frontier Code Agents project, focused on evaluating AI coding models in fraud and risk engineering. Candidates will use AI coding tools to handle complex tasks and provide technical assessments. The role requires 2+ years of experience...$55 per hour
Freelance Biology Expert with Python - AI Trainer 5 days ago - Be among the first 25 applicants This... ...We Do The Mindrift platform connects specialists with AI projects from major tech... ...Define comprehensive scoring criteria to evaluate the accuracy of the AI's answers....Part timeFreelanceRemote work$8 - $65 per hour
...you an Arabic (Levantine) language expert eager to shape the future of AI? Large-scale language models are... ...looking for Arabic (Levantine) language specialists who live and breathe phonetics,... ...to our prompt engineering and evaluation metrics. A Master’s degree in Linguistics...Hourly payContract workFor contractorsFreelanceImmediate startRemote work$55 per hour
Freelance Mathematics Expert - AI Trainer 1 week ago Be among the first 25 applicants This... ...We Do The Mindrift platform connects specialists with AI projects from major tech innovators... ...comprehensive scoring criteria to evaluate the accuracy of the AI's answers. Correct...Part timeFreelanceRemote work$274.3k
...role, you set the technical direction for AI engineering across strategic customer... ...enterprise wide initiatives. As a Chief Expert, you are the deepest technical authority... ..., bridging vision and delivery. Lead the evaluation and adoption of emerging AI capabilities...Permanent employmentFull timeWork at officeLocal areaWorldwideFlexible hoursSAP
New York, NY3 days ago$80 - $160 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Utilize strong analytical skills to contribute to high-impact evaluation projects. Collaborate with teams to improve document standards...Contract workSummer workRemote workMercor
New York, NY4 days ago- ...Biology Subject Matter Expert (AI Training) About the Role We're looking for biology experts to help evaluate and improve AI systems trained on advanced scientific content. Your deep knowledge of biology, biochemistry, or biotechnology will directly shape how...Hourly payOngoing contractContract workFreelanceRemote workFlexible hours
Alignerr
New York, NY1 day ago - ...A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities...Immediate start
Crossing Hurdles
New York, NY1 day ago - ...requires strong analytical abilities. The ideal candidate has a Bachelor’s degree, 2+ years of relevant experience, and excellent written communication. The position focuses on high-impact evaluation projects and structured guideline adherence. #J-18808-Ljbffr Obsidian
- ...relevance. This is a great opportunity for professionals with strong analytical skills and domain expertise to contribute to high-impact evaluation projects. Basic Qualifications Bachelor’s degree (minimum) At least 2+ years of professional experience in a relevant domain...
$84 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: AI Safety Red Teamer Type: Contract Compensation... ...hallucinations, and policy failures. Evaluate model robustness across misinformation, cyber...Contract workSummer workRemote workMercor
New York, NY4 days ago- ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Dermatologist (MD) — Clinical Image Interpretation & AI Evaluation Type: Contract Compensation: $270/hour Location:...Contract workSummer workRemote work
Mercor
New York, NY4 days ago $35 - $55 per hour
Biology Subject Matter Expert - AI Content Specialist $35-55/hr Remote Freelance STEM About the Role We're looking for biology experts to help train and evaluate the next generation of AI models. At Alignerr, we partner with the world's leading AI research labs — and your...Hourly payOngoing contractContract workFreelanceRemote workFlexible hours- Biology Subject Matter Expert (AI Training) We're looking for biology experts to help evaluate and improve AI systems trained on advanced scientific content. Your knowledge will directly shape how AI understands, reasons through, and communicates complex biology — making...Hourly payOngoing contractContract workFreelanceRemote workFlexible hours
$150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...independently and asynchronously to meet deadlines while improving AI evaluation processes. Qualifications Must-Have US-based disease-...Hourly payContract workFor contractorsSummer workRemote workMercor
New York, NY4 days ago$20 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Video Evaluation Generalist Type: Contract Compensation: $15–$20/hour...Contract workSummer workRemote workMercor
New York, NY4 days ago$60 - $80 per hour
...technical talent with leading AI research labs. Headquartered... ...Dorsey . Position: Retail Specialist Type: Contract... ...in real retail practice. Evaluate AI model outputs against structured... ...Collaborate with other subject matter experts to ensure consistency and...Contract workSummer workRemote workWeekday workMercor
New York, NY4 days ago- Obsidian is seeking a Medical Expert to help train AI models with high-quality healthcare reasoning data. You will design clinical scenarios, evaluate model outputs, and provide essential feedback to improve AI-driven patient care. This remote, asynchronous role defaults...Remote jobFlexible hours
- Mercor is partnering with a leading AI lab to design and evaluate disability adjudication processes for frontier AI models. You will craft realistic disability scenarios, generate determinations and evidence summaries, and assess AI-generated responses against established...
- ...Mercor, we believe the safest AI is the one that’s already been... ...for this project - human data experts who probe AI models with adversarial... ...customer AI systems Evaluation coverage expands: more scenarios... ...production Mercor customers trust the safety of their AI because you’ve...Remote work
- ...hiring PhD‑level biologists to help make advanced AI models safer. You'll apply your scientific expertise to evaluate and strengthen how these models handle... ...train you on the workflow. Responsibilities Write expert‑level prompts across specialized life‑science topics...Part timeImmediate start
$30 per hour
Prolific is seeking fluent Hindi speakers to join their Expert Network, helping to train and evaluate AI models with real legal expertise. Responsibilities include analyzing and writing tasks in Hindi, judging AI’s performance, and aiding in the improvement of AI models...Remote jobHourly payFlexible hours- Prolific in New York, NY, is seeking Chemistry Experts and Chemical Engineers to join its Expert Network. In this role, you will help train and evaluate AI models using your chemical expertise. Duties include evaluating AI-generated responses for accuracy and validating...Remote jobWork from homeFlexible hours
- Prolific is recruiting Mental Health Professionals to train and evaluate AI models as Domain Experts. You will be invited after a quick test and paid to complete AI tasks requiring focused hours. Expect earnings per task ranging with flexible hours and remote work. Successful...Remote jobFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Safety Specialist - Evaluation Expert. Be the first to apply!
Related searches
- ai scientist New York, NY
- ai data scientist New York, NY
- guest service support expert New York, NY
- fulfillment expert New York, NY
- subject matter expert New York, NY
- sql expert New York, NY
- technology expert New York, NY
- warehouse safety New York, NY
- safety training New York, NY
- safety scientist New York, NY