Linguist - AI Rater Guidelines Expert
$45 - $65 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Rater Guidelines Writer (Linguist / Instructional Designer)
Type: Contract
Compensation: $45–$65/hour
Location: Remote
Role Responsibilities
- Guide research and program teams to close the gap between ambiguous program requirements and rater-ready instructions across various domains.
- Design clear, non-contradictory rating guidelines and rubrics for consistent application, including under edge cases.
- Evaluate and revise draft guidelines for ambiguity, contradiction, and coverage gaps to ensure raters can apply them without escalation.
- Translate program specifications from domains such as finance, retail, insurance, legal, and sports into precise, discipline-specific rater instructions.
- Collaborate with subject matter experts and program leads to ensure consistency and accuracy across guideline sets.
Qualifications
Must-Have
- 3+ years of professional experience in linguistics, instructional design, technical writing, or a related field.
- Direct experience writing or refining guidelines/rubrics for human raters in a GenAI/RLHF context.
- Ability to work across multiple domains (e.g., finance, retail, insurance, legal, sports) and translate domain-specific nuances.
- Strong track record of resolving ambiguity and contradiction in written specifications.
- Ability to engage reliably for at least 35 hours/week during weekdays.
- Strong written communication skills to explain complex guidance clearly.
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 7 days ago
Similar jobs that could be interesting for youBased on the Linguist - AI Rater Guidelines Expert in San Francisco, CA vacancy
$45 - $65 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: AI Rater Guidelines Writer (Linguist / Instructional Designer) Type:... .... Collaborate with subject matter experts and program leads to ensure consistency...SuggestedContract workSummer workRemote workWeekday workMercor
San Francisco, CA15 days ago- ...TwelveLabs builds the intelligence layer to change that. Our multimodal AI models understand video the way humans do — across sight, sound,... ...partners executing at scale, keeping quality high through clear guidelines and feedback loops. Cross-Functional Partnership: Work closely...SuggestedWork at officeRemote workFlexible hours
Twelve-Labs
San Francisco, CA5 days ago $80 - $150 per hour
We are partnering with a leading AI research organisation to develop a clinician-informed... ...AI-generated conversations and provide expert judgment to calibrate automated... ...therapeutics. Experience developing clinical guidelines, assessment frameworks, or educational materials...SuggestedHourly payTraineeship10 hours per week- ...Exists At Mercor, we believe the safest AI is the one that’s already been attacked —... ...a red team for this project - human data experts who probe AI models with adversarial inputs... ...is optional and supported by clear guidelines and wellness resources. Before being exposed...SuggestedRemote work
- YO AI Labs is seeking a Pharmacovigilance Expert to contribute drug safety expertise to a healthcare AI project. You will review pharmacovigilance documentation... ..., and compliance with ICH E2F, ICH E2C(R2), and GVP guidelines. Remote contractor arrangement with flexible...SuggestedRemote jobFor contractorsFlexible hours
- IQVIA in the United States is recruiting experienced mental health professionals to join our global Central Independent Raters network for a depression study. This is a 100% remote, project-based role offering flexible hours. Candidates will administer, score, review,...Remote jobFlexible hours
- Infinitus Systems, Inc. is seeking a Computational Linguist to improve and maintain language processing models. You will be instrumental... ...collaborating closely with various stakeholders. With a focus on AI solutions for the healthcare sector, this role requires expertise...
- ...experienced Adult Inpatient Nurses (RNs) to help train and evaluate AI systems used in clinical settings. You will translate bedside... ...inpatient experience, and proven ability to follow detailed annotation guidelines while meeting tight deadlines. #J-18808-Ljbffr Obsidian
$29 - $45 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Portuguese (global) Type: Contract Compensation...Contract workSummer workRemote workMercor Inc
San Francisco, CA3 days ago- ...YO AI Labs is seeking a Hematology Expert for remote contractor work. You will contribute clinical expertise in Oncology and Hematology to evaluate trial data, review regulatory documents, and provide expert medical judgments for advanced AI healthcare projects. The...Remote jobFor contractors
- Turing is seeking experienced Accountants and Auditors to train AI systems by creating and refining realistic accounting tasks and ensuring... ...to professional standards. You will produce high-quality expert tasks, review outputs, and escalate edge cases to the SME Lead. This...For contractors
- Obsidian is seeking clinicians and researchers to help design a clinician-informed benchmark evaluating how AI chatbots respond to adolescents facing mental health challenges. You will author realistic case scenarios, review clinical realism and ethics, and help calibrate...
- We are seeking experienced AI Safety Red Teamers to identify vulnerabilities in frontier AI systems through adversarial testing. You will design challenging prompts, uncover model weaknesses, and evaluate AI behavior across complex, high-risk, and ambiguous ("grey-area...
$80 - $150 per hour
A leading behavioral health consulting firm is seeking Senior Behavioral Health Experts to work part-time and remotely on frontier AI research projects. You will be responsible for designing evaluations and testing AI systems in critical mental health contexts. The ideal...Hourly payPart timeRemote work- YO IT Consulting is seeking a Chemistry Expert to deliver authoritative responses to advanced chemistry queries for AI model improvement. The role emphasizes clarity, depth, and rigorous justification across diverse chemical topics, from foundational to specialized areas...Remote jobContract work
- YO AI Labs in the United States (Remote) seeks seasoned Adobe Marketing Technology Experts to support an AI training and evaluation project focused on enterprise marketing operations. You will test workflows with Adobe Workfront, AEM, CJA, Analytics and Experience Cloud...Remote job
$60 per hour
Prolific seeks Chemistry Experts and Chemical Engineers to join our Expert Network and evaluate AI models using your chemical expertise. Successful candidates will be invited to assess and review AI-generated chemistry tasks and ensure their accuracy. Compensation can reach...Remote jobHourly payFlexible hours$60 per hour
Prolific is seeking Accountant Subject Matter Experts to join their Expert Network to provide professional insights in AI training. Responsibilities include evaluating financial documents, providing AI model guidance, and ensuring data integrity. The ideal candidates will...Remote jobHourly payWork from homeFlexible hours- Prolific is seeking Mental Health Professionals in San Francisco to train and evaluate cutting-edge AI models. Candidates must have verified status as a Mental Health Professional and a strong understanding of psychology. Responsibilities include reviewing AI-generated...Remote jobWork from homeFlexible hours
$70 - $84 per hour
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position AI Safety Red...Contract workSummer workRemote work- Mercor is seeking an independent contractor for their Benchmark Dataset Project, focusing on evaluating AI models in visual document understanding and instruction-following. This fully remote position offers flexibility in scheduling and competitive pay. You'll collaborate...Remote jobFor contractors
- Turing, based in San Francisco, invites Employment and Labor Law experts to join projects that fine-tune AI models. You will assess discrimination, wage and hour, ERISA, and labor relations using a structured rubric. No AI experience required; your U.S. employment law...Remote jobFlexible hours
- Mercor is seeking a remote Physics PhD to tackle complex physics problems and ensure the accuracy of AI-generated solutions. The role involves independently solving physics challenges and collaborating with AI research teams. Ideal candidates will have a PhD in Physics...Remote jobImmediate start
- Mercor is hiring PhD and Master's scientists to author AI evaluation tasks (Sci Code) Mercor is partnering with leading AI labs on a new benchmark for scientific computing. You will author original, executable research problems that today's frontier models cannot solve...Part timeImmediate start
- Mercor is seeking experienced musicians to evaluate generative music AI models. You will compare AI-generated lyrics with published songs across genres and rate them against detailed quality standards in Greek and English. The role requires native or near-native Greek,...Remote workFlexible hours
- YO AI Labs is seeking an Autodesk Fusion Expert for a remote contractor engagement focused on AI-driven manufacturing workflows. You will apply Fusion 360, CAM, CNC machining, and related digital manufacturing know-how to evaluate processes and AI-generated recommendations...Remote jobFor contractors
$80 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...of AI-generated conversations and provide expert judgment to calibrate automated evaluation... ...therapeutics. Experience developing clinical guidelines, assessment frameworks, or educational...Contract workTraineeshipSummer workRemote work$24 - $35 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: AI Safety Experts — English & Thai Type: Contract... ...in higher-sensitivity projects with clear guidelines and wellness resources. Qualifications...Contract workSummer workRemote work$48 - $62 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Danish Type: Contract Compensation: $48–$...Contract workSummer workRemote workMercor
San Francisco, CA5 days ago- ...Job Description Job Description Job Title: AI Consulting Domain Remote Job Type: Contractor (Part-Time) Location: Remote... ...Job Overview We are seeking experienced AI Consulting Domain Experts to contribute their consulting expertise to an innovative project...Remote jobPart timeFor contractors
YO AI Labs
San Francisco, CA19 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Linguist - AI Rater Guidelines Expert. Be the first to apply!
Related searches
- linguistic researcher San Francisco, CA
- computational linguist San Francisco, CA
- linguist San Francisco, CA
- appraiser San Francisco, CA
- rater San Francisco, CA
- subject matter expert San Francisco, CA
- fulfillment expert San Francisco, CA
- guest service support expert San Francisco, CA
- technology expert San Francisco, CA
- global linguist solutions