Language Model Evaluator - Fully Remote | Upto $20/hr Part-time
$20 per hourMercor
San Francisco, CA
- Remote job
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist - English & Bengali
Type: Contract
Compensation: $15–$20/hour
Location: Remote
Role Responsibilities
- Conduct fact-checking using trusted public sources and external tools .
- Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
- Assess reasoning quality, clarity, tone, and completeness of responses.
- Ensure model responses align with expected conversational behavior and system guidelines.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree
- Native speaker in Bengali
- Significant experience using large language models (LLMs)
- Excellent writing skills in English
- Strong attention to detail
- Background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)
Preferred
- Prior experience with RLHF, model evaluation, or data annotation work
- Experience writing or editing high-quality written content
- Experience comparing multiple outputs and making fine-grained qualitative judgments
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 11 days ago
Similar jobs that could be interesting for youBased on the Language Model Evaluator - Fully Remote | Upto $20/hr Part-time in San Francisco, CA vacancy
$20 per hour
...Contract Compensation $15-$20/hour Location Remote Role Responsibilities Conduct... ...high-quality human evaluation data by identifying response... ...completeness of responses. Ensure model responses align with... ...Significant experience using large language models (LLMs) Excellent...Part timeRemote jobLanguageContract workSummer work$80 - $110 per hour
...most advanced AI models. Overview A leading... ...is building and evaluating frontier models and... ..., vision-language and multimodal reasoning... ...Python. This is a part-time W-2 employment... ...workforce. This role is fully remote within the United... ...at approximately 20 hours per week....Part timeRemote workLanguageHourly payWork at officeSaidGig
United States8 days ago$20 per hour
Base pay range $20.00/hr - $20.00/hr Helping AI Startups Hire 0→... ...& Europe Hourly Contract | Part-Time | Remote $20 per hour About the Role... ...with Mercor, is hiring Audio Model Trainers to support the development... ...and connect spoken language with visual information. This...Part timeRemote jobLanguageHourly payContract workInternshipFlexible hours$90 per hour
...A leading AI lab is building models that sharpen and upscale video... ...to AI output. This is a part-time W-2 employment position with... ...experience: in high-end visual evaluation across film, VFX, or professional... ...details into clear, precise language that can be captured in data...Part timeRemote jobLanguage$75 per hour
...This is a flexible, part-time opportunity that is 5-20 hours per week at... ...week). It can be done remotely and can be done... ...train their latest language model on clinical nursing... ...hospitals to create, evaluate, and refine AI-... ....5k/week) Location Fully remote, work on your...Part timeRemote workLanguageHourly payFull timeFlexible hoursEthos
New York, NY17 hours ago$80 - $110 per hour
...developing advanced AI models. This role... ...designing and evaluating machine learning and natural language processing tasks... ...for approximately 20 hours per week.... ...your own time. Work Terms This is a part-time W-2 employment... ...opportunity to work remotely within the...Part timeRemote workLanguageHourly paySaidGig
United States8 days ago$40 per hour
...forward company is seeking a Teacher Of Chemistry to train AI models by evaluating their outputs and improving their quality. The role allows flexibility as it can be either full-time or part-time and is fully remote. Applicants should possess expert-level chemistry...Part timeRemote jobHourly payFull time$20 per hour
...the flexibility of remote work and choosing your... ...prompts to test AI models, write high-quality... ...excellence, and evaluate different model outputs... ...is required. As part of the application... ...This is a full-time or part-time REMOTE... ...Projects start at $20+ per hour, with a...Part timeRemote workHourly payFull timeContract workFor contractorsSelf employmentFreelanceDataAnnotation
Wyoming, OH2 days ago$100 per hour
...This is a flexible, part-time opportunity that is 5-20 hours per week at... ...week). It can be done remotely and can be done... ...train their latest language model on financial management... ...to create, evaluate, and refine AI-generated... ...2k/week) Location: Fully remote, work on your...Part timeRemote workLanguageHourly payFull timeFlexible hoursEthos
New York, NY17 hours ago$20 - $27 per hour
...for a Primary School Educator to train AI models and evaluate their outputs. This role offers... ...work schedule, with payments ranging from $20 to $27 per hour. Ideal candidates should... ...United States will be considered for this remote position. #J-18808-Ljbffr DataAnnotationRemote jobLanguageHourly pay$60 per hour
...Analyst | $60/hr role at HelixRecruit... ...is shared as part of a referral... ...frontier large language models. Specifically,... ...create detailed evaluation rubrics for AI... ...project is 15-20 hours of work... ...extension. The work is fully remote and... ...of onboarding time including the screening...Part timeRemote workLanguageHelixRecruit
New York, NY17 hours ago$20 per hour
...chatbots in Hebrew and English. In this remote position, you'll evaluate chatbot outputs and craft... ...The ideal candidate is fluent in both languages and possesses strong grammatical skills... ...scheduling and competitive pay starting at $20 per hour are offered, making this an...Remote jobLanguageHourly payFlexible hours$14 - $20 per hour
ALTA Language Services, Inc. is seeking a remote Testing Evaluator in Atlanta, Georgia. This part-time, on-demand role involves assisting with language proficiency examinations, maintaining... ...degree. Pay ranges from $14.00 to $20.00 per test, depending on experience and...Part timeRemote jobLanguageImmediate start- ...apply their expertise to evaluate AI models and enhance their... ...understanding of medical tasks and language. Participate in a... .... Work Terms Part-time engagement with flexible hours. Remote work environment. Compensation Up to $300/hr (depending on the...Part timeRemote workLanguageHourly payTemporary workFlexible hours
SaidGig
United States7 days ago $20 per hour
...technology company specializing in AI is seeking a Copywriter to evaluate and improve AI model outputs. This role is remote, offering both full-time and part-time options with an hourly starting pay of $20+. The ideal candidate is detail-oriented, fluent in English, and...Part timeRemote jobHourly payFull timeContract work$30 per hour
Type: Hourly contract Compensation: $20-$30/hour Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous agent systems using defined rubrics and quality standards. Review multi-step agent workflows,...Remote jobLanguageHourly payContract work$20 - $30 per hour
...a specialised part-time consulting opportunity... ...in AI safety evaluation, red team... ...current and upcoming remote consulting... ...conversational model assessment, misuse... ...clear English-language evaluation... ...contractor role ~Fully remote with... ...rates between $20–$30 per hour depending...Part timeRemote workLanguageHourly payWeekly payContract workFor contractorsFlexible hours$10 - $20 per hour
...sharing a specialised part-time consulting... ...professionals experienced in language evaluation, LLM response... ...and upcoming remote consulting opportunities... ...to assess model response performance... ...contractor role. ~Fully remote with flexible... ...between $10–$20 per hour depending...Part timeRemote workLanguageHourly payWeekly payContract workFor contractorsFlexible hours$20 per hour
...company is seeking a Receptionist to assist in training AI models. This role involves evaluating the logic of AI chatbots and ensuring quality in their... ...flexibility with projects and an hourly pay starting at $20+ USD, with bonuses for high-quality work. Only candidates...Remote workHourly payFlexible hoursDataAnnotation
New York, NY3 days ago- ...apply their expertise to evaluate AI models and enhance their... ...industry-specific tasks and language. This role involves assessing... ...tasks. Work Terms Part-time, temporary engagement. Remote work flexibility.... ...Compensation Up to $300/hr (depending on the project...Part timeRemote workLanguageTemporary work
SaidGig
United States7 days ago $85 per hour
...5/hour (based on experience) Location: Remote (Global) Duration: Ongoing project (minimum... ..., potential extension) Commitment: Part-time to full-time, 15-40 hours/week (flexible... ...to the improvement of large-scale language model development workflows. Requirements Strong...Part timeRemote jobLanguageHourly payFull timeContract workFlexible hours$20.5 per hour
...Full-Time Maintenance Technician - $20.50/hr Job Category : Building Staff - Theatre Requisition... ...as either variable hour, part-time fixed, part-time... ...to work independently. Language Ability Effectively communicate... ...causes of problems. Evaluate relevant information,...Part timeLanguageHourly payFull timeRegal Entertainment
Port Charlotte, FL1 day ago$65 - $75 per hour
...Research Scientist (Remote | $65-$75/hr) Join to apply... ...advancing large language model (LLM) capabilities... ...design, solve, and evaluate real-world... ...Commitment: 10-20 hours/week, with... ...extend. Schedule: Fully remote and asynchronous... ...Employment type Part-time Job function...Part timeRemote jobLanguageContract work$100 per hour
...enhancing the capabilities of Large Language Models (LLMs) within specialized domains.... ...and collaboratively in a remote environment. Work Terms Part-time engagement with flexible hours.... ...arrangement. Compensation Up to $100/hr (depending on the project). Eligibility...Part timeRemote workLanguageFlexible hoursSaidGig
United States7 days ago$57.5k - $62k
...HEALINGTerros Health is hiring a part-time Family Nurse Practitioner for... ...the Phoenix area.Part-time; 20 hours/week2-10'sOak: 44th St... ...: $64,500-$65,000Additional Language Differential Pay Available:... ...physicals/check-ups, medical evaluations, diagnostics, treats illnesses...Part timeLanguageWork at office$50 per hour
...Position: Dutch Audio Generalist Evaluator Expert Type: Contract... ...: $50/hour Location: Remote Duration: 3–6 months Role... ...and English . Conduct model testing by running prompts and... ...Application Process (Takes 20–30 mins to complete) Complete...Remote jobContract workSummer work$55 - $70 per hour
...Research Scientist - STEM (Part-Time | $55-$70/hr) Mercor is... ...and depth of large language models (LLMs) across multiple... ...quality. Review and evaluate AI-generated responses... ...) Commitment: 10-20 hours/week (flexible... ...hours/week) Schedule: Fully remote and asynchronous —...Part timeRemote workLanguageHourly payContract workImmediate startFlexible hours$55 - $70 per hour
...Research Scientist (Part-Time | $55-70/hr) Mercor is collaborating... ...and depth of large language models (LLMs) across... ...quality. Review and evaluate AI‑generated responses... ...extension) Commitment: 10-20 hours/week (... ...hours/week) Schedule: Fully remote and asynchronous —...Part timeRemote workLanguageHourly payContract workInternshipImmediate startFlexible hours$20 per hour
Join to apply for the Language Specialist | $20/hr Remote role at Crossing Hurdles Base pay range $20/hour... ...to build and train cutting‑edge AI models. Type : Hourly Contract (Remote) Compensation... ...or distortion Follow linguistic, timing, or stylistic guidelines set by the...Remote jobLanguageHourly payContract workFlexible hours$57k - $62k
...their most challenging times, this vital opportunity... ...HealingTerros Health is hiring a part-time Family Nurse... ...area.Part-time; 20 hours/week2-10'sLocation... ...,500-$65,000Additional Language Differential Pay Available... .../check-ups, medical evaluations, diagnostics, treats illnesses...Part timeLanguageWork at office2 days per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Language Model Evaluator - Fully Remote | Upto $20/hr Part-time. Be the first to apply!
Related searches
- evaluator San Francisco, CA
- social media evaluator San Francisco, CA
- work from home web search evaluator San Francisco, CA
- quality evaluator San Francisco, CA
- language subtitle San Francisco, CA
- language consultant San Francisco, CA
- language manager San Francisco, CA
- natural language processing San Francisco, CA
- speech language San Francisco, CA
- language analyst San Francisco, CA