Bilingual Language Model Evaluator
$15 - $20 per hourRemote Jobs
About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey. Position: Generalist - English & Urdu Type: Contract Compensation: $15–$20/hour Location: Remote Role Responsibilities Conduct fact-checking using trusted public sources and external tools. Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies. Assess reasoning quality, clarity, tone, and completeness of responses. Ensure model responses align with expected conversational behavior and system guidelines. Work independently and asynchronously to meet deadlines while improving AI model performance. Qualifications Must-Have Bachelor's degree. Native speaker in Urdu. Significant experience using large language models (LLMs). Excellent writing skills in English. Strong attention to detail. Background or experience in domains requiring structured analytical thinking. Preferred Prior experience with RLHF, model evaluation, or data annotation work. Experience writing or editing high-quality written content. Experience comparing multiple outputs and making fine-grained qualitative judgments. Resources & Support For details about the interview process and platform information, please check: For any help or support, reach out to: #J-18808-Ljbffr Remote Jobs
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Bilingual Language Model Evaluator in New York, NY vacancy
- YO AI Labs invites a Tamil Bilingual Expert to contribute to a language and AI training project focused on Tamil understanding and generation. You will evaluate Tamil audio content, assess linguistic quality, provide feedback, and ensure alignment with guidelines. No prior...BilingualRemote jobFor contractorsFlexible hours
- AuraOne is seeking a Latin Bilingual Expert for a remote evaluation track. Reviewers compare paired outputs to a quality rubric, label edge cases, and generate structured feedback to retrain models. Responsibilities include evaluating outputs against rubrics, tagging issues...BilingualFor contractorsRemote work10 hours per week
- YO AI Labs is seeking Thai bilingual experts to evaluate Thai audio content and provide detailed feedback to improve AI language models. You will assess nativeness, fluency, and linguistic authenticity while documenting evaluations in English. Strong Thai proficiency and...BilingualRemote jobFor contractors
$135 - $190 per hour
Speech Language Pathologist Evaluator, Bilingual Spanish/English- Brooklyn, NY Job-Achieve Beyond Description: Apply to Achieve Beyond as a Speech Language Pathologist Evaluator, Bilingual Spanish/English in Brooklyn, NY and help children with special needs reach their...BilingualLocal areaFlexible hours$700 per day
...We're looking to hire NYS certified Speech Language Pathologists (SLPs), to conduct CSE (school-age) Speech and Language Evaluations in person. Job Type: Flexible Schedule... ...turnaround time. We have monolingual and bilingual evaluators in all areas of the city and we...BilingualFull timeContract workPart timeWork at officeLocal areaWork from homeFlexible hours$15 - $20 per hour
...seeking a Generalist proficient in English and Urdu to conduct evaluations for AI models from a remote location. The successful candidate will... ...attention to detail and significant experience with large language models. The compensation ranges from $15 to $20 per hour....BilingualRemote jobHourly pay- RWS TrainAI seeks a freelance part-time evaluator to review short-form video content and classify languages in both audio and on-screen text. You will validate locale... ...internal tools, ensuring high data quality for ML model training. Ideal candidates are based in Germany,...Remote jobPart timeFreelanceFlexible hours
$20 per hour
Prolific is seeking fluent Norwegian speakers to act as evaluators for AI training. You will perform side-by-side... ...snippets, assessing naturalness and authenticity to help models understand Norwegian language nuances. Pay is up to $20/hr with many tasks taking about...Remote jobFlexible hours- Mercor is hiring experienced Musicians to evaluate generative musical AI models in partnership with a leading AI lab. You will assess model outputs across genres in your bilingual language, focusing on lyrics, vocal quality, and musicality. Ideal candidates have 3+ years...BilingualPart time10 hours per weekFlexible hours
- ...bodies including Justice Center, OMH, EBA etc. Continually evaluate organizational needs including periodic review of Program... ...Care preferably with Adolescents. EXPERIENCE REQUIRED/LANGUAGE PREFERENCE: Bilingual, Spanish, French, Creole preferred. COMPUTER SKILLS REQUIREMENTS...BilingualWork at officeFlexible hours
$100 - $130 per unit
...provider actively building its per diem MDE evaluator network across Nassau County. We are... ...- $130 per completed, approved report | Bilingual: $140 - $165 per completed, approved report... ...workers are especially encouraged. Languages especially needed: Spanish, Haitian Creole...BilingualDaily paidFlexible hours- ...Workflow Annotator—Product Management & Marketing to remotely review evaluation prompts and responses against the company's quality rubric.... ..., label edge cases, and provide structured feedback the modeling team can use to retrain. As a contractor, you will assess frontier...Remote jobFor contractors
$135 - $190 per hour
...Speech Language Pathologist Evaluator-NYC, NY Job-Achieve Beyond Description: Apply to Achieve Beyond as a Speech Language Pathologist Evaluator in NYC, NY and help children with special needs reach their goals. Explore our New York career opportunities and make...Local areaFlexible hoursAchieve Beyond Pediatric Therapy & Autism Services
New York, NY2 days ago- AI Trainer Jobs is seeking an Illustration Quality Evaluator for a remote contractor role. Review illustration quality evaluation prompts and responses against a published rubric, compare paired outputs, and provide structured feedback to support retraining efforts. You...Remote jobPart timeFor contractors
- AI Trainer Jobs is seeking a remote contractor to evaluate people ops / recruiting prompts and responses against a evolving quality rubric. You will compare model outputs, label edge cases, and provide structured feedback for retraining. Responsibilities include evaluating...Remote jobFor contractors
$50 per hour
Combinatorics Model Evaluator is a remote review track for evaluating AI outputs across combinatorics model research review reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the correct method...Hourly payFor contractorsRemote work- Surgical Planning Safety Evaluator is a remote evaluation track for reviewing surgical planning safety evaluation prompts and responses... ...edge cases, and write the kind of structured feedback the modeling team can use to retrain. AI data reviewers help turn surgical...Remote job
- Receipt and Invoice Understanding Model Evaluator is a remote evaluation track for reviewing receipt and invoice understanding model evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind...Hourly payFor contractorsRemote work10 hours per week
- ...Job Description Job Description All About Kids is seeking a passionate and dedicated in-person Speech-Language Pathologist (SLP) Evaluator to join our team. As an SLP Evaluator, you will conduct comprehensive speech and language evaluations for children with speech...Full timeFlexible hours
All About Kids, Inc.
New York, NY6 days ago $100 per unit
Speech-Language Pathologist (Temporary / Substitute / Per Diem / Interim) Join to apply for... ...entering new clinical environments Bilingual in Spanish and English is a plus Compensation... ...disabilities. Our McCarton Integrated Model combines speech and language therapy,...BilingualDaily paidTemporary workFor contractorsInterim roleLocal areaFlexible hours$135 - $190 per hour
Speech Language Pathologist Evaluator, NYC, NY - Achieve Beyond Achieve Beyond is looking for a Speech Language Pathologist Evaluator in NYC, NY to help children with special needs reach their goals. Explore our New York career opportunities and make a positive impact...Local areaFlexible hours- AI Trainer Jobs is seeking a Mongolian Bilingual Expert for a remote evaluation track. You will review prompts and model outputs against a versioned rubric, label edge cases, and provide structured feedback for retraining. The role emphasizes accuracy, clear reasoning,...BilingualRemote jobFor contractors
- AI Trainer Jobs is seeking a Hebrew Bilingual Expert for remote evaluation work. You will review Hebrew prompts and responses, compare outputs, and provide structured feedback to support AuraOne’s quality rubric. Responsibilities include labeling edge cases, tracking severity...BilingualRemote jobHourly payContract workFor contractorsFlexible hours
$80 per hour
.../hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated implementations involving ETL pipelines , data warehouses , analytics platforms , and distributed...Contract workSummer workRemote workMercor
New York, NY8 days ago- AI Trainer Jobs is seeking a Bilingual AI Response Evaluator for a remote, contract-based role. You will review bilingual prompts and responses, compare outputs, and provide structured feedback that guides model retraining. This position emphasizes careful judgment, clear...BilingualRemote jobHourly payContract work10 hours per week
$80 - $120 per hour
Public health communications Evaluator is a remote clinical-review track for evaluating AI... ...the corrected clinical reasoning so the modeling team can close the gap. Category: Medical... ...decision support and its failure modes. Bilingual clinical experience for non-English...BilingualFor contractorsRemote work10 hours per week- ...providing services to children birth-3 years old. We are a growing agency seeking strong monolingual and bilingual Physical Therapists to provide early intervention evaluations in New York City (Manhattan, Brooklyn, Queens, Bronx, and/or Staten Island). We take great pride in...BilingualFull timeFlexible hours
Thrive By 5
New York, NY3 days ago $400 per month
...City, NJ 08401 Date Posted: 10/01/2026 Category: School Education: Masters Degree Our client is seeking Bilingual Spanish Speech Language Evaluators in Atlantic County, NJ SUMMARY OF CLIENT'S DESCRIPTION OF THIS OPPORTUNITY * Must have your own testing...BilingualContract workFor contractorsSelf employmentWork at office- Prolific Academic Ltd is seeking Licensed Pharmacists to assist in AI model training and evaluation from a home office. Successful candidates join as Domain Experts and will be paid to train and evaluate models, with tests to assess suitability. Experts review AI-driven...Remote workHome officeFlexible hours
$131 - $153 per hour
...area. We specialize in conducting initial evaluations and providing ongoing services in home,... ...Conduct evaluations in the primary language of the child Maintain confidentiality... ...Permanent Special Education certificate Bilingual Spanish a must; Bilingual extension...BilingualPermanent employmentLocal areaWork from home
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Bilingual Language Model Evaluator. Be the first to apply!
Related searches
- ai evaluator New York, NY
- education evaluator New York, NY
- work from home web search evaluator New York, NY
- evaluator New York, NY
- program evaluator New York, NY
- work from home social media evaluator New York, NY
- clinical evaluator New York, NY
- social media evaluator New York, NY
- quality evaluator New York, NY
- bilingual accountant New York, NY