Remote AI Model Evaluation & Rubric Specialist
AuraOne
New York, NY
- Remote job
AuraOne is seeking an Evaluation Harness Model Evaluation Specialist to work remotely as an independent contractor. You will review evaluation harness model outputs, label issues, and provide structured feedback to retrain the model using AuraOne's quality rubric. Strong attention to detail and clear reasoning are essential for consistent, scalable evaluation across long batches. Responsibilities include comparing paired responses, tagging edge cases, and documenting recurring failure modes to #J-18808-Ljbffr AuraOne
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Remote AI Model Evaluation & Rubric Specialist in New York, NY vacancy
- Dorado is seeking a Physics Specialist to contribute deep scientific expertise to AI model evaluation. You will craft and assess challenging physics problems, probe model reasoning at the frontier, and help identify where models fail under rigorous scientific scrutiny....Remote job
- A leading AI company is seeking a Biology Specialist to help fine-tune large language models. Ideal candidates will be pursuing or hold a Ph.D. in Biology or a related field... ...and collaborating on AI projects in a fully remote setting. Applicants with excellent communication...Remote job
$20 - $60 per hour
...project that trains next-generation AI systems. You will design realistic Fortune... ...with an advanced language model to create, edit, and evaluate Office Open XML files, with a focus on... ...Role type: Contractor. Location: Remote, participating on a customer project...Remote workHourly payContract workFor contractorsWork at officeSaidGig
United States3 days ago$75 per hour
...Join a project focused on evaluating AI models in the architecture domain, specifically in visual... ...clear ground-truth output and an objective rubric. Key Responsibilities Develop and... ...evaluation is a plus. Work Terms Remote work opportunity. Approximately 15,...Remote workHourly paySaidGig
Canada10 days ago$70 - $90 per hour
...Role Overview Help evaluate Neuron Kernel Interface development tasks... ...and evaluation of advanced AI models. You will assess kernel... ...environments. Provide clear, rubric-based written feedback. Qualifications... ...instances. Work Terms Remote role, open to candidates...Remote workHourly paySaidGig
Remote2 days ago$70 - $90 per hour
...tasks that support the training and evaluation of advanced AI models. This role focuses on assessing task... ...written feedback using defined evaluation rubrics. Key Responsibilities Evaluate... ...JAX/XLA custom calls. Work Terms Remote role open to candidates located in...Remote workHourly paySaidGig
Remote2 days ago$46 per hour
...partner is looking for a Legal Domain Expert (SME) – AI Model Evaluation based in the United States. This is a remote, flexible opportunity for an experienced legal... ...evaluation criteria, scoring frameworks, and rubrics for assessing AI-generated legal work. Provide...Remote workFull timeContract workFlexible hoursjobgether
United States1 day ago$100 - $150 per hour
...Role Overview Evaluate how well AI systems perform real-world technical sales work by defining excellence... ...you will create task-specific grading rubrics and assign scores to AI-generated and... ...strong plus Work Terms Location: Remote Engagement type: hourly...Remote workHourly paySaidGig
United Statesa month ago$100 per hour
...domain expertise to train and evaluate next-generation AI systems by producing,... ...role focuses on improving model outputs through careful content... ...review, prompt refinement, rubric-based evaluation, and clear... ...contractor, part-time. Location: Remote. Work is project-based...Remote workHourly payPart timeFor contractorsSaidGig
Indiana23 days ago$100 per hour
...business expertise to improve AI-generated content for a customer-facing evaluation and optimization project... ...refine large language model prompts using structured... ...quality assurance, and rubric-based assessments of AI... ...apply. Work Terms Remote, part-time independent...Remote workHourly payPart timeFor contractorsSaidGig
United States6 days ago$100 per hour
...is seeking finance experts to enhance AI models. Responsibilities include evaluating performance in capital markets and creating assessment rubrics. Candidates should have 2+ years in finance... ...strong financial knowledge. This remote role offers competitive hourly pay around...Remote jobHourly pay10 hours per week$100 per hour
...performance of large language models on finance tasks. You will work with AI researchers to identify... ...is required. This is a remote, US-based contract... ...Key Responsibilities Evaluate LLM performance in finance... ...Design clear evaluation rubrics and assessment criteria...Remote workHourly payContract workFor contractorsFreelance10 hours per weekFlexible hoursSaidGig
United Statesmore than 2 months ago- ...Model Grading Model Evaluation Specialist is a remote evaluation track for reviewing model grading model evaluation evaluation... ...against AuraOne's quality rubric. Reviewers compare paired outputs,... ...retrain. Why this role matters AI data reviewers help turn model grading...Remote jobHourly payFor contractors10 hours per week
$60 per hour
...and Life Science Professionals to join their Expert Network to evaluate AI-generated science. This role allows you to work from home with... ...offers a competitive pay rate of up to $60 per hour for reviewing model responses, validating technical claims, and critiquing...Remote workHourly payWork from homeFlexible hoursProlific
Dallas, TX1 day ago$60 per hour
...Prolific is seeking Biology Experts and Life Science Professionals to evaluate AI-generated science and ensure compliance with scientific standards. Responsibilities include reviewing biological inquiries, validating technical claims from public databases, and critiquing...Remote workHourly payWork from homeFlexible hoursProlific
San Jose, CA1 day ago$60 - $90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Machine Learning Engineer — Model Evaluation & Experimentation Type: Contract... ...Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week...Remote workFull timeContract workSummer workMercor
Remote1 day ago- ...Role Overview Evaluate and annotate AI-generated music in Arabic and English,... ...improve generative musical models. This role focuses on listening... ...samples using defined rubrics. Rate pairs of AI-generated... ...Work Terms Location: Remote. Engagement: Hourly, part...Remote workHourly payPart timeImmediate start10 hours per week
SaidGig
United Statesa month ago $80 - $110 per hour
...improve next generation AI systems through accurate, practical evaluation and feedback. This part... ...content against provided rubrics, then deliver concise written... ...feedback that improves model performance. Verify... ...engagement. Remote position available only...Remote workHourly payContract workPart timeFor contractorsSaidGig
Remote14 days ago- ...Prolific is seeking Biology Experts and Life Science Professionals to join an expert network that evaluates and trains AI models. This role involves reviewing AI-generated scientific content for accuracy and validation, requiring candidates with a BS, MS, or PhD in relevant...Remote workFlexible hours
Prolific
Charlotte, NC1 day ago - YO AI Labs is seeking experienced Producers across film, television, digital, or live events to support an AI evaluation project in a fully remote capacity. You will craft realistic industry scenarios,... ...table, delivering clear evaluation rubrics #J-18808-Ljbffr YO AI LabsRemote work
$40 per hour
...Massachusetts is seeking an R&D Biologist to join their team to train AI models by evaluating chatbot outputs against complex biology questions. Ideal... ...biochemistry. This position allows full-time or part-time remote work, with projects that pay hourly starting at $40+....Remote workHourly payFull timePart timeDataAnnotation
United States3 days ago- AuraOne is seeking a remote News Monitoring Task Evaluator to assess evaluation prompts and responses against our quality rubric. You will compare paired outputs, write rationales, and label edge cases to aid model retraining and quality control. Join as a contractor working...Remote jobFor contractors
- ...experienced Investment Banking SME to support the development, evaluation, and improvement of advanced AI models in finance. You will assess AI-generated analyses for... ...and prompts for high-quality outputs. The role is remote, project-based, and demands strong domain expertise...Remote job
$42 - $78 per hour
...Evaluate generative music and lyrics in Norwegian and English, helping assess output across... ...standards. Key Responsibilities Compare AI-generated lyrics with published songs to... ...-genres, and artists. Work Terms Remote, hourly independent contractor engagement...Remote workHourly payFor contractorsImmediate startFlexible hoursSaidGig
United States15 days ago$13 - $54 per hour
...Evaluate AI-generated music and lyrics across a wide range of genres, using your knowledge of the Spanish (Mexico) music scene to assess quality against detailed standards. This remote role involves working in both Spanish (Mexico) and English. Key Responsibilities...Remote workHourly payImmediate startFlexible hoursSaidGig
United States4 days ago- ...Evaluate generative music AI across a broad range of genres, applying your knowledge of Indonesian music and lyrics to detailed quality standards... ...Indonesian genres, subgenres, and artists. Work Terms Remote, hourly engagement with an immediate start. Flexible...Remote workHourly payImmediate startFlexible hours
SaidGig
United States4 days ago - ...French Audio Evaluations Specialist - Freelance AI Trainer Project World Wide - Remote Project Overview We are sourcing independent... ...advanced agentic audio models. As AI models increasingly handle... ...prompts, complex evaluation rubrics, and technical guidelines....Remote workHourly payFor contractorsFreelance
Meridial
United States4 days ago $50 - $70 per hour
...Role Overview Help improve frontier AI models by evaluating the quality of real-world professional materials and AI-generated work. You will... ...Guidelines and context will be provided. Work Terms Remote, hourly engagement. Applicants must be based in the United...Remote workHourly paySaidGig
Canada12 days ago$15 per hour
...Role Overview Apply your Punjabi music expertise to evaluate AI-generated music and lyrics across a broad range of genres. You will assess... ...Punjabi genres, subgenres, and artists. Work Terms Remote, hourly engagement with an immediate start. Flexible schedule...Remote workHourly payImmediate startFlexible hoursSaidGig
United States15 days ago$20 - $60 per hour
...Help train next-generation AI systems by creating rigorous, real-world evaluations that test how well advanced models learn, reason, and perform. This remote contract opportunity is open to recent graduates, advanced-degree holders, and professionals from any background...Remote workHourly payContract workFor contractorsSaidGig
Indiana4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote AI Model Evaluation & Rubric Specialist. Be the first to apply!
Related searches
- ai data scientist New York, NY
- ai scientist New York, NY
- software engineering manager remote New York, NY
- system engineer remote New York, NY
- remote technical project manager New York, NY
- remote recruitment consultant New York, NY
- remote property manager New York, NY
- remote reviewer New York, NY
- remote insurance agent New York, NY
- remote no experience New York, NY