AI Code Evaluator - Fully Remote | Upto $90/hr
$90 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Developer Trace Task Auditor
Type: Contract
Compensation: $70–$90/hour
Location: Remote
Role Responsibilities
- Evaluate the quality and correctness of AI-assisted software-development traces to enhance model training and evaluation.
- Assess end-to-end coding sessions produced with AI-assisted developer tools for correctness, workflow soundness, and reasoning.
- Provide clear, rubric-based written feedback to improve AI model outputs .
- Review complex coding trajectories for alignment with best practices and correctness.
- Collaborate with AI research teams to ensure consistency and relevance in training data.
- Work independently and asynchronously to meet deadlines while enhancing AI model performance .
Qualifications
Must-Have
- 3+ years professional software development.
- Hands-on experience with AI-assisted coding tools and agentic/spec-driven workflows ( Cursor , GitHub Copilot , Claude Code , or similar).
- Strong code-reading and debugging skills across full-stack or backend systems.
- Ability to evaluate multi-step coding trajectories for correctness and best practice.
Preferred
- Experience with Kiro or Amazon CodeCatalyst .
- Prior work evaluating or grading AI-generated code .
- Contributions to developer tooling .
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on publisher.careerdays.io
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 23 days ago
Similar jobs that could be interesting for youBased on the AI Code Evaluator - Fully Remote | Upto $90/hr in Abbeville, GA vacancy
$90 per hour
...and technical talent with leading AI research labs. Headquartered in... ...Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality and correctness of... ...evaluation. Assess end-to-end coding sessions produced with AI-assisted...Remote jobContract workSummer work$90 per hour
...technical talent with leading AI research labs.... ...Contract Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week... ...limits of today's best AI coding agents. Build reference... ...experience in AI training, model evaluation, or benchmark/task...Remote jobContract workSummer work$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Machine Learning Engineer — Model Evaluation & Experimentation Type: Contract Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week Role...Remote workContract workSummer workMercor
New York, NYa month ago$90 per hour
...and technical talent with leading AI research labs. Headquartered in... ...Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality, fidelity, and completeness... ...Demonstrated expertise in secure coding and remediation across common vulnerability...Remote jobContract workSummer work$90 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...Contract Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week Role Responsibilities Evaluate frontier AI models on coding, ML , and analysis tasks to identify...Remote jobContract workSummer work$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Machine Learning Engineer — Model Evaluation & Experimentation Type: Contract Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week Role...Remote workHourly payWeekly payContract workFor contractorsSummer workMercor
New York, NYa month ago$30 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Contract Compensation: $30/hour Location: Remote Commitment: 10 hours/week Role Responsibilities... ...questions, and correcting errors. Evaluate the experience and deliverables, providing...Remote jobHourly payWeekly payContract workFor contractorsSummer work$20 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Contract Compensation $15-$20/hour Location Remote Role Responsibilities Conduct fact-checking... ...tools. Generate high-quality human evaluation data by identifying response strengths, areas...Remote jobContract workPart timeSummer work$90 per hour
...technical talent with leading AI research labs.... ...Contract Compensation: $65–$90/hour Location: Remote Commitment: 10 hours/week... ...Responsibilities Review and evaluate written content and AI-... ...experience. ~ Familiarity with codes and standards like ASCE ,...Remote jobContract workPart timeSummer workMercor
New York, NY6 days ago$90 per hour
...technical talent with leading AI research labs.... ...Contract Compensation: $65–$90/hour Location: Remote Commitment: 10 hours/week... ...Responsibilities Review and evaluate written content and AI-... ...experience. ~ Familiarity with codes and standards (e.g., ASCE...Remote jobContract workPart timeSummer work10 hours per weekMercor
San Francisco, CA27 days ago$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Contract Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week Role... ...Experience in AI training , model evaluation, or quality review of AI-generated work...Remote jobContract workSummer work$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Contract Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week Role... ...provide clear recommendations. Evaluate model performance on tasks, ensuring...Remote jobContract workSummer work$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Contract Compensation: $60–$90/hour Location: Remote Commitment: 35 hours/week Role... ...spot checks and recommendations. Evaluate how models handle tasks and verify the...Remote jobHourly payWeekly payContract workFor contractorsSummer work$90 per hour
...technical talent with leading AI research labs.... ...Contract Compensation: $90/hour Location: Remote Role Responsibilities... ...that merely look correct. Evaluate responsive behavior and semantic... ...model evaluation work. Code review or technical assessment...Remote workContract workSummer workLocal area$90 per hour
...technical talent with leading AI research labs.... ...Contract Compensation: $65–$90/hour Location: Remote Commitment: 10 hours/week... ...Responsibilities Review and evaluate written content and AI-... ...Familiarity with building codes and standards (e.g., IBC )...Remote jobContract workPart timeSummer work10 hours per weekMercor
New York, NY20 days ago$50 - $60 per hour
...seeks detail-oriented researchers to contribute to an advanced AI research and evaluation project. You will create research questions, develop... ...series chip is mandatory, with macOS 15+ as applicable. This remote, part-time role offers $50-$60/hour and ~10-20 hours per week...Remote jobPart timeImmediate start10 hours per week$25 per hour
Vettedlance is seeking contributors to review and improve AI-derived educational answers. This remote, part-time role pays $25 per hour and includes a short paid assessment. You will rate responses for accuracy, verify sources, and add clear annotations, working at least...Remote jobHourly payPart time- Mercor is seeking a board-certified obstetrician-gynecologist to evaluate health AI models. You will read de-identified patient charts and judge... ...gynecology questions for clinical accuracy and safety. This remote, hourly contractor role offers flexible part-time hours....Remote jobHourly payContract workPart timeFor contractorsImmediate startFlexible hours
$30 per hour
...Hourly contract Compensation: $20-$30/hour Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous... ...Requirements Strong experience in LLM evaluation, AI output analysis, QA/testing, UX research, or...Remote jobHourly payContract work$20 per hour
Productive Playhouse Inc. seeks a remote AI evaluation tester to assess chatbots and language models through structured conversations, with occasional voice-based tasks. You will submit deliverables such as write-ups, ratings, screenshots, or recordings as required. Requirements...Remote jobFlexible hours- Meridial is seeking independent contractor experts to evaluate AI-generated outputs focusing on JSON structure, formatting accuracy, and... ...responses in English and the chosen language of the cohort. This is a remote contractor role in the United States. #J-18808-Ljbffr...Remote workContract workFor contractors
$19 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Compensation: $11–$19/hour Location: Remote Duration: Up to 6 months Commitment... ...0 hours/week Role Responsibilities Evaluate AI-generated music across diverse genres...Remote jobContract workSummer workImmediate start$120 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...: Real estate / hospitality / events Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated...Remote jobContract workSummer workWork at office- ...Secure Code Review AI Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team...Remote jobHourly payFor contractors10 hours per week
$120 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...: Real estate / hospitality / events Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated...Remote jobContract workSummer workWork at office$30 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Compensation: $30/hour Location: Remote Commitment: 10 hours/week Role Responsibilities... ...questions, and correcting errors. Evaluate the experience and deliverables,...Remote jobHourly payWeekly payContract workFor contractorsSummer work$17 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Compensation: $17/hour Location: Remote Duration: Up to 6 months Commitment... ...+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation...Remote workContract workSummer workImmediate start- ...Dialect Code-Switching AI Evaluator is a remote evaluation track for reviewing dialect code switching ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the...Remote jobHourly payFor contractors10 hours per week
$70 - $90 per hour
...Network Engineering (Remote | $70–$90/hr) Position: Network Engineering... ...Opportunity Contribute to an AI research project focused on... ...verification checks, and evaluating AI-generated network configurations... ...network behaviour. Use coding agents to support automation...Remote workHourly payContract workSynthires
United States1 day ago$90 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality and correctness of Kubernetes tasks...Remote jobContract workSummer workMercor
San Francisco, CA7 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Code Evaluator - Fully Remote | Upto $90/hr. Be the first to apply!
Related searches
- evaluator Abbeville, GA
- ai evaluator Abbeville, GA
- education evaluator Abbeville, GA
- quality evaluator Abbeville, GA
- social media evaluator Abbeville, GA
- software engineer internship remote Abbeville, GA
- remote banking Abbeville, GA
- remote tasks Abbeville, GA
- implementation project manager remote Abbeville, GA
- devops aws developer (remote) Abbeville, GA