Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Evaluator - Software Expert

$80 - $120 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated artifacts against domain-specific quality rubrics.
  • Identify factual, aesthetic, and presentation errors in outputs.
  • Provide clear, structured written feedback to improve quality.
  • Work independently and asynchronously to meet deadlines.
  • Utilize expertise in Microsoft Office and Google Workspace , especially Slides .

Qualifications

Must-Have

  • 5+ years of relevant professional experience in Software / AI / IT / data .
  • Native or professional fluency in English .
  • Highly proficient in Microsoft Office and Google Workspace , especially Slides .

Preferred

  • Advanced degree ( Master's or higher) from a reputable institution.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 28 days ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in New York, NY vacancy
  • $80 - $120 per hour

     ...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour... 
    Software
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    7 days ago
  • $40 per hour

    Review and improve AI answers written in Maltese so they read naturally and get the facts right. You rate responses, fix weak ones and write examples that show the model what good looks like. What you will do Rate Maltese AI responses for accuracy, fluency, tone and instruction... 
    Suggested
    Remote job
    Hourly pay
    Weekly pay
    Contract work
    Part time
    For contractors
    Flexible hours

    Vettedlance

    New York, NY
    3 days ago
  • Vettedlance is seeking a meticulous Polish language reviewer to enhance AI answers. You will rate Polish responses for accuracy, fluency, and tone, rewrite weak answers, and ensure factual correctness. You will also fact-check claims, flag unsafe content, and craft prompts... 
    Suggested
    Remote job
    Hourly pay
    Part time
    Flexible hours

    Vettedlance

    New York, NY
    2 days ago
  • Synthires is seeking a Film & Video Craft Expert to remotely evaluate AI-generated videos across lighting, framing, color, and performance. This part-time role emphasizes concise feedback and strong written communication. Applicants should be fluent in English and have... 
    Suggested
    Remote job
    Part time

    Synthires

    New York, NY
    5 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    9 days ago
  • Job Description/ Responsibilities: AI-Driven Test Automation Transformation: • Lead...  ...adoption of GenAI-powered automation across the Software Testing Lifecycle (STLC), driving...  ...Innovation & Experimentation: • Continuously evaluate emerging AI technologies and identify... 
    Software

    TechDigital Group

    New York, NY
    4 days ago
  • Thomson Reuters is seeking a Sr. Legal AI Prompter/Evaluator in New York to lead hands‑on development of LLM‑driven legal tools. You will design...  ..., run experiments, and build evaluation schemes with SMEs, software engineers, and data scientists to ensure accurate and... 
    Software

    Thomson Reuters

    New York, NY
    3 days ago
  •  ...years of experience specializing in Microsoft, Salesforce, BI, AI, and Cloud Computing. We have established longstanding client relationships...  ...and query performance tuning. Solid understanding of software development life cycle (SDLC) principles. Excellent problem-solving... 
    Software
    Contract work

    Vertex Computer Systems

    New York, NY
    3 days ago
  • Role overview Help design and implement systems that coordinate AI services and software components to support useful business workflows. The role sits at the intersection of product and engineering, with an emphasis on turning practical needs into working, connected solutions... 
    Software
    Full time
    Remote work

    Kollabit

    New York, NY
    2 days ago
  • 3+ years of experience implementing AI or legal-technology solutions in a law firm, corporate...  .../common-interest considerations, • Evaluate legal-AI vendors and platforms (e.g.,...  ...deployment, • Apply security patches and software updates across endpoints and core servers... 
    Software
    Full time
    Contract work
    Work at office
    Remote work

    Winston Staffing

    New York, NY
    7 hours ago
  • $20 - $45 per hour

    Coding & Software Remote Hourly · $20-$45/hr $20-$45/hr Compensation Contract, part-time Engagement Remote Location Posted Open...  ...United States Uzbekistan Vietnam The work You will help evaluate AI systems by reviewing software code, technical documentation, and... 
    Software
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • Описание: Ciklum is looking for an Expert Back End Engineer to join our team full-time in...  ...experiences. Задачи: Take ownership of the entire software development lifecycle-from ideation and...  ...and actively experimenting with how AI changes Software Development Lifecycle 7+... 
    Software
    Full time
    Shift work

    Ciklum

    New York, NY
    4 days ago
  •  ...Sciences Researchers with strong Python programming skills to contribute to projects evaluating and improving advanced AI systems. You will combine scientific expertise with practical software abilities to create, solve, and review challenging problems across biology and... 
    Software
    Flexible hours

    Gramian Consulting Group

    New York, NY
    2 days ago
  • $84k - $115k

    Secure Every Identity, from AI to Human Identity is the key to unlocking the potential...  ...2D/3D visualizations). Tool Mastery: Expert-level fluency with Figma and Adobe Creative...  ...you have Working knowledge of 3D spatial software or spatial visualization tools (SketchUp,... 
    Software
    Local area
    Worldwide
    Flexible hours

    Okta

    New York, NY
    3 days ago
  •  ...Software Architect / Solutions ArchitectAt Air Apps, we believe in thinking bigger—and moving...  ...on a mission to create the world's first AI-powered Personal & Entrepreneurial...  ...role in shaping our technology strategy, evaluating new tools and frameworks, and ensuring best... 
    Software
    Work at office
    Worldwide
    Relocation

    Air Apps

    New York, NY
    5 days ago
  • $170k - $200k

     ...We are seeking an experienced Senior AI Engineer to serve as a hands‑on technical...  ...and context engineering, model selection, evaluation, and token-usage optimization to improve...  ...preferred. ~7+ years of experience in software, data, or machine learning engineering, including... 
    Software
    Work experience placement
    Work at office
    Remote work

    Amalgamated Bank of NY

    New York, NY
    3 days ago
  •  ...Get AI-powered advice on this job and more exclusive features. Direct message the job poster from Synergy Interactive Responsibilities...  ..., and Writing/Editing Industries IT Services and IT Consulting, Advertising Services, and Software Development #J-18808-Ljbffr... 
    Software
    Contract work

    Synergy Interactive

    New York, NY
    1 day ago
  •  ...edge Artificial Intelligence (AI) based product designed to...  ...experienced, Subject Matter Experts (SMEs) to partner with our Product...  ...Medical Bills, etc.) Product Evaluation: Actively use and evaluate...  ...with computers and learning new software applications. Familiarity... 
    Software

    CCC Intelligent Solutions

    New York, NY
    2 days ago
  • $125k - $225k

     ...Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing... 
    Software
    Permanent employment
    Work experience placement
    Work at office
    Local area
    3 days per week
    New York, NY
    a month ago
  •  ...for a Technical Solution Architect with AI, Azure and .NET expertise. This is a 12+month...  ...Solutions Architect with 12+ years of software development experience , including 5+...  ...model integration, AI observability, model evaluation, and Human-in-the-Loop controls . ·... 
    Software
    Local area
    Remote work
    3 days per week

    Donnelly & Moore Corporation

    New York, NY
    20 days ago
  • Code4nord is seeking a skilled AI Agent Developer for an exciting private-sector project. This fully remote role involves designing...  ...shaping project direction. The ideal candidate will have a strong software development background and hands-on experience in AI. Team... 
    Software
    Remote job
    Full time

    Code4nord

    New York, NY
    2 days ago
  • Role Title: AI Product Evaluator Role Type: Contractor Location: India, Japan or United Kingdom, Remote Our team is engaging expert AI Product Evaluators to contribute directly to a customer...  ...participation in digital product, software QA, or AI research projects is a plus... 
    Software
    For contractors
    Remote work

    Remotebridge

    New York, NY
    4 days ago
  •  ...the architecture, design, and technical governance of a modern, AI-enabled enterprise application platform built using C#, .NET 8+,...  ...This senior technical leadership role requires 12+ years of software development experience, including at least 5 years serving as a... 
    Software
    Full time
    Contract work

    Seneca Resources

    New York, NY
    2 days ago
  •  ...services organization is seeking an accomplished Director-level Software Engineer to design and develop cloud-native applications supporting...  ...intelligence applications. Modern data engineering or analytics tools. Exposure to enterprise AI-enabled applications.... 
    Software

    Madison-Davis, LLC

    New York, NY
    1 day ago
  • Thomson Reuters seeks a Sr. Legal AI Prompter / Evaluator to advance CoCounsel Legal through prompts, skills, and agent configurations. You will...  ...accurate and legally sound. You will partner with SMEs, software engineers, and data scientists to improve product quality.... 
    Software

    Thomson Reuters

    New York, NY
    5 days ago
  • $65 - $75 per hour

    job summary: Senior AI Developer We're looking for a Senior AI Agent Developer...  ...and intent classification standards and evaluate changes against measurable quality criteria...  ...practical experience 8+ years of software engineering experience, including at least... 
    Software
    Hourly pay
    Contract work

    Randstad Digital

    New York, NY
    3 days ago
  • $160k - $180k

     ...Artemis Artemis is building the future of AI-driven cybersecurity, helping companies...  ...learning from Oxford. Our team of software engineers, AI researchers, security engineers...  ...administration and lifecycle management: evaluating tools, managing licenses, and cutting... 
    Software
    Full time

    artemis

    New York, NY
    3 days ago
  •  ...of business-critical calculation logic. Establish responsible AI-assisted engineering practices, enabling the team to use...  ...practices, observability and operational resilience. AI-assisted software engineering: Experience using AI-powered development tools such... 
    Software
    Full time
    Local area

    Shaw Daniels Solutions

    New York, NY
    1 day ago
  • $40 per hour

     ...STEM training data for frontier AI models. Our data is used directly in training and evaluation pipelines at leading AI labs to...  ...domains. We are looking for experts in Physics to design rigorous,...  ...tools or domain-specific physics software (e.g., finite element tools, circuit... 
    Software
    Contract work
    Part time
    Remote work

    Anyone AI Inc.

    New York, NY
    4 days ago
  • $135k

     ...application development, AWS cloud-native technologies, and modern AI/GenAI solutions. The ideal candidate will provide technical...  ...Financial Services domain. Qualifications • 8+ years of overall software development experience, with strong experience in .NET... 
    Software
    Full time

    VGAI Solutions

    New York, NY
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!