App
Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Evaluator - Software Expert

$80 - $120 per hour

Mercor

New York, NY

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated artifacts against domain-specific quality rubrics.
  • Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
  • Provide clear, structured written feedback to improve AI model outputs .
  • Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
  • Work independently and asynchronously to meet deadlines.

Qualifications

Must-Have

  • 5+ years of relevant professional experience in Software / AI / IT / data .
  • Native or professional fluency in English .
  • Highly proficient in Microsoft Office and Google Workspace , especially Slides .

Preferred

  • Advanced degree ( Master's or higher ) from a reputable institution.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Apply
Vacancy posted 12 days ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in New York, NY vacancy
  • AI Evaluator - Software Expert

    $80 - $120 per hour

     ...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour... 
    Software
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    3 days ago
  • Baseball Expert AI Evaluator

    $100 - $120 per unit

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Baseball Fan Live MLB Game AI Evaluator (US) Type: Contract Compensation...  ...schedules broadcasts highlights and expert-style analysis. Run the same question... 
    Suggested
    Full time
    Contract work
    Work experience placement
    Summer work
    Remote work

    Mercor

    New York, NY
    14 days ago
  • Remote: AI Design Evaluator & Visual Graphics Expert

    $30 per hour

     ...About Prolific Prolific is not just another player in the AI space – we are building the biggest pool of quality human...  ...Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image generation... 
    Suggested
    Remote work
    Work from home
    Flexible hours

    Prolific

    New York, NY
    2 hours ago
  • OpenFOAM Expert— AI Training (Remote, Freelance)
     ...and OpenFOAM, ideally with experience applying the software across different industries and engineering...  ...Location: US, Canada, UK, or Europe Engagement Remote, project-based expert role Supporting AI model training and evaluation for a leading AI lab .... 
    Software
    Hourly pay
    Freelance
    Remote work

    Braintrust

    New York, NY
    2 days ago
  • AI Safety Practitioner - Expert Evaluator

    $60 - $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    14 days ago
  • Developer & Infrastructure Expert - Remote
     ...Description Job Description Developer & Infrastructure Expert Role Type: Contractor Location: Remote Job...  ...experienced Developer & Infrastructure Experts to evaluate AI-powered workflows across software development, cloud infrastructure, DevOps, SRE, and platform... 
    Software
    Remote job
    For contractors

    YO AI Labs

    New York, NY
    8 days ago
  • Biology Evaluator - Domain Expert - AI Trainer

    $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...and Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    3 days ago
  • Lead Software Engineer, Cloud ERP (Finance)

    $172.5k - $260.1k

     ...DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive...  ...visionary technical leader to guide and support software engineers working on Workday Finance,...  ...tools to help our recruiters assess and evaluate candidates’ resumes and qualifications throughout... 
    Software
    Full time

    Salesforce

    New York, NY
    1 day ago
  • AI Code Evaluator - Fully Remote | Upto $90/hr

    $90 per hour

     ...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Location: Remote Role Responsibilities Evaluate the quality and correctness of AI-assisted software-development traces to enhance model training and... 
    Software
    Remote job
    Contract work
    Summer work

    Mercor

    New York, NY
    15 days ago
  • Materials Science PhD - AI Evaluator

    $70 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Material Science PhD Coding Experts Type: Contract Compensation: $...  ...peer-reviewed journals. Prior scientific software or research engineering experience. Start... 
    Software
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    New York, NY
    27 days ago
  • Expert Senior Manager, Data Sciences, Software and Data Services, Coro

    $164.5k - $219k

    General Information Job Title Expert Senior Manager, Data Sciences, Software and Data Services, Coro Job ID...  ...opportunities to work with major AI ecosystem partners through Bain's...  ...the team applies to prompt design, evaluation and output quality assuranceDevelop... 
    Software
    Permanent employment
    Full time
    Apprenticeship
    Local area
    Home office
    3 days per week

    Bain & Company

    New York, NY
    3 days ago
  • Financial Services - Data Center Infrastructure & Migration - Tech Consulting - Manager - NYC

    $166.9k - $278.1k

     ...stakeholders in a controlled change environment.Required software and technical toolingHands-on experience using...  ...network management and automation tooling sufficient to evaluate readiness and coordinate specialists.AI-enabled deliveryExperience applying Microsoft Copilot... 
    Software
    Summer holiday
    Immediate start
    Flexible hours

    EY (Ernst & Young)

    New York, NY
    1 day ago
  • MuleSoft Technical Architect- Enterprise

    $190k - $232k

     ...Our team is comprised of customer strategy experts, Salesforce-certified experts across all...  ...mission-critical functions, or activate AI-powered innovation, we connect strategy to...  ...following: Boomi, Informatica, TIBCO, Software AG, CastIron, Jitterbit, or similar products... 
    Software
    Temporary work
    Local area

    Slalom

    New York, NY
    4 days ago
  • Solutions Engineer
     ...Capital Markets Enterprise AI is moving from pilots to...  ...architecture and practical technical evaluations while helping customers shape...  ...the Broader Subject Matter Expert Stay ahead of trends in...  ...architecture, solution architecture, software engineering or a comparable... 
    Software
    Full time

    Recognition One

    New York, NY
    1 day ago
  • Salesforce Financial Services Cloud Senior Consultant- Enterprise
     ...mission-critical functions, or activate AI-powered innovation, we connect strategy to...  ...including configuration of the FSC Platform* Expert level understanding of various components...  ...enterprise systems as part of a complete software product lifecycle* Familiarity with agile... 
    Software
    Temporary work
    Local area

    Slalom

    New York, NY
    4 days ago
  • Technical Architect

    $224k - $308k

    Secure Every Identity, from AI to HumanIdentity is the key to unlocking the potential of AI. Okta secures AI by building the trusted...  ...Knowledge, Skills, and Abilities:8+ years of overall IT / software development experience, solution design and technical architecture... 
    Software
    Local area
    Worldwide
    Flexible hours

    Okta

    New York, NY
    1 day ago
  • Sr .Net Full Stack Engineer
     ...rebuild functionality using modern architecture, clean code, and AI-assisted development workflows. \n \n Job Requirement: \n...  ...tools . \n \n Required Skills: \n \n ~8+ years of software development and production support experience. \n ~ Strong experience... 
    Software
    Contract work

    Object Data Inc

    New York, NY
    7 days ago
  • Biology and Bioinformatics AI Evaluation Expert

    $45 - $90 per hour

     ...bioinformatics workflows used in AI benchmarking and model...  ...recommended practices. Design, evaluate, and critique biology, bioinformatics...  ...or open-source bioinformatics software. Open to candidates in:...  ...better artificial intelligence. Experts review examples, test model behavior... 
    Software
    Hourly pay
    Contract work
    Part time

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • Sr. SEO Specialist

    $75k - $90k

     ...campaigns for medium-to-large brands and websites Experience with AEO/AI SEO strategies, including featured snippet and AIO targeting,...  ...Surfer, Peec AI, Profound AI, etc. Familiarity with common CMS software, especially WordPress with WooCommerce, Shopify, Magento, and... 
    Software
    Full time
    Casual work
    Work at office
    Remote work
    Work from home
    Flexible hours

    Searchbloom

    New York, NY
    4 days ago
  • Salesforce Health Cloud Solution Architect
     ...mission-critical functions, or activate AI-powered innovation, we connect strategy to...  ...deserve better. As a Salesforce subject matter expert and healthcare technologist, you take...  ...(3-5+ years) experience with declarative software configuration (Epic/Cerner, Veeva, etc.)... 
    Software
    Local area
    Shift work

    Slalom

    New York, NY
    1 day ago
  • Senior C#/.NET Software Engineer - AI & Agentic Systems
     ...Role Overview We are seeking a Senior Software Engineer with a strong C#/.NET...  ...foundation who has expanded into modern AI development using Python, LLMs, and agentic...  ...and AI-assisted software development . Evaluate emerging AI development tools, frameworks... 
    Software
    Full time

    The Phoenix Group

    New York, NY
    3 days ago
  • FPGA Design Engineer - Vivado Expert

    $45 - $55 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Location: Remote Role Responsibilities Evaluate the functionality of Vivado on Windows...  ...of tasks without the provision of software licenses or hardware. Collaborate asynchronously... 
    Software
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    16 days ago
  • Event & Experiences Designer

    $84k - $115k

     ...Secure Every Identity, from AI to Human Identity is the key to unlocking the potential...  ...2D/3D visualizations). Tool Mastery: Expert-level fluency with Figma and Adobe...  ...have Working knowledge of 3D spatial software or spatial visualization tools (SketchUp,... 
    Software
    Local area
    Worldwide
    Flexible hours

    Okta

    New York, NY
    7 days ago
  • Engineering Lead
     ...of business-critical calculation logic. Establish responsible AI-assisted engineering practices, enabling the team to use...  ...practices, observability and operational resilience. AI-assisted software engineering: Experience using AI-powered development tools such... 
    Software
    Full time
    Local area

    Shaw Daniels Solutions

    New York, NY
    1 day ago
  • Remote AI Neuroscience Scientist: Benchmarking Expert Reasoning
    LatchBio seeks an AI Neuroscientist to serve as the scientific authority on ground truth for AI agents across biological workflows....  ...behavioral data, and curate multi-modal datasets. Collaboration with software engineers and neuroscientists will shape benchmarks for normal... 
    Software
    Remote job

    LatchBio

    New York, NY
    3 days ago
  • Data Cloud Success Architect (U.S. Citizen)

    $123.1k - $186.3k

     ...SalesforceSalesforce is the #1 AI CRM, where humans with agents...  ...Agentforce + Data360 subject-matter expert for the broader Success...  ...Signature and Strategic customers to evaluate and recommend optimization...  ...implementing enterprise software solutions in the Analytics/CDP... 
    Software
    Full time
    Work experience placement
    Work at office
    Remote work
    Flexible hours
    3 days per week

    Salesforce

    New York, NY
    4 days ago
  • Solutions Engineer
     ...Enterprise AI is moving from pilots to production, and the constraint...  ....• Own and streamline the evaluation process, guiding prospects...  ...Solace Agent Mesh Subject Matter Expert • Position Solace Agent Mesh...  ...role, selling complex enterprise software solutions.• Strong ability to... 
    Software
    Worldwide

    Solace

    New York, NY
    5 days ago
  • Member of Technical Staff
     ...About Beacon Software Beacon is acquiring and operating a portfolio of vertical SaaS companies. Most private equity firms scale by adding...  ...of the 8th wonder of the world - compounding growth. How We Use AI in Our Hiring Process: To ensure transparency, we want candidates... 
    Software

    Beacon Holdings

    New York, NY
    2 days ago
  • Senior Engineer, Data Management

    $140k - $150k

     ...is looking for a Senior Data Engineer to help shape our data and AI technology roadmap. You'll own the full lifecycle of complex...  ...data tooling.Solid grounding in data structures, algorithms, and software architecture, with experience documenting and testing complex systems... 
    Software
    Worldwide

    Hearst Corporation

    New York, NY
    2 days ago
  • Expert Senior Manager, AI Engineering

    $293.5k

    General Information Job Title Expert Senior Manager, AI Engineering Job ID 104335 Work...  ...research, model experimentation, and evaluation design to production system...  ...For (Qualifications):8-12+ years in software engineering, ML engineering, or applied... 
    Software
    Permanent employment
    Full time
    Apprenticeship
    Work at office
    Local area
    Work from home
    Home office
    3 days per week

    Bain & Company

    New York, NY
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!

Related searches