AI Evaluator - Software Expert
$80 - $120 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in outputs.
- Provide clear, structured written feedback to improve quality.
- Work independently and asynchronously to meet deadlines.
- Utilize expertise in Microsoft Office and Google Workspace , especially Slides .
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree ( Master's or higher) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...SoftwareContract workSummer workWork at officeRemote work$40 per hour
Review and improve AI answers written in Maltese so they read naturally and get the facts right. You rate responses, fix weak ones and write examples that show the model what good looks like. What you will do Rate Maltese AI responses for accuracy, fluency, tone and instruction...SuggestedRemote jobHourly payWeekly payContract workPart timeFor contractorsFlexible hours- Vettedlance is seeking a meticulous Polish language reviewer to enhance AI answers. You will rate Polish responses for accuracy, fluency, and tone, rewrite weak answers, and ensure factual correctness. You will also fact-check claims, flag unsafe content, and craft prompts...SuggestedRemote jobHourly payPart timeFlexible hours
- Synthires is seeking a Film & Video Craft Expert to remotely evaluate AI-generated videos across lighting, framing, color, and performance. This part-time role emphasizes concise feedback and strong written communication. Applicants should be fluent in English and have...SuggestedRemote jobPart time
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location...SuggestedContract workSummer workWork at officeRemote work- Job Description/ Responsibilities: AI-Driven Test Automation Transformation: • Lead... ...adoption of GenAI-powered automation across the Software Testing Lifecycle (STLC), driving... ...Innovation & Experimentation: • Continuously evaluate emerging AI technologies and identify...Software
- Thomson Reuters is seeking a Sr. Legal AI Prompter/Evaluator in New York to lead hands‑on development of LLM‑driven legal tools. You will design... ..., run experiments, and build evaluation schemes with SMEs, software engineers, and data scientists to ensure accurate and...Software
- ...years of experience specializing in Microsoft, Salesforce, BI, AI, and Cloud Computing. We have established longstanding client relationships... ...and query performance tuning. Solid understanding of software development life cycle (SDLC) principles. Excellent problem-solving...SoftwareContract work
- Role overview Help design and implement systems that coordinate AI services and software components to support useful business workflows. The role sits at the intersection of product and engineering, with an emphasis on turning practical needs into working, connected solutions...SoftwareFull timeRemote work
- 3+ years of experience implementing AI or legal-technology solutions in a law firm, corporate... .../common-interest considerations, • Evaluate legal-AI vendors and platforms (e.g.,... ...deployment, • Apply security patches and software updates across endpoints and core servers...SoftwareFull timeContract workWork at officeRemote work
$20 - $45 per hour
Coding & Software Remote Hourly · $20-$45/hr $20-$45/hr Compensation Contract, part-time Engagement Remote Location Posted Open... ...United States Uzbekistan Vietnam The work You will help evaluate AI systems by reviewing software code, technical documentation, and...SoftwareHourly payContract workPart timeFor contractorsRemote work- Описание: Ciklum is looking for an Expert Back End Engineer to join our team full-time in... ...experiences. Задачи: Take ownership of the entire software development lifecycle-from ideation and... ...and actively experimenting with how AI changes Software Development Lifecycle 7+...SoftwareFull timeShift work
- ...Sciences Researchers with strong Python programming skills to contribute to projects evaluating and improving advanced AI systems. You will combine scientific expertise with practical software abilities to create, solve, and review challenging problems across biology and...SoftwareFlexible hours
$84k - $115k
Secure Every Identity, from AI to Human Identity is the key to unlocking the potential... ...2D/3D visualizations). Tool Mastery: Expert-level fluency with Figma and Adobe Creative... ...you have Working knowledge of 3D spatial software or spatial visualization tools (SketchUp,...SoftwareLocal areaWorldwideFlexible hours- ...Software Architect / Solutions ArchitectAt Air Apps, we believe in thinking bigger—and moving... ...on a mission to create the world's first AI-powered Personal & Entrepreneurial... ...role in shaping our technology strategy, evaluating new tools and frameworks, and ensuring best...SoftwareWork at officeWorldwideRelocation
$170k - $200k
...We are seeking an experienced Senior AI Engineer to serve as a hands‑on technical... ...and context engineering, model selection, evaluation, and token-usage optimization to improve... ...preferred. ~7+ years of experience in software, data, or machine learning engineering, including...SoftwareWork experience placementWork at officeRemote work- ...Get AI-powered advice on this job and more exclusive features. Direct message the job poster from Synergy Interactive Responsibilities... ..., and Writing/Editing Industries IT Services and IT Consulting, Advertising Services, and Software Development #J-18808-Ljbffr...SoftwareContract work
- ...edge Artificial Intelligence (AI) based product designed to... ...experienced, Subject Matter Experts (SMEs) to partner with our Product... ...Medical Bills, etc.) Product Evaluation: Actively use and evaluate... ...with computers and learning new software applications. Familiarity...Software
$125k - $225k
...Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing...SoftwarePermanent employmentWork experience placementWork at officeLocal area3 days per week- ...for a Technical Solution Architect with AI, Azure and .NET expertise. This is a 12+month... ...Solutions Architect with 12+ years of software development experience , including 5+... ...model integration, AI observability, model evaluation, and Human-in-the-Loop controls . ·...SoftwareLocal areaRemote work3 days per week
- Code4nord is seeking a skilled AI Agent Developer for an exciting private-sector project. This fully remote role involves designing... ...shaping project direction. The ideal candidate will have a strong software development background and hands-on experience in AI. Team...SoftwareRemote jobFull time
- Role Title: AI Product Evaluator Role Type: Contractor Location: India, Japan or United Kingdom, Remote Our team is engaging expert AI Product Evaluators to contribute directly to a customer... ...participation in digital product, software QA, or AI research projects is a plus...SoftwareFor contractorsRemote work
- ...the architecture, design, and technical governance of a modern, AI-enabled enterprise application platform built using C#, .NET 8+,... ...This senior technical leadership role requires 12+ years of software development experience, including at least 5 years serving as a...SoftwareFull timeContract work
- ...services organization is seeking an accomplished Director-level Software Engineer to design and develop cloud-native applications supporting... ...intelligence applications. Modern data engineering or analytics tools. Exposure to enterprise AI-enabled applications....Software
- Thomson Reuters seeks a Sr. Legal AI Prompter / Evaluator to advance CoCounsel Legal through prompts, skills, and agent configurations. You will... ...accurate and legally sound. You will partner with SMEs, software engineers, and data scientists to improve product quality....Software
$65 - $75 per hour
job summary: Senior AI Developer We're looking for a Senior AI Agent Developer... ...and intent classification standards and evaluate changes against measurable quality criteria... ...practical experience 8+ years of software engineering experience, including at least...SoftwareHourly payContract work$160k - $180k
...Artemis Artemis is building the future of AI-driven cybersecurity, helping companies... ...learning from Oxford. Our team of software engineers, AI researchers, security engineers... ...administration and lifecycle management: evaluating tools, managing licenses, and cutting...SoftwareFull time- ...of business-critical calculation logic. Establish responsible AI-assisted engineering practices, enabling the team to use... ...practices, observability and operational resilience. AI-assisted software engineering: Experience using AI-powered development tools such...SoftwareFull timeLocal area
$40 per hour
...STEM training data for frontier AI models. Our data is used directly in training and evaluation pipelines at leading AI labs to... ...domains. We are looking for experts in Physics to design rigorous,... ...tools or domain-specific physics software (e.g., finite element tools, circuit...SoftwareContract workPart timeRemote work$135k
...application development, AWS cloud-native technologies, and modern AI/GenAI solutions. The ideal candidate will provide technical... ...Financial Services domain. Qualifications • 8+ years of overall software development experience, with strong experience in .NET...SoftwareFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
- evaluator New York, NY
- work from home web search evaluator New York, NY
- program evaluator New York, NY
- clinical evaluator New York, NY
- work from home social media evaluator New York, NY
- ai evaluator New York, NY
- education evaluator New York, NY
- quality evaluator New York, NY
- social media evaluator New York, NY
- transcript evaluator New York, NY






