AI Evaluator - Software Expert
$80 - $120 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
- Provide clear, structured written feedback to improve AI model outputs .
- Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
- Work independently and asynchronously to meet deadlines.
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree ( Master's or higher ) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in New York, NY vacancy
- A tech company focusing on AI research is looking for experienced Krita users for a flexible, project-based contract opportunity. This role allows you to earn while evaluating AI-generated content related to digital painting and concept art. Candidates should have at least...SuggestedRemote jobContract workFlexible hours
$60 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance...SuggestedContract workSummer workRemote workMercor
New York, NY3 days ago$30 per hour
About Prolific Prolific is not just another player in the AI space - we are building the biggest pool of quality human... ...Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image generation...SuggestedRemote jobWork from homeFlexible hours- ...Remote Mathematics Expert (AI/LLM) - 34877 Remote Mathematics Expert (AI/LLM) - 34877 1... ...researchers to align problem types with evaluation goals, particularly in areas where models... ...Information Technology Industries Software Development Referrals increase your chances...SoftwareHourly payContract workPart timeFor contractorsFreelanceInternshipRemote workWork from homeWorldwideAfternoon shift
Turing Inc
New York, NY3 days ago - ...solutions that help win more projects. We recently announced a new AI-Proposal product Shred.ai to continue this mission. We’re a... ...WordPress plug‑ins, and workflow automation. Familiarity with AEC software (Deltek, Unanet, Microsoft Dynamics) and DAM workflows. Customer...SoftwareWork at officeLocal areaWork from home2 days per week
OpenAsset
New York, NY2 days ago $190k - $232k
...Our team is comprised of customer strategy experts, Salesforce-certified experts across all... ...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...following: Boomi, Informatica, TIBCO, Software AG, CastIron, Jitterbit, or similar products...SoftwareTemporary workLocal area$118.98k - $195.47k
Lead Software Engineer About the TeamThe Group Benefits API Team is responsible for building and operating APIs and integrations that support... ...is passionate about building robust platforms, enabling Agentic AI solutions, and creating meaningful impact through high‑quality...SoftwareFull timeVisa sponsorshipWork visaFlexible hours$128.5k - $171.5k
General Information Job Title Expert Consultant, Coro, AI Engineer Job ID 106501 Work Areas... ...to ensure commercial success. Coro’s software and data solutions are an innovative... ...skills as needed, including building evaluation approaches, working with data...SoftwarePermanent employmentFull timeTemporary workApprenticeshipLocal area$115k - $141k
...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...including configuration of the FSC Platform* Expert level understanding of various components... ...enterprise systems as part of a complete software product lifecycle* Familiarity with agile...SoftwareTemporary workLocal area$224k - $308k
Secure Every Identity, from AI to HumanIdentity is the key to unlocking the potential of AI. Okta secures AI by building the trusted... ...Knowledge, Skills, and Abilities:8+ years of overall IT / software development experience, solution design and technical architecture...SoftwareLocal areaWorldwideFlexible hours- ...knowledge of CI/CD pipelines and version control systems (e.g., GitHub, GitLab, Bitbucket) ~ Experience using AI-assisted coding tools in professional software development (e.g., GitHub Copilot, Cursor, Claude, or similar) ~ Experience working in regulated...SoftwareWork at officeShift work2 days per week
Blue Shield of CA
New York, NY1 day ago - ...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...deserve better. As a Salesforce subject matter expert and healthcare technologist, you take... ...(3-5+ years) experience with declarative software configuration (Epic/Cerner, Veeva, etc.)...SoftwareLocal areaShift work
- ...Requirements for the Senior React & Ruby Position: 4+ years of software development experience. Commercial experience: React.js 5+ years... ...+ years, and Next.js: 2+ years Expertise in AWS, TypeScript and AI-assisted coding is a must. Other requirements: Strong technical...SoftwareFull timeRemote work
LEMON LLC
New York, NY21 hours ago - ...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...This role combines solution architecture, software development, and technical thought... ...augmented generation, model orchestration, evaluation, and responsible AI controls. • Working knowledge...SoftwareTemporary workLocal area
$293.5k
General Information Job Title Expert Senior Manager, AI Engineering Job ID 104335 Work... ...research, model experimentation, and evaluation design to production system... ...For (Qualifications):8-12+ years in software engineering, ML engineering, or applied...SoftwarePermanent employmentFull timeApprenticeshipWork at officeLocal areaWork from homeHome office3 days per week$180k - $270k
...ID, enterprise SSO/SCIM, and our SaaS and AI platforms at the center. You’ll design... ...enjoy this role if you are…Technical Domain Expert: Deeply fluent in Microsoft Entra ID (... ...reduce toil over time.Endpoint Lifecycle & Software Distribution: Own the architectural engineering...SoftwareRemote work- ...About Beacon Software Beacon is acquiring and operating a portfolio of vertical SaaS companies. Most private equity firms scale by adding... ...infrastructure, observability, or developer tooling projects. How We Use AI in Our Hiring Process: To ensure transparency, we want...Software
BEACON SOFTWARE COMPANY
New York, NY21 hours ago $172.5k - $260.1k
...DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...visionary technical leader to guide and support software engineers working on Workday Finance,... ...tools to help our recruiters assess and evaluate candidates’ resumes and qualifications throughout...SoftwareFull time$100k - $143k
...foundations that power enterprise intelligence, reporting, and AI-driven decision-making. Working closely with senior analytics engineers... ...practices, Agile delivery methodologies, and collaborative software development workflows.Interest in artificial intelligence,...SoftwareLocal area3 days per week$40 per hour
A leading AI training company in the United States is seeking a Software Engineer to evaluate and improve AI chatbots. The role offers flexibility to choose projects and work hours, as well as payment starting at $40 per hour. Candidates should be proficient in Python or...SoftwareRemote jobHourly payContract work- ...the boundaries of what’s possible. We invest heavily in advanced AI capabilities—specifically our Process Intelligence Graph—to turn... ...having served as a CTO at a consulting firm, as Field CTO at a software vendor, or Director/VP of Enterprise Architecture within a major...SoftwareImmediate start
EngineersOfAI
New York, NY3 days ago - Enterprise AI is moving from pilots to production, and the constraint is no longer the... ...AI platform.Own and streamline the evaluation process, guiding prospects through workshops... ...engineering role, selling complex enterprise software solutions.Strong ability to lead...SoftwareWorldwide
$130k - $160k
...Science, Engineering, or related field10+ years of professional software development experience, with 3+ years in an architect or technical... ...opportunity regardless of any basis protected by law.Use of AI in Recruitment:Globant may use AI and machine‑learning technologies...SoftwareRemote work- ...project, including investigation, evaluation, and recommendation of... ...Proficient in 3D modeling using software such as Revit, Rhino,... ...unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr...SoftwareFull timeWork at office
MORRIS ADJMI ARCHITECTS
New York, NY21 hours ago $70 - $75 per hour
...Get AI-powered advice on this job and more exclusive features. Job Title: Full Stack... ...DevOps, and other developers throughout the software development lifecycle. Participate in... ...unlocking community knowledge in a new way. Experts add insights directly into each article,...SoftwareFull timeRemote workVeracity Software Inc
New York, NY1 day ago$134k - $265k
...Infrastructure Patching Automation (Mainframe & AS/400)Join our AI & Engineering team by transforming technology platforms, driving... ...deploy, and operate integrated/verticalized sector solutions in software, data, AI, network, and hybrid cloud infrastructure. These solutions...SoftwareLocal area- ...Integration, Operating Systems Management, Problem Solving, Programming, Software Development, Software Development Life Cycle, Web Application... .... We equip our associates with future-focused skills and AI tools that enable us to advance our mission. Together, we are fighting...SoftwareFull timeWork experience placementWork at office
- ...knowledge of CI/CD pipelines and version control systems (e.g., GitHub, GitLab, Bitbucket) Experience using AI-assisted coding tools in professional software development (e.g., GitHub Copilot, Cursor, Claude, or similar) Experience working in regulated environments (...SoftwareWork at officeShift work2 days per week
$116.88k - $158.13k
...professional achievement and career growth. The Innovation and AI Solutions Engineer position is part of the firm’s Innovation Department... ...solutions. The role will support our attorneys and staff with software development, low/no-code solutions, and data initiatives. The...SoftwareFull timeWork at officeRemote workWorldwide$131.3k - $177.6k
...experience- 4+ years of specific technology domain areas (e.g. software development, cloud computing, systems engineering, infrastructure... ...modernizing customer requirements to the cloud- Proficient in the use of AI coding tools to improve quality and velocity.- Experience...SoftwareFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
Related searches
- program evaluator New York, NY
- work from home web search evaluator New York, NY
- clinical evaluator New York, NY
- evaluator New York, NY
- ai evaluator New York, NY
- work from home social media evaluator New York, NY
- quality evaluator New York, NY
- education evaluator New York, NY
- social media evaluator New York, NY
- subject matter expert New York, NY