AI Evaluator - Software Expert
$80 - $120 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in outputs.
- Provide clear, structured written feedback to improve quality.
- Work independently and asynchronously to meet deadlines.
- Utilize expertise in Microsoft Office and Google Workspace , especially Slides .
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree ( Master's or higher) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 7 days ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in New York, NY vacancy
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...SoftwareContract workSummer workWork at officeRemote work$30 per hour
...About Prolific Prolific is not just another player in the AI space – we are building the biggest pool of quality human... ...Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image generation...SuggestedRemote workWork from homeFlexible hoursProlific
New York, NY1 day ago- Mercor is seeking experienced musicians to evaluate generative music AI models, collaborating with a leading AI lab. You will assess AI-generated music and lyrics in Telugu and English, applying detailed quality standards across genres. Responsibilities include comparing...SuggestedPart timeImmediate start
- A tech company focusing on AI research is looking for experienced Krita users for a flexible, project-based contract opportunity. This role allows you to earn while evaluating AI-generated content related to digital painting and concept art. Candidates should have at least...SuggestedRemote jobContract workFlexible hours
- ...Overview Mercor is partnering with a leading AI research organization to engage... ...sales engineers for a project focused on evaluating how well AI systems perform real-world technical... ...experience with technically complex B2B software — SaaS, cloud infrastructure, cybersecurity...Software
- We are seeking experienced AI Safety Practitioners to evaluate the safety, quality, and alignment of frontier AI models across complex, policy-sensitive, and ambiguous ("grey-area") topics. You will assess AI-generated responses, apply safety policies, and help improve...Worldwide
- ...analyzes it for better understanding about the data set, and builds AI tools that automate certain processes. Duties typically include:... ...unique insights otherwise unobtainable. Familiarity with agile software development project management frameworks (e.g., Scrum, Kanban,...SoftwareWork at officeLocal area
$172.5k - $260.1k
...DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...visionary technical leader to guide and support software engineers working on Workday Finance,... ...tools to help our recruiters assess and evaluate candidates’ resumes and qualifications throughout...SoftwareFull time- ...Remote Mathematics Expert (AI/LLM) - 34877 Remote Mathematics Expert (AI/LLM) - 34877 1... ...researchers to align problem types with evaluation goals, particularly in areas where models... ...Information Technology Industries Software Development Referrals increase your chances...SoftwareHourly payContract workPart timeFor contractorsFreelanceInternshipRemote workWork from homeWorldwideAfternoon shift
Turing
New York, NY2 days ago - ...enterprise organizations accelerate digital transformation through data, AI, cloud, and integration solutions. We partner with industry-... ..., development, and implementation of advanced technologies, software solutions, and digital products. Our services include...SoftwareFull timeWorldwide
$164.5k - $219k
General Information Job Title Expert Senior Manager, Data Sciences, Software and Data Services, Coro Job ID... ...opportunities to work with major AI ecosystem partners through Bain's... ...the team applies to prompt design, evaluation and output quality assuranceDevelop...SoftwarePermanent employmentFull timeApprenticeshipLocal areaHome office3 days per week$190k - $232k
...Our team is comprised of customer strategy experts, Salesforce-certified experts across all... ...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...following: Boomi, Informatica, TIBCO, Software AG, CastIron, Jitterbit, or similar products...SoftwareTemporary workLocal area$115k - $141k
...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...including configuration of the FSC Platform* Expert level understanding of various components... ...enterprise systems as part of a complete software product lifecycle* Familiarity with agile...SoftwareTemporary workLocal area$224k - $308k
Secure Every Identity, from AI to HumanIdentity is the key to unlocking the potential of AI. Okta secures AI by building the trusted... ...Knowledge, Skills, and Abilities:8+ years of overall IT / software development experience, solution design and technical architecture...SoftwareLocal areaWorldwideFlexible hours- ...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...deserve better. As a Salesforce subject matter expert and healthcare technologist, you take... ...(3-5+ years) experience with declarative software configuration (Epic/Cerner, Veeva, etc.)...SoftwareLocal areaShift work
- ...Code4nord is seeking a skilled AI Agent Developer for an exciting private-sector project. This fully remote role involves designing... ...shaping project direction. The ideal candidate will have a strong software development background and hands-on experience in AI. Team...SoftwareFull timeRemote work
Code4nord
New York, NY1 day ago - ...mission-critical functions, or activate AI-powered innovation, we connect strategy to... ...This role combines solution architecture, software development, and technical thought... ...augmented generation, model orchestration, evaluation, and responsible AI controls. • Working knowledge...SoftwareTemporary workLocal area
- ...StatusNeo is a global AI-native transformation firm helping enterprises design, engineer... ...-driven industries to transform how software is built, operated, and scaled in an AI-first... ...knowledge systems. Experience evaluating LLM performance using frameworks such...SoftwareFull time
Statusneo
New York, NY1 day ago - ...solutions that help win more projects. We recently announced a new AI-Proposal product Shred.ai to continue this mission. We’re a... ...WordPress plug‑ins, and workflow automation. Familiarity with AEC software (Deltek, Unanet, Microsoft Dynamics) and DAM workflows....SoftwareWork at officeLocal areaWork from home2 days per week
- ...the boundaries of what’s possible. We invest heavily in advanced AI capabilities—specifically our Process Intelligence Graph—to turn... ...having served as a CTO at a consulting firm, as Field CTO at a software vendor, or Director/VP of Enterprise Architecture within a major...SoftwareImmediate start
EngineersOfAI
New York, NY2 days ago $180k - $270k
...ID, enterprise SSO/SCIM, and our SaaS and AI platforms at the center. You’ll design... ...enjoy this role if you are…Technical Domain Expert: Deeply fluent in Microsoft Entra ID (... ...reduce toil over time.Endpoint Lifecycle & Software Distribution: Own the architectural engineering...SoftwareRemote work- ...Vice President, IT Data, Analytics, AI & Innovation About the Company Recognized provider of enterprise software support services Industry Information Technology... ...cases. A key aspect of the position is to evaluate and scale emerging technologies that support...Software
- Enterprise AI is moving from pilots to production, and the constraint... ....• Own and streamline the evaluation process, guiding prospects... ...Solace Agent Mesh Subject Matter Expert • Position Solace Agent Mesh... ...role, selling complex enterprise software solutions.• Strong ability to...SoftwareWorldwide
$100k - $143k
...foundations that power enterprise intelligence, reporting, and AI-driven decision-making. Working closely with senior analytics engineers... ...practices, Agile delivery methodologies, and collaborative software development workflows.Interest in artificial intelligence,...SoftwareLocal area3 days per week- ...knowledge of CI/CD pipelines and version control systems (e.g., GitHub, GitLab, Bitbucket) Experience using AI-assisted coding tools in professional software development (e.g., GitHub Copilot, Cursor, Claude, or similar) Experience working in regulated environments (...SoftwareWork at officeShift work2 days per week
- ...Lineate Lineate is a US-based international software development company with over two decades... ...real, measurable results. We deliver AI-driven custom solutions for FinTech,... ...Team Augmentation: Providing specialized experts in FinTech, AdTech, and HealthTech to integrate...SoftwareFull timeContract workWork at office
Lineate
New York, NY1 day ago $293.5k
General Information Job Title Expert Senior Manager, AI Engineering Job ID 104335 Work... ...research, model experimentation, and evaluation design to production system... ...For (Qualifications):8-12+ years in software engineering, ML engineering, or applied...SoftwarePermanent employmentFull timeApprenticeshipWork at officeLocal areaWork from homeHome office3 days per week$70 - $75 per hour
...Get AI-powered advice on this job and more exclusive features. Job Title: Full Stack... ...DevOps, and other developers throughout the software development lifecycle. Participate in... ...unlocking community knowledge in a new way. Experts add insights directly into each article,...SoftwareFull timeRemote workVeracity Software Inc
New York, NY1 day ago$140k - $150k
...is looking for a Senior Data Engineer to help shape our data and AI technology roadmap. You'll own the full lifecycle of complex... ...data tooling.Solid grounding in data structures, algorithms, and software architecture, with experience documenting and testing complex systems...SoftwareWorldwide- ...About Beacon Software Beacon is acquiring and operating a portfolio of vertical SaaS companies. Most private equity firms scale by... ...the 8th wonder of the world - compounding growth. How We Use AI in Our Hiring Process: To ensure transparency, we want candidates...Software
BEACON SOFTWARE COMPANY
New York, NY3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
Related searches
- clinical evaluator New York, NY
- ai evaluator New York, NY
- work from home web search evaluator New York, NY
- social media evaluator New York, NY
- program evaluator New York, NY
- evaluator New York, NY
- work from home social media evaluator New York, NY
- education evaluator New York, NY
- quality evaluator New York, NY
- ads evaluator New York, NY