Analytical Evaluator - AI Feedback - AI Trainer
$70 per hourFull-time
Mercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
- Ensure consistent and honest judgment, including critical assessments when warranted.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-500 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Native English fluency required.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on click.appcast.io
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
#hiringmercorVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback - AI Trainer in Seattle, WA vacancy
$136k - $184k
...and translate business problems into data science or analytical problems- Design, develop, and evaluate highly innovative statistics and ML models- Guide and... ...performance experience- 1+ years of working with or evaluating AI systems experience- Experience applying theoretical...SuggestedFull timeTemporary workSeasonal workFlexible hoursAmazon
Bellevue, WA3 days ago$136k - $184k
...about large-scale data challenges and motivated to apply ML/AI and advanced analytics to solve real customer problems.Reporting to the Science... ...group of researchers experience- 1+ years of working with or evaluating AI systems experience- 1+ years of creating or...SuggestedFlexible hoursAmazon
Seattle, WA1 day ago$136k - $184k
Are you excited about driving high-visibility, strategic analytics solutions to streamline Amazon's capacity planning? NACP - Analytics &... ...Our mission is to simplify analytics with innovative data science/AI products.This role is for a motivated individual with a strong...SuggestedFlexible hoursAmazon
Bellevue, WA4 days ago$159.2k - $215.3k
Are you passionate about solving complex logistics challenges? Our Analytics team is at the forefront of enhancing delivery experiences through data-driven solutions and innovative technology. As a Research Scientist, you will join a team dedicated to optimizing our delivery...SuggestedImmediate startFlexible hoursAmazon
Bellevue, WA5 days ago$136k - $184k
...successful in the role, you will need deep analytical skills and a strong scientific... ...justify trade-offs. Drive excellence in evaluation: ground-truth construction with Quality... ...1+ years of working with or evaluating AI systems experience- 1+ years of creating...SuggestedFlexible hoursAmazon
Bellevue, WA2 days ago- ...Overview: The Reporting & Analytics Consultant IV is a technical leader responsible for... ...to, seeking, and addressing performance feedback; providing feedback to others; creating... ...while ensuring consistency and coherency; evaluating and summarizing data and results;...
Kaiser Permanente
Renton, WA5 days ago - Senior AI Trainer Role Type: Contractor, Remote Location: North America, Latin America, Europe... ...in quality reviews and incorporate feedback to maintain standards. Required... ...independently in a remote environment. Analytical and problem-solving mindset. Preferred...Remote jobFor contractors
$132.1k - $178.8k
...above all else, is passionate about data and analytics. They are an expert with data modeling,... ...support forecasting, experimentation, ML/AI use cases, self-service analytics, and... ...domain- Build guardrails, validation, and evaluation mechanisms, both automated and human-in-the...Flexible hoursAmazon
Seattle, WA5 days ago$63 - $68 per hour
...Austin, TX / Seattle, WAAbout the Role:We build data systems that power innovation at scale. The Data Engineering team partners with analytics, finance, and product organizations to design and build next-generation data platforms that enable critical business and customer...Hourly payFull timeSRI Tech
Seattle, WA1 day ago$132.1k - $178.8k
...customers in over 240 countries and territories.The Prime Video Product Analytics (PVPA) team is a centralized analytics organization supporting... ...development to advanced analytics, customer insights modeling, AI-driven solutions, and self-service reporting.This role sits...Flexible hoursAmazon
Seattle, WA3 days ago$142.8k - $193.2k
...of years. Amazon Live has a mature data platform powering reporting across 9 marketplaces, 14+ dashboards, and real-time creator analytics. The opportunity ahead is different: proving the causal value of video to brands, Amazon's programmatic systems, and product decision...Flexible hoursAmazon
Seattle, WA2 days ago$154.6k - $209.1k
The AWS Analytics Engineering (AAE) organization is the analytics backbone of AWS — we build and operate the data platform that powers business... ...and Lambda. The candidate should have hands-on experience with AI/ML technologies, including LLMs, and a strong understanding of...Temporary workFlexible hoursAmazon
Seattle, WA23 hours ago$132.1k - $178.8k
The AWS Analytics Engineering (AAE) organization is the analytics backbone of AWS — we build and operate the data platform that powers business... ...data sets from multiple sources- Experience working with Data & AI related technologies, including, but not limited to, AI/ML,...Flexible hoursAmazon
Seattle, WA4 days ago$189.4k - $256.2k
...products? Do you want to influence the AWS investment in Cloud and AI technology and see how your recommendations influencing AWS VP... ...modeling, causal inference, econometrics, product growth analytics, and forecasting models) and democratize how AWS senior leaders...Flexible hoursAmazon
Seattle, WA1 day ago$149k - $253.5k
...organization. The Team Data & Analytics is building the DNA of an AI native Scribd, Inc. Bringing... ...quality, observability, lineage, and evaluation systems that make these assets trustworthy... ...because transparency, thoughtful feedback, and visible decision making build...Full timeFor contractorsLocal areaImmediate startHome officeFlexible hours- Pinterest is seeking a Sr. Software Engineer, Data Analytics and Engineering to improve the quality, reliability and velocity of data science... ...investigations across cross-functional teams. You’ll apply AI to accelerate analysis while upholding data governance, collaborate...
$130.4k - $176.3k
We are looking for a Senior Business Intelligence Engineer to build the diagnostic analytics layer for Amazon's global data center operations. You will move our analytics capability beyond reporting *what* happened to explaining *why* — identifying which factors drive metric...Flexible hoursAmazon
Seattle, WA1 day ago$82.6k - $162.8k
...forecasting across finance and other functions. Using advanced, scalable data and tech platforms, we also generate actionable insights and analytics, streamline decision making and enable data driven profitable growth. Manages risk and opportunities associated with diverse and...Local areaRemote workDeloitte
Seattle, WA5 days ago- Yoodli AI Roleplays in Seattle seeks a principal data scientist to own the data function end-to-end, from infrastructure to judgment... ...the data foundation serves internal decisions and customer analytics, while shaping product strategy and demonstrating #J-18808-Ljbffr...
- ...-performance research and trading infrastructure. The position requires a strong quantitative degree and advanced C++ skills, with a passion for research, collaboration, and solving complex problems with rigorous analytics. #J-18808-Ljbffr Evergreen Statistical Trading
- ...the Supply Chain BI & Data Science team to own architecture and strategy for core data models in Databricks and for the self-serve analytics platform across procurement, inventory, logistics, and supplier performance. This role blends data modeling expertise with...
$130.4k - $176.3k
...looking for a Sr. Business Intelligence Engineer to join the AWS Analytics Engineering (AAE) team supporting Amazon Quick. Formerly Amazon... ...evolved from a standalone BI service into a comprehensive, generative-AI-powered business intelligence platform that combines traditional...Work at officeFlexible hoursAmazon
Seattle, WA4 days ago$151.2k - $204.6k
-Amazon Web Services (AWS)'s Analytics Engineering organization is seeking a Senior Product... ...cloud architecture, and increasingly, the AI and machine learning workloads reshaping... ...Legal, Service Teams, and Account Teams to evaluate pricing strategies, structure complex...Flexible hoursAmazon
Seattle, WA5 days ago- Coinbase is seeking an Analytics Engineer to join the Compliance Technology team within the Platform organization. You will build end-to-end data pipelines powering compliance detection and decisioning, collaborating with Compliance, Product, and Engineering to ensure monitoring...Remote job
$183.04k - $320.32k
...career you love? It’s Possible.At Pinterest, AI isn't just a feature, it's a powerful... ...looking for a Sr. Software Engineer , Data Analytics and Engineering to improve the quality,... ...workflows.Strong track record of critically evaluating and verifying AI-assisted work through...Work at officeLocal areaRelocationRelocation packagePinterest
Seattle, WA5 days ago$202.3k - $273.7k
The AWS Analytics Engineering (AAE) organization is the analytics backbone of AWS. We build and operate the data systems that drive business... ...team's architecture, tools and practices, including adopting AI/ML-assisted data engineering where it adds value.Set and enforce...Full timeTemporary workSeasonal workFlexible hoursAmazon
Seattle, WA3 days ago$285k - $380k
Recruiting Analytics Data EngineerNew York City, NY; San Francisco, CA | New York City, NY | Seattle, WA; Seattle, WAAbout AnthropicAnthropic... ...'s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and...Work at officeVisa sponsorshipFlexible hours$143.7k - $194.4k
...our team and drive the next generation of our data platform and AI powered products at AWS scale. You will own end to end design, implementation... ...orchestration systems, multi agent AI frameworks, GenAI powered analytics capabilities, and scalable platform services. You will operate...Temporary workInternshipFlexible hoursAmazon
Seattle, WA5 days ago$182k - $242k
CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams... ...March 2025. Learn more at What You’ll Do We're hiring a Senior Analytics Engineer to own analytics for the Machine Infrastructure Supply...Permanent employmentTemporary workCasual workWork at officeFlexible hours$100k - $300k
...and exceed the vision and mission of the business. As the Senior Analytics Engineer / Analytics Engineer, you will transform complex data... ...Customer Data (zero/first/third-party) preferred Curiosity to leverage AI to accelerate workflows and generate business solutions A...Work at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback - AI Trainer. Be the first to apply!
