User Research Expert - Evaluator
$80 - $120 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: User/customer research and feedback synthesis Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
- Provide clear, structured written feedback to improve AI model outputs .
- Work independently and asynchronously to meet deadlines and enhance AI model performance .
- Collaborate with subject matter experts to ensure consistency and quality across datasets.
Qualifications
Must-Have
- 5+ years of relevant professional experience in User/customer research and feedback synthesis .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree (Master's or higher) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the User Research Expert - Evaluator in New York, NY vacancy
- ...cutting‑edge technology and expert insights. Backed by EQT, a top... ...Website: Linkedin Job Title: Evaluator - Political Science Location:... ...the question posed in the user prompt. The goal is to assess... ...Stay updated with the latest research, guidelines, and advancements...SuggestedContract workRemote workWorldwide
- Mercor is hiring experienced musicians to evaluate generative music AI models, in partnership with a leading AI lab. You will assess AI-generated music across a wide range of genres and rate it against detailed quality standards. Start date: immediate. Duration: up to...SuggestedHourly payImmediate start
- Obsidian is seeking a Spanish (Mexico) Audio Generalist Evaluator Expert for a high-impact audio AI research project. The role focuses on transcription, annotation, and evaluation tasks to help train advanced language models. Candidates need to be fluent in Spanish (Mexico...SuggestedTemporary work10 hours per week
$100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ...Larry Summers , and Jack Dorsey . Position: ChatGPT Power User Type: Contract Compensation: $100 total Location:...SuggestedHourly payContract workSummer workRemote workMercor
New York, NY2 days ago- ...opportunity to utilize technology skills to advance rural agricultural development. Volunteers will be experienced program developers and user experience analysts with demonstrated skills in UI/UX and Salesforce programming to support the redesign of web and mobile...SuggestedRemote work
- 1. Role Overview Mercor is partnering with a leading AI research organization to engage experienced sales engineers for a project focused on evaluating how well AI systems perform real-world technical sales work. Rather than producing deliverables yourself, you will define...
$30 per hour
...human data in the world. Over 35,000 AI developers, researchers, and organizations use Prolific to gather data from paid... ...Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image generation...Remote jobWork from homeFlexible hours$177k - $260k
...private content includes equity research, company filings, event transcripts, expert calls, news, trade journals, and... ...content capabilities that enable users to unearth even more comprehensive... ...with data analysis and SQL, can evaluate quality in AI-generated outputs,...Local areaShift work- Seeking a full-time Remote AI Research Evaluator with a PhD in Quantitative Finance to assess and enhance AI models' capabilities in financial reasoning and quantitative analysis through flexible, contract-based work. Key responsibilities Assessing the factuality and...Remote jobFull timeContract workFlexible hours
$40 - $50 per hour
...A remote training organization is seeking a Sociology Expert to analyze humanities texts and synthesize themes across cultures. Offering a pay range of $40 to $50 per hour for a commitment of 10–40 hours per week. Candidates should possess a Bachelor's degree in a relevant...Hourly payRemote work10 hours per weekCrossing Hurdles
New York, NY3 days ago$170 per hour
...Subject Matter Expert: Legal & Regulatory Research (Remote Advisory)About ProlificProlific is not just another player in the AI space – we are building the biggest pool of quality human data in the world.Over 35,000 AI developers, researchers, and organizations use Prolific...Hourly paySelf employmentRemote workWork from homeFlexible hoursProlific
New York, NY1 day ago$55 - $65 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Role Responsibilities Review and evaluate AI-generated clinical outputs based on... ...for AI training datasets . Provide expert feedback on nursing assessments and documentation...Contract workSummer workRemote workMercor
New York, NY2 days ago- ...Job Description Job Description Research & Insights Expert – PowerPoint Review & Reconstruction Role Type: Contractor Location: Remote... ..., or business analytics. Experience with AI training, evaluation, or rubric development. Strong ability to translate complex...For contractorsRemote work
YO AI Labs
New York, NY10 days ago - ...Job Description Job Description Equity Research Expert Role Type: Contractor Location: Remote We are seeking experienced Equity... ..., and sector insights. You'll apply your expertise to evaluate real-world equity research content and help train next-generation...For contractorsRemote work
YO AI Labs
New York, NY13 days ago $80 - $110 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...Up to 20 hours/week Role Responsibilities Review and evaluate research papers in chemistry for correctness, novelty, and...Contract workPart timeSummer workRemote workMercor
New York, NY2 days ago$120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Dorsey . Position: Equity Research Expert Type: Contract Compensation:... ...filings to support AI model training . Evaluate the accuracy of AI-generated equity...Hourly payContract workFor contractorsSummer workRemote workMercor
New York, NY2 days ago$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...and Jack Dorsey . Position: Process improvement / SOPs Evaluator Type: Contract Compensation: $80–$120/hour...Contract workSummer workWork at officeRemote work$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...and Jack Dorsey . Position: Training / onboarding / L&D Evaluator Type: Contract Compensation: $80–$120/hour...Contract workSummer workWork at officeRemote work$50 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract... ...Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar...Contract workSummer workRemote workFlexible hours$14 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Dorsey . Position: Music Audio Expert - Telugu Type: Contract Compensation... ...hours/week Role Responsibilities Evaluate AI model output lyrics , voice...Contract workSummer workImmediate startRemote work$80 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Dorsey . Position: UI / UX Design Expert Type: Contract Compensation:... ...Work independently and asynchronously to evaluate and improve AI model performance ....Contract workSummer workRemote work$150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... .... Position: Investment Banking Expert Type: Contract Compensation:... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Contract workSummer workRemote work$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Jack Dorsey . Position: B2B Sales Expert Type: Contract Compensation:... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Contract workSummer workRemote work$90 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use...Contract workSummer workLocal areaRemote work$85 - $142 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...Summers , and Jack Dorsey . Position: Preclinical R&D Expert – In-house Drug Development Type: Contract...Contract workSummer workRemote workMercor
New York, NY2 days ago$60 - $90 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...sound scientific reasoning versus superficial reasoning. Evaluate models by reviewing attempts and identifying immediate mistakes...Contract workSummer workImmediate startRemote workMercor
New York, NY2 days ago$75 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...Preferred ~ Prior data annotation, labeling, or AI evaluation experience Application Process (Takes 20–30 mins to complete...Contract workSummer workRemote workMercor
New York, NY2 days ago$14.5 per hour
...apply for the AI Web Search Evaluator role at Welo Data Welo Data... ...0,000 AI training and domain experts. As a Web Search Evaluator... ...engine results, ensuring users find the most relevant and useful... ...and digital trends. Enjoy researching topics online and provides...Hourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hoursWelo Data
New York, NY1 day ago$60 - $70 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy...Contract workSummer workRemote work$80 - $135 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...Summers , and Jack Dorsey . Position: Research Physics Expert Type: Contract Compensation: $80–$135/hour...Contract workSummer workRemote workMercor
New York, NY2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to User Research Expert - Evaluator. Be the first to apply!
Related searches
- guest service support expert New York, NY
- fulfillment expert New York, NY
- subject matter expert New York, NY
- sql expert New York, NY
- technology expert New York, NY
- clinical evaluator New York, NY
- ai evaluator New York, NY
- quality evaluator New York, NY
- social media evaluator New York, NY
- education evaluator New York, NY