JSON Evaluator - Fully Remote | Upto $175/hr
$175 per hourMercor
San Francisco, CA
- Remote job
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: JSON-fluent & Rubrics Expert
Type: Contract
Compensation: $50–$175/hour
Location: Remote
Commitment: 20–40 hours/week
Role Responsibilities
- Evaluate JSON-formatted AI model outputs against defined grading rubrics.
- Read and understand written task descriptions and rubrics.
- Parse, read, and assess JSON-based outputs, grading them against provided rubrics.
- Ensure attention to detail and consistency in applying rubric criteria.
- Work independently to meet tight deadlines within 12-24 hours .
Qualifications
Must-Have
- Available to start today, July 24 , running to July 27-28 .
- Comfortable reading and interpreting JSON files .
- Based in the US .
- Access to a desktop or laptop computer (no Chromebooks).
Preferred
- Prior Mercor experience, especially in applying detailed rubrics and evaluation criteria consistently.
Start Date
- Today, July 24
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the JSON Evaluator - Fully Remote | Upto $175/hr in San Francisco, CA vacancy
$120 per hour
About the role We are hiring expert Evaluators in Real estate / hospitality / events to review and assess AI-generated work products (documents... ...deep subject-matter expertise to grade outputs. This is a remote, hourly engagement. Requirements (must have) 5+ years of...Remote jobHourly payWork at office$20 per hour
...English & Urdu Type Contract Compensation $15-$20/hour Location Remote Role Responsibilities Conduct fact-checking using trusted public sources and external tools. Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and...Remote jobContract workPart timeSummer work$120 per hour
...Larry Summers , and Jack Dorsey . Position: Investment analysis / valuation / credit Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality...Remote jobContract workSummer workWork at office$17 per hour
...Audio Expert - Thai Type: Contract Compensation: $17/hour Location: Remote Duration: Up to 6 months Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards in various music...Remote jobContract workSummer workImmediate start$50 per hour
...Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Dutch Audio Generalist Evaluator Expert Type: Contract Compensation: $50/hour Location: Remote Duration: 3–6 months Role Responsibilities Transcribe audio and video with...Remote jobContract workSummer work$120 per hour
..., Larry Summers , and Jack Dorsey . Position: Media / journalism / communications Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality...Remote jobContract workSummer workWork at office$120 per hour
..., Larry Summers , and Jack Dorsey . Position: Media / journalism / communications Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality...Remote jobContract workSummer workWork at office$190 per hour
...Plug In Connectors Expert Type: Contract Compensation: $50–$190/hour Location: Remote Commitment: 20+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows, including personal health , travel , and activity...Remote jobHourly payContract workFor contractorsSummer workTrial period$50 - $175 per hour
...Summers , and Jack Dorsey . Position: JSON-fluent & Rubrics Expert Type: Contract Compensation: $50–$175/hour Location: Remote Commitment: 20–40 hours/week Role Responsibilities Evaluate JSON-formatted AI model outputs against defined...Remote workContract workSummer workStart working today$175 per hour
...Larry Summers , and Jack Dorsey . Position: CNC Machining Expert Type: Contract Compensation: $120–$175/hour Location: Remote Role Responsibilities Contribute to text-only, objective, verifiable rubrics for grading CNC machining solutions...Remote jobContract workPart timeSummer work$20 per hour
A tech company is seeking a Receptionist to assist in training AI models. This role involves evaluating the logic of AI chatbots and ensuring quality in their outputs. Applicants should be fluent in English, detail-oriented, and possess strong writing skills. This position...Remote workHourly payFlexible hoursDataAnnotation
New York, NY4 days ago$30 per hour
Type: Hourly contract Compensation: $20-$30/hour Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous agent systems using defined rubrics and quality standards. Review multi-step agent workflows,...Remote jobHourly payContract work$70 per hour
Mercor is seeking a Generalist Expert to improve AI model performance by evaluating AI-generated responses and providing structured feedback. This remote position requires strong analytical and written communication skills, along with a Bachelor's degree from a top-500...Remote jobContract work$39 per hour
...Expert - English (UK/Australia) Type: Hourly contract Compensation: $39 per hour Location: Remote Commitment: Part-time, 10+ hours per week Role Responsibilities Evaluate AI model output lyrics, voice generation, and other standards Score quality of musical...Remote jobHourly payContract workPart time10 hours per week$208k - $416k
...and next steps. Our partner is looking for an AI Interaction Evaluator based in the United States. This is a senior-level contract... ...-assisted development should look like. The engagement is fully remote, flexible, and designed for experienced engineers who want to...Remote workHourly payContract workTemporary workImmediate start10 hours per weekFlexible hours$17 per hour
...Dorsey . Position: Personal Care Aide - ONET Occupation Study Type: Contract Compensation: $17/hour Location: Remote Role Responsibilities Complete a one-time online survey/interview about your daily work as a personal care aide. It takes up...Remote jobContract workSummer workMercor
New York, NY6 days ago$142.38k - $175.94k
...also have up to four weeks per year of fully remote work! Responsibilities Lead sourcing... ...and social platforms. Screen and evaluate candidate qualifications, technical skills... ...salary range for this role is $142,380-$175,941. However, the starting base pay will...Remote workFull timeWork at officeWork from homeBrex
Washington DC14 hours ago$40 per hour
...leading AI development company is seeking experienced quantitative professionals to evaluate AI-generated analyses and design quantitative problems for AI training. This fully remote role offers flexibility in project selection and scheduling, with competitive pay starting...Remote workHourly payDataAnnotation
Juneau, AK3 days ago$40 per hour
...A leading AI development firm is looking for experienced quantitative professionals to evaluate AI-generated work and design problems for AI training. This fully remote position allows for a flexible schedule, offering competitive pay starting at $40+ per hour. Ideal candidates...Remote workHourly payFlexible hoursDataAnnotation
Bismarck, ND3 days ago- ...contracting company is seeking skilled Vectorworks users based in the United States for part-time, fully remote projects related to AI research. Individuals will evaluate AI-generated content based on their architectural design experience, making this an ideal chance for...Remote workPart timeFlexible hours
Handshake
New York, NY1 day ago $40 per hour
...innovative AI development company is seeking experienced quantitative professionals to contribute to AI advancements. This fully remote role involves evaluating AI-generated analyses and ensuring they are technically accurate and valid in real-world scenarios. Candidates...Remote workHourly payFlexible hoursDataAnnotation
Santa Fe, NM4 days ago$40 per hour
...development firm seeks experienced quantitative professionals to evaluate AI-generated work, applying their skills in statistical analysis, predictive modeling, and technical writing. This fully remote opportunity offers a flexible schedule and projects paid hourly, starting...Remote workHourly payFlexible hoursDataAnnotation
Sioux Falls, SD3 days ago$40 per hour
...development firm is seeking experienced quantitative professionals to evaluate AI-generated analysis and solve complex technical problems.... ...methods, and experience with analytical coding. This fully remote position allows for flexible scheduling and competitive hourly...Remote workHourly payFlexible hoursDataAnnotation
Oklahoma City, OK3 days ago$40 per hour
...thinking AI solutions company is seeking experienced quantitative professionals to evaluate AI-generated analyses and contribute to the development of cutting-edge AI systems. This fully remote role offers flexible scheduling and competitive pay starting at $40+ per hour....Remote workHourly payFlexible hoursDataAnnotation
Lincoln, NE3 days ago$40 per hour
...firm is looking for experienced quantitative professionals to evaluate AI-generated quantitative work. Candidates should have 2+ years... ...providing feedback to improve AI models. This position offers fully remote work with a flexible schedule and competitive pay starting at...Remote workHourly payFlexible hoursDataAnnotation
Montgomery, AL3 days ago$40 per hour
...A leading AI development firm is seeking experienced quantitative professionals to evaluate AI-generated work and solve complex problems related to data analysis. This fully remote role allows flexibility in schedule and project selection, catering to those with a background...Remote workHourly payDataAnnotation
Boston, MA4 days ago$40 per hour
...A forward-thinking analytics company is seeking quantitative professionals to evaluate AI-generated work, ensuring accuracy in statistical analysis and predictive modeling. This fully remote role offers a flexible schedule and competitive pay starting at $40 per hour....Remote workHourly payFlexible hoursDataAnnotation
Saint Paul, MN3 days ago$40 per hour
...A leading AI development firm is seeking experienced quantitative professionals to evaluate AI-generated quantitative analysis and provide impactful feedback. This fully remote role allows for flexible scheduling and competitive pay starting at $40 per hour. Candidates...Remote workHourly payFlexible hoursDataAnnotation
Honolulu, HI4 days ago$40 per hour
...annotation company is seeking professionals in quantitative fields to enhance AI development. This fully remote role allows individuals to set flexible schedules while evaluating AI-generated analyses and solving complex quantitative problems. Candidates should have 2+...Remote workHourly payFlexible hoursDataAnnotation
Oregon, WI2 days ago$60 per hour
A cutting-edge AI company is looking for experienced quantitative professionals to evaluate AI-generated statistical analyses and provide crucial feedback. In this fully remote role, you can set your own schedule and earn competitive pay up to $60 per hour. Ideal candidates...Remote workHourly payDataAnnotation
Louisiana, MO3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to JSON Evaluator - Fully Remote | Upto $175/hr. Be the first to apply!
Related searches
- work from home web search evaluator San Francisco, CA
- evaluator San Francisco, CA
- ai evaluator San Francisco, CA
- quality evaluator San Francisco, CA
- education evaluator San Francisco, CA
- social media evaluator San Francisco, CA
- operations coordinator remote San Francisco, CA
- head of marketing remote San Francisco, CA
- content designer remote San Francisco, CA
- remote coordinator San Francisco, CA