JSON Evaluator - Fully Remote | Upto $175/hr
$175 per hourMercor
San Francisco, CA
- Remote job
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: JSON-fluent & Rubrics Expert
Type: Contract
Compensation: $50–$175/hour
Location: Remote
Commitment: 20–40 hours/week
Role Responsibilities
- Evaluate JSON-formatted AI model outputs against defined grading rubrics.
- Read and understand written task descriptions and rubrics.
- Parse, read, and assess JSON-based outputs, grading them against provided rubrics.
- Ensure attention to detail and consistency in applying rubric criteria.
- Work independently to meet tight deadlines within 12-24 hours .
Qualifications
Must-Have
- Available to start today, July 24 , running to July 27-28 .
- Comfortable reading and interpreting JSON files .
- Based in the US .
- Access to a desktop or laptop computer (no Chromebooks).
Preferred
- Prior Mercor experience, especially in applying detailed rubrics and evaluation criteria consistently.
Start Date
- Today, July 24
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 20 days ago
Similar jobs that could be interesting for youBased on the JSON Evaluator - Fully Remote | Upto $175/hr in San Francisco, CA vacancy
$120 per hour
...Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey. Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Responsibilities Evaluate AI-generated artifacts against domain-specific quality rubrics....Remote workContract workSummer workWork at officeMercor Inc
United States2 days ago$120 per hour
...D'Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality...Remote jobContract workSummer workWork at office$90 per hour
.... Position: ML Challenge Task Auditor Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks for training and evaluating...Remote jobContract workSummer work$120 per hour
..., Larry Summers , and Jack Dorsey . Position: Real estate / hospitality / events Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality...Remote jobContract workSummer workWork at office$42 per hour
...Expert - Urdu Type: Contract Compensation: $14–$42/hour Location: Remote Duration: Up to 6 months Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres and rate it against quality standards....Remote jobContract workSummer workImmediate start$19 per hour
...Expert - Bengali Type: Contract Compensation: $11–$19/hour Location: Remote Duration: Up to 6 months Commitment: 20 hours/week Role Responsibilities Evaluate AI-generated music across diverse genres and rate against quality standards....Remote jobContract workSummer workImmediate start$90 per hour
...Position: AI Developer Trace Task Auditor Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality and correctness of AI-assisted software-development traces to enhance model training and evaluation...Remote jobContract workSummer work$160 per hour
...: Contract Compensation: $80–$160/hour Location: Remote Role Responsibilities Review and assess domain-specific documents... ...Provide structured feedback to improve document standards and evaluation processes. Utilize Excel , PowerPoint , and Word for...Remote jobContract workSummer work$17 per hour
...Audio Expert - Thai Type: Contract Compensation: $17/hour Location: Remote Duration: Up to 6 months Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards in various music...Remote jobContract workSummer workImmediate start$120 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Clinical / biomedical / pharma Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality...Remote jobContract workSummer workWork at office$90 per hour
...Dorsey . Position: SWE-Bench Task Auditor Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks . Assess repository-...Remote jobContract workSummer work$42 per hour
...Expert - Hebrew Type: Contract Compensation: $17–$42/hour Location: Remote Duration: Up to 6 months Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres for quality and creativity. Compare...Remote jobContract workSummer workImmediate start$190 per hour
...Plug In Connectors Expert Type: Contract Compensation: $50–$190/hour Location: Remote Commitment: 20+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows, including personal health , travel , and activity...Remote jobHourly payContract workFor contractorsSummer workTrial period$175 per hour
...Larry Summers , and Jack Dorsey . Position: CNC Machining Expert Type: Contract Compensation: $120–$175/hour Location: Remote Role Responsibilities Contribute to text-only, objective, verifiable rubrics for grading CNC machining solutions...Remote jobContract workPart timeSummer work$175 per hour
...Jack Dorsey . Position: Epidemiologist — Patient Population Sizing Type: Contract Compensation: $150–$175/hour Location: Remote Duration: 1–2 week pilot Commitment: 10–20 hours Role Responsibilities Create and critique rubrics that...Remote jobHourly payContract workPart timeFor contractorsSummer work$80 - $120 per hour
...$120 per hour We are hiring expert Evaluators in Special education / IEP to review and... ...expertise to grade outputs. This is a remote, hourly engagement. Requirements... ...an independent contractor. This is a fully remote role that can be completed on your...Remote workHourly payContract workFor contractorsWork at officeWeekday AI
United States3 days ago$120 per hour
About the role We are hiring expert Evaluators in Special education / IEP to review and assess AI-generated work products (documents... ...deep subject-matter expertise to grade outputs. This is a remote, hourly engagement. Requirements (must have) 5+ years of relevant...Remote workHourly payWork at officeObsidian
United States3 days ago$40 per hour
...Fashion Stylist | Upto $40/hr Remote Join to apply for the Fashion Stylist | Upto $40/hr Remote role at Crossing Hurdles This range is provided... ...(Training support will be provided) Design and evaluate advanced shopping‑assistant AI systems by creating realistic...Remote workHourly payContract workCrossing Hurdles
New York, NY2 days ago$20 - $80 per hour
...generation AI systems by supplying precise, real-world evaluation, annotation, and feedback. This remote contractor role focuses on how AI models learn,... ...to work both independently and collaboratively in a fully remote environment. Preferred Qualifications Experience...Remote workHourly payContract workFor contractorsSaidGig
United Statesmore than 2 months ago- ...Position Summary This is a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible... ..., and other content to support AI training datasets. LLM evaluation: reviewing AI-generated responses for accuracy, reasoning quality...Remote workHourly payFor contractorsFlexible hours
CNTXT AI
Brooklyn, NY12 days ago $110 per hour
...Engineers with deep, hands-on expertise in JAX and kernel-level programming (Pallas/Triton). This role involves AI model training and evaluation work, including writing and assessing MLOps tasks and solutions to generate high-quality training data for frontier AI systems....Remote workFull timeWeekday workObsidian
United States4 days ago$75k - $175k
...your recruiter to learn more. Base pay range $75,000.00/yr - $175,000.00/yr Direct message the job poster from Goodwin Recruiting... ...Offer: No limits on territory, industry focus or income potential Fully remote home-based business ownership Extensive support to help you hit...Remote workContract workLocal areaWork from homeGoodwin Recruiting
New York, NY2 days ago- ...HR Policy Agent Evaluator is a remote review track for evaluating AI outputs in policy review workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can train on it....Remote jobHourly payFor contractors10 hours per week
$9 - $30 per hour
ALTA Language Services, Inc. is looking for a remote Testing Evaluator in Atlanta, GA. This part-time position involves assisting with language proficiency examinations and evaluating tests according to set criteria. Candidates should have strong communication skills,...Remote workPart timeALTA Language Services
Atlanta, GA19 hours ago- ...leading digital solutions provider is seeking a Personalized Ads Evaluator for a part-time position. This entry-level role involves... ...and cultural communities. The position allows up to 20 hours of remote work weekly and requires passing a basic qualification exam. #J...Remote workPart time
TELUS Digital AI Data Solutions
United States2 days ago $175 per hour
...Detection, AppSec) Type: Contract Compensation: $125–$175/hour Location: Remote Role Responsibilities Participate in two to three... ...and processes used in enterprise cyber defense work. Evaluate how AI assistance currently impacts your work and suggest...Remote jobHourly payContract workSummer work$14.5 per hour
A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical...Remote workHourly payPart timeFlexible hoursWelo Data
United States2 days ago- ...Seeking a full-time Remote AI Research Evaluator with a PhD in Quantitative Finance to assess and enhance AI models' capabilities in financial reasoning and quantitative analysis through flexible, contract-based work. Key responsibilities Assessing the factuality and...Remote workFull timeContract workFlexible hours
Virtual Vocations Inc
United States1 day ago $175 per hour
...Detection, AppSec) Type: Contract Compensation: $125–$175/hour Location: Remote Role Responsibilities Participate in two to three... ...and processes used in enterprise cyber defense work. Evaluate AI assistance in current workflows and suggest...Remote jobHourly payContract workSummer work$20 per hour
...specializing in AI is hiring a Digital Web Designer. In this remote role, you will evaluate AI-generated designs and provide feedback to enhance the... ...skills. The position offers flexible hours and pays $20+ USD/hr for general projects and $40+ for design-focused tasks. #J-...Remote workFlexible hoursDataAnnotation
United States2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to JSON Evaluator - Fully Remote | Upto $175/hr. Be the first to apply!
Related searches
- ai evaluator San Francisco, CA
- quality evaluator San Francisco, CA
- social media evaluator San Francisco, CA
- education evaluator San Francisco, CA
- evaluator San Francisco, CA
- remote reviewer San Francisco, CA
- senior php developer remote San Francisco, CA
- remote sales consultant San Francisco, CA
- medicare sales agent remote San Francisco, CA
- web design internship remote San Francisco, CA