JSON Specialist - Rubric Expert
$50 - $175 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: JSON-fluent & Rubrics Expert
Type: Contract
Compensation: $50–$175/hour
Location: Remote
Commitment: 20–40 hours/week
Role Responsibilities
- Evaluate JSON-formatted AI model outputs against defined grading rubrics.
- Read and understand written task descriptions and rubrics.
- Parse, read, and assess JSON-based outputs, grading them against provided rubrics.
- Ensure attention to detail and consistency in applying rubric criteria.
- Work independently to meet tight deadlines within 12-24 hours .
Qualifications
Must-Have
- Available to start today, July 24 , running to July 27-28 .
- Comfortable reading and interpreting JSON files .
- Based in the US .
- Access to a desktop or laptop computer (no Chromebooks).
Preferred
- Prior Mercor experience, especially in applying detailed rubrics and evaluation criteria consistently.
Start Date
- Today, July 24
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 21 days ago
Similar jobs that could be interesting for youBased on the JSON Specialist - Rubric Expert in San Francisco, CA vacancy
$126k
...an experienced business professional to be a member of the Electric Incident Investigations (EII) Team as an Incident Investigator, Expert. The EII team is responsible for reporting Electric Incidents (EIRs) to the California Public Utilities Commission (CPUC),...SuggestedWork experience placementWork at officeRemote workFlexible hoursPG&E Corporation
Daly City, CA2 days ago$70 - $100 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Computational Astrophysics & Cosmology Expert Type: Contract Compensation: $70–$100/hour Location: Remote Commitment: 15–20 hours/week Role...SuggestedContract workSummer workRemote workMercor
San Francisco, CA2 days ago$84 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Atomic Layer Deposition (ALD) Experts Type: Contract Compensation: $84/hour Location: Remote Commitment: 10–40 hours/week Role Responsibilities...SuggestedContract workSummer workRemote workMercor
San Francisco, CA2 days ago$126k
...an experienced business professional to be a member of the Electric Incident Investigations (EII) Team as an Incident Investigator, Expert. The EII team is responsible for reporting Electric Incidents (EIRs) to the California Public Utilities Commission (CPUC),...SuggestedWork experience placementWork at officeLocal areaRemote workFlexible hoursPacific Gas And Electric Company
Oakland, CA1 day ago$70 - $100 per hour
...Larry Summers , and Jack Dorsey . Position: Procurement Expert Type: Contract Compensation: $70–$100/hour Location... ...-reasoned final approval recommendation. Author step-level rubrics and golden responses that capture how an experienced...SuggestedContract workSummer workRemote workMercor
San Francisco, CA21 days ago$60 - $80 per hour
...Jack Dorsey . Position: Insurance Specialist Type: Contract Compensation:... ...Evaluate AI model outputs against structured rubrics and provide clear, written feedback on... ...Collaborate with other subject matter experts to ensure consistency and accuracy in training...Contract workSummer workRemote workWeekday workMercor
San Francisco, CA2 days ago$60 - $90 per hour
...Summers , and Jack Dorsey . Position: Enterprise Sales Domain Expert Type: Contract Compensation: $60–$90/hour... ...Create expert reference examples for model training. Refine rubrics used to score AI outputs. Ensure they capture distinctions between...Contract workSummer workWork at officeRemote workMercor
San Francisco, CA2 days ago$80 per hour
...Larry Summers , and Jack Dorsey . Position: Accounting Expert Type: Contract Compensation: $70–$80/hour Location... ...codification , PCAOB standards , IFRS , and ISA . Author rubrics that distinguish authentic accounting judgment from generic...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$50 per hour
...Larry Summers , and Jack Dorsey . Position: K-12 Education Expert Type: Contract Compensation: $40–$50/hour... ...documentation, and administrator-level communications. Author rubrics that distinguish authentic K-12 educator judgment from generic...Contract workSummer workWork at officeRemote workMercor
San Francisco, CA2 days ago- ...verification logic is rigorous, and Docker-based lab environments accurately recreate exploitable conditions , and provide clear, rubric-based written feedback. Basic Qualifications • 3+ years of hands-on experience in application security, penetration testing, or vulnerability...Temporary work
Mercor
San Francisco, CA21 days ago $50 - $60 per hour
..., and Jack Dorsey . Position: Government & Public Policy Expert Type: Contract Compensation: $50–$60/hour Location... ...reasoning. Provide structured feedback to refine evaluation rubrics and raise quality standards. Qualifications Must-Have ~...Contract workSummer workLocal areaRemote workMercor
San Francisco, CA2 days ago- ...fidelity, Trainium-specific performance-optimization quality, and cross-platform numerical-correctness standards , and provide clear, rubric-based written feedback. Basic Qualifications • 2+ years of hands-on experience developing or optimizing kernels using the Neuron...Temporary work
Mercor
San Francisco, CA21 days ago $80 - $120 per hour
...$80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks. Provide clear, structured...Contract workSummer workWork at officeRemote workMercor
San Francisco, CA2 days ago$120 - $175 per hour
...Larry Summers , and Jack Dorsey . Position: CNC Machining Expert Type: Contract Compensation: $120–$175/hour... ...Responsibilities Contribute to text-only, objective, verifiable rubrics for grading CNC machining solutions across provided CAD models...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$60 per hour
...D'Angelo , Larry Summers , and Jack Dorsey . Position: Expert Interviewer Type: Contract Compensation: $50–$60/hour... ...technical depth, communication, and AI-evaluation fit using a provided rubric. Confirm each expert's availability, work authorization, and...Contract workPart timeSummer workRemote workWeekday workMercor
San Francisco, CA2 days ago$45 - $65 per hour
...domains. Design clear, non-contradictory rating guidelines and rubrics for consistent application, including under edge cases.... ...specific rater instructions. Collaborate with subject matter experts and program leads to ensure consistency and accuracy across guideline...Contract workSummer workRemote workWeekday workMercor
San Francisco, CA2 days ago- ...confidence. Role Summary Campfire is hiring an Integrations Specialist to sit on the Implementation team and own the customer-facing... ...bank feeds ~ Comfortable with SQL and structured data (CSV/JSON/XML) for diagnosing issues ~ Strong troubleshooting instincts...
Campfire
San Francisco, CA2 days ago $70 - $90 per hour
...Summers , and Jack Dorsey . Position: Trainium (NKI) Kernel Expert Type: Contract Compensation: $70–$90/hour Location... ...-platform numerical-correctness standards and provide clear, rubric-based feedback. Collaborate with AI research teams to ensure...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$45 - $55 per hour
...training data quality and downstream performance. Develop evaluation rubrics and benchmarks for assessing AI-generated tasks and solutions. Collaborate with subject matter experts to ensure consistency, relevance, and coverage across datasets. Work independently...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$90 - $100 per hour
...Larry Summers , and Jack Dorsey . Position: Regulatory Law Expert Type: Contract Compensation: $90–$100/hour... ...regulatory filings, and enforcement response documents. Author rubrics that distinguish authentic regulatory judgment from generic law...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$70 - $90 per hour
...tasks. Assess multi-service serverless designs , IaC fidelity , and cross-service integration correctness. Provide clear, rubric-based written feedback to improve AI model training . Review event-driven architecture for correctness, including IAM/...Contract workSummer workLocal areaRemote workMercor
San Francisco, CA2 days ago$60k - $74k
...sleep well so they live well is at the core of what we do. The right mattress matters, and so does the right career. Our Sleep Experts® are at the heart of what we do: helping customers find their dream mattress and get better sleep. By joining our team, you'll get...Local areaMattress Firm
Daly City, CA4 days ago$150 - $175 per hour
...Location: Remote Duration: 1–2 week pilot Commitment: 10–20 hours Role Responsibilities Create and critique rubrics that analyze commercial drugs and drug-development programs. Size the patient population for a given drug or indication, focusing...Hourly payContract workFor contractorsSummer workRemote workMercor
San Francisco, CA2 days ago$60k - $72k
...sleep well so they live well is at the core of what we do. The right mattress matters, and so does the right career. Our Sleep Experts® are at the heart of what we do: helping customers find their dream mattress and get better sleep. By joining our team, you’ll get...Local areaMattress Firm
San Francisco, CA1 day ago- ...Description Job Description Manufacturing Specialist Job Type: Contractor Location:... .... Key Responsibilities Provide expert feedback on CAM, CNC, and Fusion 360 manufacturing... .... Develop and validate evaluation rubrics for manufacturing workflows. Provide...For contractorsRemote work
YO AI Labs
San Francisco, CAa month ago $60 per hour
...Reviewer — Document QC Type: Contract Compensation: $60/hour Location: Remote Role Responsibilities Create rubrics to grade legal documents that are understandable by laymen. Review processed legal documents against a written QC rubric to ensure...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$50 per hour
...AI projects with leading LLM companies. Pay rate: $50+/hour (depends on role and candidate expertise) Assessment: Shortlisted experts complete an evaluation before selection. Assignments: Contract roles with defined start/end dates; up to 40 hrs/week. About...Contract workRemote workFlexible hoursTuring
San Francisco, CA1 day ago- ...PHP Python React Rust TypeScript ASP.NET NodeJS .NET Framework Golang HTML5 More: Job Title: Coding Expert Job Type: Contract (Part-time) Location: Remote (US, Canada, United Kingdom, Ireland, Australia, New Zealand) Job Summary:...Contract workPart timeRemote work
SwissDevJobs.ch
San Francisco, CA2 days ago - ...Job Title: Subject Matter Expert (SME) About the Role: At Matter, we specialize in advancing sensors and geospatial AI to capture extreme-resolution images of natural and artificial materials from space to surface. Our cutting-edge sensors simultaneously measure...
Matter Intelligence
San Francisco, CA2 days ago - ...end users of Maps or test the location results of user queries submitted via Web Search or Maps interface. The Mapping City/Country Expert will be responsible for high‑impact work in improving city transit data and providing country‑specific expertise. The City/Country...Local area
TSMG Holding
San Francisco, CA3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to JSON Specialist - Rubric Expert. Be the first to apply!
Related searches
- letter of credit specialist San Francisco, CA
- employment placement specialist San Francisco, CA
- coding specialist San Francisco, CA
- consulting specialist San Francisco, CA
- policy specialist San Francisco, CA
- intervention specialist San Francisco, CA
- outreach specialist San Francisco, CA
- instructional technology specialist San Francisco, CA
- air specialist San Francisco, CA
- placement specialist San Francisco, CA