JSON Specialist - Rubric Expert
$50 - $175 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: JSON-fluent & Rubrics Expert
Type: Contract
Compensation: $50–$175/hour
Location: Remote
Commitment: 20–40 hours/week
Role Responsibilities
- Evaluate JSON-formatted AI model outputs against defined grading rubrics.
- Read and understand written task descriptions and rubrics.
- Parse, read, and assess JSON-based outputs, grading them against provided rubrics.
- Ensure attention to detail and consistency in applying rubric criteria.
- Work independently to meet tight deadlines within 12-24 hours .
Qualifications
Must-Have
- Available to start today, July 24 , running to July 27-28 .
- Comfortable reading and interpreting JSON files .
- Based in the US .
- Access to a desktop or laptop computer (no Chromebooks).
Preferred
- Prior Mercor experience, especially in applying detailed rubrics and evaluation criteria consistently.
Start Date
- Today, July 24
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the JSON Specialist - Rubric Expert in San Francisco, CA vacancy
$261k - $290k
...calls for a single, dedicated owner to realize its full potential.We are creating a dedicated Director role as a GitHub Subject Matter Expert (SME) — the accountable owner who turns that momentum into results. GitHub is the tip of the spear for our Zero Legacy and Azure...SuggestedTemporary workLocal area- A dynamic technology firm is seeking an experienced professional to analyze and improve electronic communications accessibility. Ideal candidates will have over 5 years of relevant experience, along with strong knowledge of the Web Content Accessibility Guidelines (WCAG...SuggestedContract work
- A leading aesthetic treatment provider in San Francisco is seeking an Aesthetic Body Specialist to perform tailored body treatments and consultations for clients. The role requires strong customer service skills, a High School Diploma or GED, and preferably an Aesthetician...SuggestedWeekend workAfternoon shift
- A leading IT consulting firm in California seeks a NetSuite Developer with significant experience in scripting and platform customization. The ideal candidate will have at least 6 years in IT, including 5 years focused on NetSuite ERP/CRM, demonstrating both technical expertise...Suggested
- ADP WFN RECRUITMENT TEST COMPANY is seeking a Calibration Specialist to ensure the accuracy and reliability of measurement instruments. Your role will involve developing and maintaining calibration procedures that meet industry standards. The ideal candidate will have...Suggested
- Looking for people with experience using Shopify’s agent. We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request. Contract and Payment Terms You will be engaged as an independent contractor...Contract workFor contractorsRemote work
$35 - $38 per hour
A Healthcare organization in San Francisco is looking for an Accounts Payable Specialist with Oracle experience. This hybrid role offers a great opportunity for a temp-to-hire position with a pay range of $35.00 to $38.00 per hour. The ideal candidate should have over...Hourly payTemporary workRemote work- ...to build a high-quality evaluation dataset for CNC manufacturing. We seek experienced CNC machinists to author and validate grading rubrics for CNC machining work, bringing real production-floor judgment to determine whether a machining approach would be approved or...
$50 - $70 per hour
...include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Shopify Expert Type: Contract Compensation: $50–$70/hour Location: Remote Role Responsibilities Evaluate and optimize...Contract workSummer workRemote work- DSV Air & Sea Inc. in South San Francisco, USA, is seeking a Customs Specialist - Entry Writer to handle CBP transactions, research, client communication and recordkeeping. The role focuses on entry processing, classification checks and ensuring timely clearance while...
$150 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Psychiatry Expert Type: Contract Compensation: $150/hour Location: Remote Role Responsibilities Evaluate how large language models...Contract workSummer workRemote work10 hours per week$70 - $100 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Particle & Nuclear Physics Expert Type: Contract Compensation: $70–$100/hour Location: Remote Commitment: 15–20 hours/week Role...Contract workSummer workRemote work- ...Role Overview Turing invites Legal Experts to join projects that fine-tune AI models like ChatGPT. If you enjoy applying legal reasoning... ...across multiple areas of law. Apply a structured legal rubric to assess issue spotting, accuracy, reasoning, and application...FreelanceRemote workFlexible hours
Turing
Daly City, CA22 hours ago - ...Rules, international arbitration rules such as ICC and LCIA). Experts qualified in either or both tracks are encouraged to apply. Contributors... ...litigation scenarios, draft reference outputs, and write rubrics that capture how senior litigators think. Key Responsibilities...
$120 per hour
About the role We are hiring expert Evaluators in Customer success / support operations to review and assess AI-generated work products... ...AI-generated artifacts against domain‑specific quality rubrics. Identify factual, aesthetic, and presentation errors. Provide...Remote jobHourly payWork at office- ...private wealth institutions. Contributors design enterprise wealth and asset management scenarios, draft reference outputs, and write rubrics that capture how senior F500 investment professionals think. Key Responsibilities Construct enterprise wealth management scenarios...
- ...state, and major municipal government operations. Contributors design public‑sector scenarios, draft reference outputs, and write rubrics that capture how senior government and civic leaders think. Key Responsibilities Construct public‑sector scenarios spanning large‑scale...
$120 per hour
...$80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks. Provide clear, structured...Remote jobContract workSummer workWork at officeMercor
San Francisco, CA5 days ago$100 per hour
...&L. Role Overview: Turing is looking for Portfolio Management experts to work with our researchers to improve the performance of AI models... ...-adjusted return analysis, and portfolio optimization. Create rubrics to assess model capabilities on tasks like performance...Remote jobHourly payFreelanceFlexible hours- ...quality evaluation dataset for CNC manufacturing and are looking for experienced CNC machinists to help author and validate grading rubrics for CNC machining work. You'll bring real production-floor judgment to determine whether a machining approach would be approved or...
- ...International track (EU regulatory frameworks and UK regulatory bodies). Experts qualified in either or both tracks are encouraged to apply.... ...regulatory law scenarios, draft reference outputs, and write rubrics that capture how senior regulatory counsel think. Key...Fixed term contract
$78.3k - $201.7k
Pownall Lab - Associate/Full Specialist Biochemistry and Biophysics UC San FranciscoThe Pownall Lab at UCSF is seeking a full or associate specialist interested in understanding the molecular mechanisms controlling chromatin structure and function in developing zebrafish...$75.6k - $194.8k
Associate or Full Specialist - Liu labThe Specialist will execute a research program at the intersection of RNA-based nanomedicine, CRISPR epigenome editing, and neuro-oncology, embedded within a cross-institutional collaboration between UCSF Brain Tumor Center and UC...Local area$50 per hour
...projects with leading LLM companies. Pay rate: 50+/hour (depends on role and candidate expertise). Assessment: Shortlisted experts complete an evaluation before selection. Assignments: Contract roles with defined start/end dates; up to 40 hrs/week. About...Contract workRemote workFlexible hoursTuring
San Francisco, CA4 days ago- ...International track (UK Employment Rights Act, EU labor directives). Experts qualified in either or both tracks are encouraged to apply.... ...design HR scenarios, draft reference outputs, and write rubrics that capture how senior HR leaders think. Key Responsibilities...
$78.3k - $201.7k
Associate or Full Specialist - Diaz labSpecialist Series (Associate or Full Specialist), full-time, one-year appointment, Department of Neurological Surgery/Diaz Lab. The incumbent will support computational and translational brain-tumor research by developing, documenting...Full time$75.6k - $194.8k
Feinberg Lab - Associate/Full Specialist PositionThe Feinberg lab is seeking an Associate/Full Specialist to join our group and contribute to new and ongoing projects. Our lab focuses on sensorimotor integration in mice. The candidate will be expected to assist graduate...$224.3k
General Information Job Title Expert Manager, Product Management & Innovation Job ID 107751 Work Areas Product Management & Innovation Employment Type Permanent Full-Time Location(s) New York, San Francisco Description & Requirements...Permanent employmentFull timeWork at officeLocal areaHome office3 days per week- ...coaching, connection, and community. Join us and be part of a team that's changing lives every day! JOB SUMMARY The Sales and Service Expert (SSE) delivers an outstanding and well-rounded sales and service experience to all guests and members through a compelling approach...Work experience placementWork at officeLocal area
RadNet
San Francisco, CA3 days ago - Mercor is partnering with a biotech and pharma research team on expert human-evaluation work — creating and critiquing rubrics that analyze commercial drugs and drug-development programs. We're seeking a payer / market access expert for a paid pilot, with the potential...Remote jobPart time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to JSON Specialist - Rubric Expert. Be the first to apply!
Related searches
- senior safety specialist San Francisco, CA
- health specialist San Francisco, CA
- outreach specialist San Francisco, CA
- invoice specialist San Francisco, CA
- information technology specialist San Francisco, CA
- rehabilitation specialist San Francisco, CA
- asset protection specialist San Francisco, CA
- change management specialist San Francisco, CA
- housing program specialist San Francisco, CA
- equal employment specialist San Francisco, CA