SWE-Bench Task Auditor Mercor · Remote — United States $70-90/hr →
$70 - $90 per hourDorado
Brooklyn, NY
- Remote job
Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI lab's models. You'll assess repository-level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric-based written feedback. Basic Qualifications 3+ years professional software engineering Real open-source contribution or maintainer experience (merged PRs, committer / maintainer roles) Strong ability to audit reference patches, test runners, and Docker isolation, and to detect answer leakage / reward hacking Fluency across common ecosystems (Python and at least one of Java / Go / TypeScript / C++) Preferred Qualifications Familiarity with SWE-Bench (Verified) or similar repository benchmarks Maintainer history on major Python OSS (Django, Flask, scikit-learn, sympy, pytest, etc.) Prior code-review or task-grading experience #J-18808-Ljbffr Dorado
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the SWE-Bench Task Auditor Mercor · Remote — United States $70-90/hr → in Brooklyn, NY vacancy
$70 - $90 per hour
Evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate a frontier AI lab's models. You'll assess experiment design, model-selection reasoning, and evaluation methodology — and provide clear, rubric-based...Remote job$70 - $90 per hour
...engineering benchmark tasks are accurate, reproducible... ...Familiarity with SWE Bench Verified or similar repository... ...experience. Work Terms Remote role open to candidates located in the United States. Hourly engagement. Compensation ~$70 to $90 per hour....Remote workHourly paySaidGig
Remote5 days ago$70 - $90 per hour
...and evaluate benchmark tasks so AI systems can... ...cutting-edge AI systems. Remote projects can offer... ...OpenTrain is recruiting a SWE-Bench Task Auditor to evaluate... ...candidates located in the United States Approximately 40 hours... ...weekly Compensation of $70 to $90 per hour Part-time...Remote workHourly payContract workPart timeFor contractorsFlexible hours$70 - $90 per hour
...SWE-Bench Task Auditor is a remote review track for evaluating AI outputs across audit workflows. Reviewers grade calculations, narrative reasoning... ...must be authorized to work from US. Compensation $70–$90 / hr Application process Apply through AuraOne's...Remote jobFor contractorsWork experience placement10 hours per week$70 - $90 per hour
...Role Overview Assess Kubernetes tasks for quality, technical correctness, and production... ...SRE, platform engineering, or task grading. Work Terms Remote role open to candidates located in the United States. Hourly engagement. Compensation $70 to $90 per hour....Remote workHourly paySaidGig
Remote5 days ago$70 - $90 per hour
...Help strengthen the applied machine-learning tasks used to train and evaluate advanced AI models.... ...learning. Previous task-grading or peer-review experience. Work Terms Remote, United States. Hourly engagement. Compensation ~$70 to $90 per hour....Remote workHourly paySaidGig
Remote5 days ago$70 - $90 per hour
...serverless and infrastructure-as-code tasks that support the training and evaluation... ...reviewing code or grading technical tasks. Work Terms Remote role open to candidates located in the United States. Hourly engagement. Compensation $70 to $90 per hour....Remote workHourly payLocal areaSaidGig
Remote5 days ago$240k - $300k
...Manager, Expenses#### Seattle, Washington, United States**Why join us**Brex is the intelligent... ...eliminate manual expense and accounting tasks for customers so they can focus on what... ...have up to four weeks per year of fully remote work!### **Responsibilities*** Own the end...Remote workWork at officeWork from home$70k - $80k
Washington, District of Columbia, United States Senior Manager, HR Operations Role Overview At... ..., D.C. This position is remote for eligible U.S.-based... ...focused initiatives and tasks. Supervise the People Operations... ...range for this role is $70,000-$80,000 A comprehensive...Remote workWork at officeFlexible hours2 days per week1 day per week- ...onsiterole at our GP HQ in downtown Atlanta, GA. While GP offers some remote work flexibility, there is a need for hybrid/weekly in-office... ...and protected veteran status. Except where prohibited by state law, all offers of employment are conditioned upon successfully...Remote workWork at officeLocal areaWorldwideRelocationFlexible hours3 days per week
$70k - $90k
Legal Billing Auditor at CURE Auto Insurance (Citizens United Reciprocal Exchange)... ...Base Pay Range $70,000.00/yr - $90,000.00/yr Essential... ...entries, redundant tasks, excessive hours,... ...onsite 4 days/week, remote 1 day/week.... ...compliance with federal, state, and local laws....Remote workWork at officeLocal areaFlexible hours1 day per week$15 - $35 per hour
...Hindi-speaking transcriptionists across the United States to produce accurate, high-quality... ...systems to truly listen. This is a fully remote, flexible contract role perfect for members... ...with the structure of meaningful, task-based work Contribute to AI development...Remote workHourly payOngoing contractContract workFreelanceWorldwideFlexible hours$90 per hour
...Child, Family and School Social Workers | $90/hr Remote | Mercor Organization: Mercor (referral partner of Crossing Hurdles) Position: Child, Family, and School Social Workers Type: Hourly contract Compensation: $50–$90 per hour (based on skills and experience...Remote workHourly payFull timeContract workWork experience placement10 hours per weekFlexible hoursCrossing Hurdles
United States2 days ago$180 per hour
...Health Services Managers | $180/hr Remote | Mercor Join to apply for the Health Services Managers... ..., policy, and administrative tasks in the healthcare industry. Collaborate... ...Get notified about new Health Services Manager jobs in United States. #J-18808-LjbffrRemote workHourly payFull timeContract workWork experience placementFlexible hoursCrossing Hurdles
United States2 days ago$70k - $90k
...Senior Premium Auditor The Senior Premium Auditor... ...leverages technology and remote collaboration. This position... ...agencies, independent state bureaus, and company... ...to effectively multi-task and work in a fast-paced... ...range for this role is $70,000 - 90,000. Please note...Remote workWork at officeFlexible hoursAmTrust Financial
Cleveland, OH2 days ago$29.75 - $32.7 per hour
...between $29.75 and $32.70/hr based on experienceSpecialized... ....Ability to multi-task and prioritize needs to... ...Top Workplace for Remote Work 2024Great Place to... ...by applicable federal, state, or local laws. Ensemble... ...authorized to work within the United States. Applications...Remote workTemporary workLocal areaEnsemble Health Partners
Reading, PA4 days ago$60 - $70 per hour
...Teaching Assistants | $60-$70/hr role at HelixRecruit This... ...part of a referral program Mercor is recruiting Teaching Assistants... .... This is a fully remote role that can be completed on... ...about new Teaching Assistant jobs in United States . #J-18808-LjbffrRemote workHourly payFull timeContract workPart timeFor contractorsWork experience placementHelixRecruit
New York, NY3 days ago- ...schedule and complete exciting tasks using an innovative web-based evaluation... ...Being a resident in the United States for the last Year and having... ...Salesperson” roles. Lincoln, NE $70,000.00-$80,000.00 1 week ago Sales Representative - Remote Role with Training Provided and...Remote workPart timeImmediate startWork from homeFlexible hours
$50k - $70k
...Factory Inspector who can support department objectives while traveling 90%-100% of the time. Candidates should have a degree and knowledge... ...full-time, hybrid position includes a salary range of $50,000 - $70,000 and excellent benefits, including medical, dental, life...Remote jobFull time- Work-from-Home Administrative Assistant (U.S. Only) Location: Remote (United States) Job Type: Full-Time or Part-Time Start Date: Immediate... ...someone who is organized, self-motivated, and able to manage tasks independently. What We Offer 100% remote position (U.S.-based...Remote workFull timePart timeImmediate startWork from homeFlexible hours
$100 per hour
Personal Financial Advisor | $200/hr Remote | Mercor role at Crossing Hurdles This range is provided... ...reflect the responsibilities and tasks of a Personal Financial Advisor.... ...about new Personal Financial Advisor jobs in United States. #J-18808-Ljbffr Crossing HurdlesRemote jobHourly payFull timeContract workFlexible hours$65k - $70k
...DAILY MEMBERSHIP (ADM) AUDITOR Job Location:... ...Salary: $65,000 - $70,000 Grade: 23... ...perform administrative tasks as needed on each... ...to meet the Audit Unit's established quality... ...process In-state travel is required... ...up to two days of remote work (contingent upon...Remote workDaily paidFull timeTemporary workWork at officeState of Arizona
Phoenix, AZ7 days ago$30 per hour
...Base pay range $30.00/hr - $30.00/hr Direct message... ...Hiring: Field Asset Auditor (Remote + Global Travel... ...characteristic under federal, state, or local law. Our... ...Asset Manager jobs in United States . Washington, United... ...ago United States $70,000.00-$90,000.00 29 minutes ago...Remote workDaily paidContract workLocal areaImmediate startWorldwide$70 per hour
...ExpertType:ContractCompensation:$60-$70/hourLocation:Remote Role Responsibilities Construct higher... ...enrollment planning. Build education tasks in academic program development,... ...opportunity. Originally posted on Himalayas #J-18808-Ljbffr United States Digital Space LLCRemote jobSummer work- ...Processing, Inspection and Testing Our auditors enjoy traveling domestically and... ...“Auditor” roles. New York, NY $70,000.00-$90,000.00 2 weeks ago Massachusetts, United States $70,000.00-$90,000.00 2 weeks... ...ago Senior Accountant, Revenue (Remote Role) United States $108,630.00-...Remote workDaily paidPart timeFlexible hours
- ...Organization (PEO) serving 70,000 employees. TPG's... ...of PEO services in the United States serving the blue and... ...' Compensation Premium Auditor to join our team. The Workers... ...detail-oriented tasks; interact, coordinate,... ...ability to function as a remote employee in a distributed...Remote workFull timeContract workWork at officeNight shift
The PEOPLEASE Group
Remote7 days ago - ...intermediaries across major financial markets worldwide. In the United States, Ascent combines local regulatory expertise with global... ...critical onboarding and AML/KYC function supporting U.S. clients. Remote working with collaboration across the U.S., Asia, and EMEA. Competitive...Remote workWork at officeLocal areaWorldwide
- ...Number: 193278Apply: JobWe are seeking an HR Business Partner to work in partnership... ...role at the Albion, MI, that will also have remote and travel responsibilities to support... ...the largest privately held company in the United States. We are always interested in talented...Remote workHourly payWork at officeWorldwideFlexible hours
$70k - $90k
...effectively and work efficiently with remote team members. Ability to work within team... ...salary range for this position is $70,000 - $90,000 annually, plus comprehensive benefits... ...Mountain time. Anyone/anywhere in the United States is encouraged to apply. You must be eligible...Remote job$55 - $70 per hour
...Postdoctoral Research Scientist (Part-Time | $55-70/hr) Mercor is collaborating with a leading global... ...up to 40 hours/week) Schedule: Fully remote and asynchronous — work at your... ...Publishing Location: North Carolina, United States (Posted 2 weeks ago) #J-18808-Ljbffr Call...Remote workHourly payContract workPart timeInternshipImmediate startFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to SWE-Bench Task Auditor Mercor · Remote — United States $70-90/hr →. Be the first to apply!
Related searches