ML Challenge Task Auditor
$70 - $90 per hourTemporary
Mercor
Evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate a frontier AI lab's models. You'll assess experiment design, model-selection reasoning, and evaluation methodology, and provide clear, rubric-based written feedback.
Basic Qualifications
- 3+ years hands-on applied/experimental ML (experiment design, model selection, hyperparameter tuning, evaluation methodology)
- Strong grasp of data-quality rigor: leakage detection, metric gaming, and train/test/CV hygiene
- Proficiency with standard ML frameworks (PyTorch, TensorFlow, scikit-learn, XGBoost)
- Ability to critique ML claims against evidence and reproduce results
Preferred Qualifications
- Competition / benchmark experience (e.g., Kaggle)
- Graduate research or publication record in applied ML
- Prior task-grading or peer-review experience
Note: this role evaluates applied/experimental ML rigor, it is not an LLM-application-building or MLOps role.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the ML Challenge Task Auditor in San Francisco, CA vacancy
- ...correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate a frontier AI lab's models. You'll... .... Basic Qualifications • 3+ years hands-on applied/experimental ML (experiment design, model selection, hyperparameter tuning, evaluation...SuggestedTemporary work
Mercor
San Francisco, CA24 days ago - ...Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI lab's models. You'll assess repository-level tasks, reference patches, test harnesses, and grading integrity , and provide clear, rubric...SuggestedTemporary work
Mercor
San Francisco, CA24 days ago $70 - $90 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: AI Developer Trace Task Auditor Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality and...SuggestedContract workSummer workRemote workMercor
San Francisco, CA5 days ago$70 - $90 per hour
...Evaluate the quality, correctness, and cloud-architecture soundness of AWS serverless and infrastructure-as-code tasks used to train and evaluate a frontier AI lab's models. You'll assess multi-service serverless designs, IaC fidelity, and cross-service integration correctness...SuggestedContract workLocal area$70 - $90 per hour
...Evaluate the quality, correctness, and production-readiness of Kubernetes tasks used to train and evaluate a frontier AI lab's models. You'll assess cluster-operations scenarios, manifest correctness, and failure-mode troubleshooting , and provide clear, rubric-based written...SuggestedContract work$54k - $58k
...detailed findings; coordinate all analytical, technical and research tasks. Manipulate and organize data via Excel. Gather payroll... ...Program: Confidential support for personal or work-related challenges. FAMILY: Hybrid & Remote Flexibility: Work-life balance matters...Work at officeRemote workFlexible hoursDay shift$137.1k - $188.3k
...earn recognition for your work. We offer a collegial culture, challenging projects, and excellent compensation and benefits, not to mention... ..., scoping, testing, reporting, and coordination with external auditors. Partner with process and control owners to support SOX...Full timeWorldwideFlexible hoursDolby Laboratories, Inc.
San Francisco, CAmore than 2 months ago$80k - $150k
Senior Auditor Employment Type: Full Time, Mid-level Department: Financial Services CGS is seeking a Senior Auditor who will assist... ...client’s specific needs. We are committed to solving the most challenging and dynamic problems. For the past seven years, we’ve been growing...Full timeFlexible hours- ...clear path, not just a promise. We are searching for a Senior Auditor. As a Senior Auditor with our firm, you’ll be responsible for... ...with clients built around understanding their operations and challenges.You’ll support all aspects of the engagements that you manage,...Casual workWork at officeLocal areaRemote workWork from home
$70 - $90 per hour
..., Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: AWS Serverless & Infrastructure-as-Code Task Auditor Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality and...Contract workSummer workLocal areaRemote workMercor
San Francisco, CA5 days ago- ...Merchandising & Audits See all information pertaining to rate of pay and tasks to be completed on the CCMI website. Must have merchandising experience. This is not a daily job, nor will it lead to Full Time. These are part time assignments to earn extra income if your...Extra incomeFull timePart time
- ...Level Department: Financial CGS is seeking a highly skilled Auditor to provide general auditing and accounting services in support... ...client’s specific needs. We are committed to solving the most challenging and dynamic problems. For the past seven years, we’ve been growing...Full timeWork experience placementInterim roleWork at officeLocal areaFlexible hours
$70 - $90 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: SWE-Bench Task Auditor Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality,...Contract workSummer workRemote workMercor
San Francisco, CA5 days ago- ...field. 1-3 years experience preferred Junior Compliance Officer Tasks At a minimum, the Junior Compliance Officer shall perform the... ...run queries, prepare spreadsheets, and conduct research. Assist Auditors and Criminal Investigators with evaluation of Employment Eligibility...Contract workFor contractorsLocal areaRemote workWeekend work
- jobr.pro is seeking an IA Lead for Stablecoins and Digital Assets to design and lead a comprehensive internal audit strategy. This critical role requires extensive experience in auditing, risk management, and thorough knowledge of stablecoin mechanics. The candidate will...
Jobr
San Francisco, CA1 day ago - ...partner career path or growth opportunities. Are you up for the challenge? About the Team The Audit & Assurance team at Crowe... ...a professional fashion. ~ You will need to be able to multi-task since planning, executing, and wrapping up various engagements...Full timeSeasonal workLocal areaWorldwideHome office
Crowe Global
San Francisco, CA5 days ago $8,265 - $10,785 per month
Job Posting:Lead Utility Auditor-Evaluator Public Utilities Commission JC-533190 - JC-533190 - Lead Utility Auditor-Evaluator SENIOR MANAGEMENT AUDITOR $8,265.00 - $10,785.00 per Month New to State candidates will be hired into the minimum salary of the classification...Permanent employmentFull timeTemporary workWork at officeRemote work2 days per week$61.72k
...competitive service position will be at the discretion of the agency and is neither guaranteed nor implied. In this position as an Auditor, you will work as an advanced trainee performing work assignments and completing training, serving as a support auditor on a team,...Permanent employmentFor contractorsTraineeshipInternshipWork at office- ...Job Description Job Description Job Title: Medical Auditor Job Type: Contractor Location: Remote Job Overview We are... ...findings clearly. ~ Experience managing audit programs and navigating the unique challenges of academic healthcare environments....Remote jobFor contractors
- ...Inventory Auditor Our customer provides inventory services to the healthcare delivery industry. They audit inventories of medical, surgical, and pharmaceutical products as well as capital and patient care equipment. Our brand is our professionalism, accuracy, and punctuality...
- ...Merchandising & Audits See all information pertaining to rate of pay and tasks to be completed on the CCMI website. Must have merchandising experience. This is not a daily job, nor will it lead to Full Time. These are part time assignments to earn extra income if your...Extra incomeFull timePart time
- Job Description Job Description Company Description Medicine moves forward here. So will you. ADAMAS Consulting helps the world's pharmaceutical and biotechnology companies uphold the highest standards of quality, compliance and patient safety. Since 2007 we...FreelanceRemote work
Adamas Consulting
San Francisco, CA25 days ago - Direct message the job poster from Insight Global Job Overview Insight Global is seeking experienced Senior Associates and Managers to support a Big 4 firm during its DAT busy season. This role focuses on IT SOX compliance, control remediation, and project management for...Contract work3 days per week
- ...up to 6 hours of business law (provide transcripts); OR 2. A certificate as a Certified Public Accountant or a Certified Internal Auditor, obtained through written examination (provide certification); OR 3. Completion of the requirements for a degree that included...Full timeApprenticeshipWork at office
- The Judicial Council of California is accepting applications for the position of Auditor with the Audit Services office. The Auditor conducts compliance audits of superior courts and other judicial branch entities based on the Judicial Council’s policies and procedures....Work at officeRemote workFlexible hoursNight shift2 days per week
$70 - $90 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Kubernetes Task Auditor Type: Contract Compensation: $70–$90/hour Location: Remote Role Responsibilities Evaluate the quality and...Contract workSummer workRemote work- ...opportunities to complete your registration Merchandising & Audits available. See all information pertaining to rate of pay and tasks to be completed on the CCMI website (link below) Must have merchandising experience This is not a daily job, nor will it lead...Extra incomeFull timePart timeImmediate start
- ...outsourcing models. Coach team members to develop skills and deliver quality client service. Address conflicts and engage in challenging conversations with stakeholders to resolve issues. Uphold professional and technical standards, including independence...Hourly payLocal area
$50 - $55 per hour
...generated content in wholesale and technical sales to strengthen reasoning and rigor in model outputs . Review complex sales tasks for alignment with domain principles and methodologies. Provide clear, structured feedback to AI research teams to improve...Contract workSummer workRemote work- ...Program and Project Supervisor, the SMA (Specialist) is responsible for planning, organizing, and leading the work of junior level auditors performing various complex financial, management, and compliance audits, such as water and sewer utility audits, balancing account...Full timeWork at officeRemote work2 days per week
Public Utilities Commission
San Francisco, CA12 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to ML Challenge Task Auditor. Be the first to apply!
Related searches
- inventory auditor San Francisco, CA
- lease auditor San Francisco, CA
- safety auditor San Francisco, CA
- data auditor San Francisco, CA
- remote auditor San Francisco, CA
- auditor San Francisco, CA
- coding auditor San Francisco, CA
- sales auditor San Francisco, CA
- remote nurse auditor San Francisco, CA
- field auditor San Francisco, CA