Practicing Physician for AI Model Evaluation

Role Overview

Shape how advanced AI systems reason about real clinical work. In this senior clinical medicine role, you will partner with an AI research team to evaluate medical knowledge tasks, define high-quality clinical standards, and develop benchmarks that measure meaningful model improvement.

Key Responsibilities

• Review clinical knowledge tasks and model outputs for missing behaviors, weak reasoning, unsafe recommendations, guideline misalignment, and answers that would not withstand clinical scrutiny.
• Write detailed instruction specifications and gold-standard solutions for clinical problems.
• Create new clinical tasks that reflect real-world medical practice.
• Design challenging clinical evaluations and benchmarks, and help develop medicine-specific skills and tools with the research team.
• Collaborate with researchers and adjacent-domain specialists to calibrate standards and translate clinical judgment into clear, teachable criteria.

Qualifications

• MD or DO from an accredited medical school, with completed residency training in a recognized specialty.
• At least 4 years of post-residency clinical practice. Residency and fellowship training do not count toward this requirement.
• Active, unrestricted license to practice medicine in at least one U.S. state and board certification in your specialty.
• Established expertise in a clinical specialty, such as internal medicine, oncology, radiology, emergency medicine, surgery, psychiatry, or a medical subspecialty.
• Senior-level clinical leadership or decision ownership, such as Attending Physician, Medical Director, Division Chief, Associate or Full Professor, or Chief Medical Officer.
• Professional, hands-on experience using large language models and the judgment to distinguish sound clinical reasoning from plausible but incorrect answers.
• Excellent written communication and the ability to provide precise, well-structured feedback.
• Experience in utilization management, clinical informatics, or medical affairs is a plus.

Work Terms

• Full-time W-2 employment, with a reliable commitment of 40 hours per week for an initial 6-month engagement.
• Hybrid role based in the Bay Area, California. You must live in the Bay Area and be available to work on-site with the client team multiple days per week when required.
• This is not a remote role. Candidates relocating to the Bay Area must do so at their own expense; relocation assistance is not available.
• You will work within the client’s tools alongside its research team and receive client-issued accounts and equipment.

Compensation

$70 to $110 per hour.

Eligibility

Qualified applicants are considered without discrimination based on legally protected characteristics. Reasonable accommodations are available for qualified individuals with disabilities and disabled veterans during the application process.

Application Process

Applicants may discover this opportunity through an online talent platform. Employment, onboarding, payroll, benefits, and compliance are managed by the employer of record, while the successful candidate is placed directly within the client team.