Internal Medicine AI Evaluation Specialist
Part-time
Weekday AI
Role Description
Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.
In this role, you will:
- Develop realistic clinical scenarios.
- Evaluate AI-generated responses against evidence-based standards.
- Help improve how AI systems reason about diagnosis, treatment planning, and patient management within Internal Medicine.
We welcome Internal Medicine attending physicians, hospitalists, fellows, final-year residents, and other qualified Internal Medicine professionals who are passionate about applying their expertise to the future of healthcare AI.
Qualifications
- Attending Physicians: Board-certified in Internal Medicine; Current, active, and unrestricted medical license.
- Resident Physicians: Final-year Internal Medicine resident; Recent residency graduates must be board-eligible.
- Fellows: Board-certified or board-eligible in Internal Medicine or a related subspecialty; Current, active, and unrestricted medical license.
- Professionals with expertise in Internal Medicine subspecialties are encouraged to apply, including:
- Cardiology
- Hematology & Oncology
- Endocrinology
- Nephrology
- Gastroenterology
- Pulmonology
- Other Internal Medicine subspecialties
Requirements
- Strong knowledge of evidence-based medicine and clinical practice guidelines.
- Experience in clinical teaching, medical research, or quality improvement initiatives.
- Excellent written communication and analytical skills.
- Interest in artificial intelligence, clinical decision support, or healthcare technology.
Benefits
- Fully remote with 100% asynchronous work.
- Standard commitment of approximately 20 hours per week, with opportunities for additional hours based on project demand.
- Flexible working schedule.
- Weekly payments.
- Rolling onboarding with ongoing engagement opportunities.
- Compensation is determined by clinical experience, expertise, and seniority.
Equal Opportunity Statement
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.
Contract Information
- Independent contractor engagement.
- Fully remote work completed on your own schedule.
- Projects may be extended, modified, or concluded based on business needs and individual performance.
- Work does not involve access to confidential or proprietary information from any employer, client, or institution.
- Weekly payments are processed through supported payment platforms based on approved work completed.
- Please note that visa sponsorship is not available for this opportunity.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Internal Medicine AI Evaluation Specialist in Remote vacancy
- A leading AI research accelerator in San Francisco is seeking... ...a Remote Resident Medical Specialist (MD/DO). The successful... ...clinical scenarios for AI evaluation, ensuring diagnostic... ...and board certification in Internal or Emergency Medicine. This role requires excellent...SuggestedRemote job
- A leading research accelerator for AI labs is seeking a medical professional specializing in internal or emergency medicine. In this role, you will evaluate AI-generated medical reasoning to ensure accuracy and effectiveness. The position offers full remote engagement,...SuggestedRemote job
$130 per hour
Obsidian is partnering with a leading AI lab seeking Internal Medicine experts to design clinical scenarios and evaluate AI outputs. This role offers flexible remote engagement at 20 hours per week, starting post-onboarding, with rates between $130 and $300 per hour depending...SuggestedRemote jobHourly payFlexible hours$300 per month
...Scientific builds and deploys AI scientist agents to... ...science and the development of new medicines. We are an ambitious team run... ...product gaps and work with our internal Applied AI and engineering teams... ...support pre-sales technical evaluation. Requirements ~2+ years...SuggestedFull timeWork at officeRemote workEdison Scientific Inc.
San Francisco, CA1 day ago- ...vision at Postman. The Opportunity We're seeking an AI Engineer Intern to work alongside our AI team on large-scale AI and Agentic... ...TensorStore/Grain) and similar tools. Run large-scale experiments, evaluate models against defined metrics, and support ablation studies...SuggestedFull timeInternshipWork at officeFlexible hours3 days per week
Postman
Remote1 day ago $300.81k
...shape the future of precision medicine at scale, this is where you... ...class Commercial Data & Applied AI team. We are currently... ...storage) that integrate with internal systems and data sources Partner... ...for prompt engineering, evaluation, and responsible AI use as the...Full timeWorldwide2 days per week3 days per weekBilliontoone
Remote1 day ago- **Job Title: AI Trainer || Image Quality Evaluator || English** **Location**: Remote | Work from Home **Employment... ...data Annotation and Transcription Specialists with strong proficiency in English.... ...requirements. * Collaborate with internal QA teams and project managers to...Contract workRemote workWork from homeMonday to FridayDay shift
iMerit Technology
Louisianaa month ago $35 - $42 per hour
...project availability Help Shape the Future of AI in Finance, Audit & Risk Volga Partners... ...to join our growing network ofon-call AI Evaluation Specialistssupporting the development,... ...Financial Investigations Internal Controls Audit Procedures Regulatory Compliance...Hourly payExtra incomePermanent employmentContract workTemporary workRemote workFlexible hours$35 - $50 per hour
...FieldAI ’s Irvine team is where embodied AI meets real robots, real sensors, and real... ...field. As an Agentic AI/ML Engineer Intern, you will build agentic solutions that turn... ...hands-on role where you will prototype, evaluate, and ship agent-native solutions that...Full timeInternshipLocal areaField AI
Remote1 day ago$8 - $65 per hour
Medicine Specialist (Fluent in German) - Freelance AI Trainer Project World Wide - Remote Are you a fluent German‑speaking... ...who live and breathe internal medicine, pharmacology, pathology... ...improvements to our prompt engineering and evaluation metrics. A medical degree (MD,...Hourly payContract workFor contractorsFreelanceImmediate startRemote work- ...AI Engineer Job Description PulsePoint is a fast-growing AdTech company using real... ...components to improve the efficiency of internal teams and user-facing communications, spanning... ...to chat interfaces. Experiment and evaluate new frameworks, models, and architectures...Remote workFlexible hours
PulsePoint
United States5 days ago $109.2k - $174.6k
...pioneering team that's revolutionizing medicine through mRNA technology, with... ...seeking a Power Platform & AI Engineer to support the... ...treating the Power Platform as an internal product and driving both... ...alerting, and exception handling. Evaluate when to extend solutions with...Permanent employmentFull timeWork at officeWork from homeWorldwide$99.3k - $198.7k
...nutritionals and branded generic medicines. Our 115,000 colleagues serve... ...Career development with an international company where you can grow... ...OPPORTUNITY This Senior AI Engineer position can work out... ...integrations, inference services, and evaluation workflows to drive advanced...Remote workAbbott
North Chicago, IL19 hours ago$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Audit & Controls Specialist (External / Internal SOX) Type: Contract Compensation... ...to enhance AI model training and evaluation. Develop SOX controls documentation...Contract workSummer workRemote workMercor
Dallas, TX9 days ago- ...To support AI development, the part-time AI Evaluation Specialist will review and assess AI-generated outputs for quality and usability while collaborating with teams to refine evaluation standards in a remote contract role. Key responsibilities Review and critically...Contract workPart timeRemote work
Virtual Vocations Inc
United States4 days ago $84.58k - $126k
...NYU Grossman School of Medicine is one of the nation's top-ranked... ...team as a Engineer II - Applied AI. As a ML Engineer, you... ...tasks ~ Implement our internal security and data protection... ...analysis, prototyping, development, evaluation, testing, deployment,...Work at officeNYULMC
New York, NY5 days ago$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...and Jack Dorsey . Position: Public health communications Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work$70 per hour
...clients Compensation: $70 per hour Join a cutting-edge AI research initiative focused on improving the quality, accuracy,... ...with exceptional critical thinking and communication skills to evaluate AI-generated responses across a variety of topics. In this role...Hourly payWeekly payContract workFor contractorsRemote workFlexible hoursWeekday AI
United States4 days ago$70 per hour
...AI Research Initiative Role Compensation: $70 per hour Join a cutting-edge AI research initiative focused on improving the quality... ...with exceptional critical thinking and communication skills to evaluate AI-generated responses across a variety of topics. In this...Hourly payContract workRemote workFlexible hoursWeekday
United States3 days ago- ...What Is An Ai Evaluation Specialist? An AI evaluation specialist assesses and tests artificial intelligence systems to ensure they perform accurately, reliably, and safely. They measure how well AI models complete tasks such as answering questions, generating content...Work at officeRemote work
Valence
United States5 days ago - A tech company specializing in AI projects is seeking skilled LibreSprite users to assist in evaluating AI-generated visual content. As an independent contractor, you can work flexibly from anywhere, contributing around 5-20 hours per week depending on project needs. Ideal...For contractorsRemote work
Handshake
United States2 days ago - Medical Specialist (Fluent in Arabic) - Freelance AI Trainer Project World Wide - Remote Are... ...prompt engineering and evaluation metrics. Challenge advanced... ...Background in medicine, healthcare, or biomedical... ...topics across medicine: internal medicine, pharmacology,...Hourly payContract workFor contractorsFreelanceRemote work
$150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...: Remote Role Responsibilities Evaluate how large language models reason through... ...Medical degree : MD or DO . DO and international degrees qualify. ~ Active board certification...Contract workSummer workPrivate practiceRemote work10 hours per weekMercor
Nashville, TN1 day ago- ...in their careers. Three reasons to intern at SAP # Culture of collaboration: meet... ...do: Position title: SAP iXp Intern – AI Engineer Intern (Customer Success) Location... ...emerging AI trends (LLMs, retrieval, evaluation) and summarize applicability to SAP scenarios...Permanent employmentFull timeApprenticeshipInternshipWorldwideFlexible hours
SAP
Naperville, ILa month ago $65 per hour
Prolific is seeking registered nurses in Las Vegas, NV, to assist in training AI models. You will review AI responses, rate their accuracy and safety, and write feedback to enhance AI learning. Candidates must be verified registered nurses, have recent clinical experience...Hourly paySelf employmentWork from homeFlexible hours- ...Join a pioneering AI initiative focused on building next-generation evaluation benchmarks for frontier AI models. We are seeking analytical and technically skilled professionals to identify where advanced AI systems fail in subtle, real-world scenarios. Working in a red...Full timeContract workFor contractorsRemote workFlexible hours
Weekday
Remote22 days ago - ...Join a fast-paced AI evaluation initiative supporting one of the world's leading AI research organizations. We are seeking detail-oriented professionals to evaluate AI-generated outputs by applying structured grading rubrics with precision and consistency. This is...Temporary workImmediate start
Weekday
Remote22 days ago $25 - $30 per hour
...Bilingual Traditional Chinese AI Evaluation Specialist is a remote Chinese specialist track for evaluating chinese evaluation outputs against native-speaker standards. Reviewers spot fluency, register, and cultural-context errors that automated checks miss, and write...For contractorsRemote work10 hours per weekAuraOne Human Data
Remotea month ago- OpenTrain AI seeks a Video Game AI Evaluation Expert on a contractor, part-time basis. You will design prompts for game development, esports, platforms, and communities; assess AI responses for accuracy, completeness and nuance; and develop evaluation datasets with clear...Remote jobPart timeFor contractors
$80 per hour
...Science Expert with Python experience for part-time, remote projects. This role focuses on tasks related to AI systems, including designing problems, evaluating solutions, and validating calculations. Ideal candidates will have a degree in Computer Science, proficiency...Remote jobHourly payPart time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Internal Medicine AI Evaluation Specialist. Be the first to apply!
