Applied Data Scientist, Health AI Evaluation & Datasets

Role Description

As an Applied Data Scientist, Health AI Evaluation & Datasets, you own the design, measurement quality, and clinical validity of datasets used to train, fine-tune, and evaluate health-domain models. You bring clinical or biomedical fluency and data science rigor.

• Translate customer goals into dataset specifications, taxonomies, rubrics, sampling plans, and acceptance criteria.
• Design training and evaluation datasets across various clinical artifacts and use cases.
• Design evaluations for retrieval-augmented and source-grounded health AI systems.
• Define sampling strategies, label schemas, inter-annotator agreement targets, and quality thresholds.
• Build statistical and ML checks that make healthcare datasets trustworthy.
• Partner with Applied Research Scientists and AI/ML Research Engineers to instrument datasets into evaluation and post-training pipelines.
• Evaluate health AI behavior beyond surface accuracy.
• Own data quality from source intake through delivery.
• Stay current on the health AI landscape.
• Support customer discovery and proposal work.
• Contribute to Innodata internal IP.

Qualifications

• 5+ years of data science experience, including at least 2+ years with healthcare or comparable regulated health data.
• Working knowledge of healthcare data and standards.
• Hands-on experience designing ML datasets.
• Familiarity with LLM-based health AI workflows.
• Strong Python and SQL skills.
• Statistical literacy across sampling design and bias analysis.
• Solid grasp of healthcare privacy, compliance, and governance.
• Ability to work credibly with various stakeholders.
• A bias toward clinical realism.
• Degree in a relevant field or equivalent demonstrated experience.
• Clinical credentials are not required, but candidates with relevant backgrounds are encouraged.

Requirements

• Ability to work with clinicians, biomedical SMEs, and health AI customers.
• Experience with healthcare privacy and compliance standards.
• Statistical literacy and ability to analyze data effectively.

Benefits

• The expected salary range for this position is $150,000 – $175,000 USD per year, based on experience, skills, and qualifications.

Company Description

Innodata (Nasdaq: INOD) is a global data engineering company focused on enabling the responsible advancement of artificial intelligence through high-quality data and expertise.