Applied Data Scientist, Finance AI Evaluation & Datasets

Role Description

As an Applied Data Scientist, Financial AI Evaluation & Datasets, you own the design, measurement quality, and domain validity of the datasets used to train, fine-tune, evaluate, and monitor financial-domain LLMs, vision-language models, multimodal document models, and AI agents.

• Translate customer goals into concrete dataset specifications, taxonomies, rubrics, and acceptance criteria.
• Design training and evaluation datasets across the financial AI surface: financial QA, filings and earnings analysis, credit and underwriting, fraud/AML investigation, and compliance.
• Foreground unstructured and multimodal financial data in dataset design.
• Design datasets and evaluations for retrieval-augmented and source-grounded systems.
• Evaluate agentic and workflow-integrated financial AI systems.
• Develop evaluation methodology that goes beyond surface accuracy.
• Define sampling strategies, label schemas, and adjudication workflows with Language Data Scientists and finance SMEs.
• Build the statistical and ML tooling that makes large financial datasets trustworthy.
• Build evaluation and dataset-quality evidence to support financial-services model risk management.
• Partner with the AI/ML Research Engineer to instrument datasets into training, evaluation, and monitoring pipelines.
• Own data quality end-to-end, from intake through delivery.
• Reason about financial workflow context.
• Support the Technical Solutions Architect during customer discovery and proposals.
• Stay current on the financial AI landscape.
• Contribute to Innodata internal IP.

Qualifications

• 5+ years of data science experience, with at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
• Real working knowledge of financial data and workflows.
• Hands-on experience with unstructured and multimodal financial data.
• Familiarity with financial standards or protocols such as XBRL, ISO 20022, or GAAP/IFRS reporting concepts is strongly preferred.
• Hands-on experience designing datasets for ML.
• Familiarity with LLM-based and multimodal financial AI workflows.
• Strong Python and SQL; comfort with pandas, scikit-learn, or equivalent.
• Statistical literacy.
• Solid grasp of financial services privacy, compliance, and governance.
• Excellent collaboration skills.
• A bias toward financial workflow realism.
• Degree in a relevant field — statistics, data science, economics, finance, or a related quantitative field.
• Experience designing evaluations for LLMs, VLMs, or multimodal models in financial reasoning.
• Experience with document AI, OCR/post-OCR quality, or table and chart extraction for complex financial documents.
• Familiarity with agentic evaluation, AI observability, experiment tracking, or tools such as Weights & Biases or LangFuse.
• Familiarity with model risk management frameworks, validation documentation, fairness/bias auditing, or consumer protection analysis.
• Experience with multilingual or cross-border financial data.

Requirements

• The expected salary range for this position is $150,000 – $175,000 USD per year, based on experience, skills, and qualifications.

Company Description

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale.