Applied Data Scientist, Finance AI Evaluation & Datasets

Role Description

As an Applied Data Scientist, Financial AI Evaluation & Datasets, you own the design, measurement quality, and domain validity of the datasets used to train, fine-tune, evaluate, and monitor financial-domain LLMs, vision-language models, multimodal document models, and AI agents.

• Translate customer goals into concrete dataset specifications, taxonomies, rubrics, and acceptance criteria.
• Design training and evaluation datasets across the financial AI surface: financial QA, filings and earnings analysis, credit and underwriting, fraud/AML investigation, and compliance.
• Foreground unstructured and multimodal financial data in dataset design.
• Design datasets and evaluations for retrieval-augmented and source-grounded systems.
• Evaluate agentic and workflow-integrated financial AI systems.
• Develop evaluation methodology that goes beyond surface accuracy.
• Define sampling strategies, label schemas, and adjudication workflows with Language Data Scientists and finance SMEs.
• Build the statistical and ML tooling that makes large financial datasets trustworthy.
• Build evaluation and dataset-quality evidence to support financial-services model risk management.
• Partner with the AI/ML Research Engineer to instrument datasets into training, evaluation, and monitoring pipelines.
• Own data quality end-to-end, from intake through delivery.
• Reason about financial workflow context.
• Support the Technical Solutions Architect during customer discovery and proposals.
• Stay current on the financial AI landscape.
• Contribute to Innodata internal IP.

Qualifications

• 5+ years of data science experience, with at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
• Real working knowledge of financial data and workflows.
• Hands-on experience with unstructured and multimodal financial data.
• Familiarity with financial standards or protocols such as XBRL, ISO 20022, or GAAP/IFRS reporting concepts is strongly preferred.
• Hands-on experience designing datasets for ML.
• Familiarity with LLM-based and multimodal financial AI workflows.
• Strong Python and SQL; comfort with pandas, scikit-learn, or equivalent.
• Statistical literacy.
• Solid grasp of financial services privacy, compliance, and governance.
• Excellent collaboration skills.
• A bias toward financial workflow realism.
• Degree in a relevant field — statistics, data science, economics, finance, or a related quantitative field.
• Experience designing evaluations for LLMs, VLMs, or multimodal models in financial reasoning, filings analysis, or fraud/AML contexts.
• Experience with document AI, OCR/post-OCR quality, or table and chart extraction for complex financial documents.
• Familiarity with agentic evaluation, AI observability, experiment tracking, or tools such as Weights & Biases or LangFuse.
• Familiarity with model risk management frameworks, validation documentation, fairness/bias auditing, or consumer protection analysis.
• Experience with multilingual or cross-border financial data.

Requirements

• The expected salary range for this position is $150,000 – $175,000 USD per year, based on experience, skills, and qualifications.

Benefits

• Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers.
• Innodata will never ask for payment, banking details, or sensitive personal information during the application process.
• To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide.
• If you believe you’ve been targeted by a recruitment scam, please report it to Innodata.