Asst Dir-Data Scientist

Moody's OpportunityAt Moody's, we unite the brightest minds to turn today's risks into tomorrow's opportunities. We do this by striving to create an inclusive environment where everyone feels welcome to be who they are—with the freedom to exchange ideas, think innovatively, and listen to each other and customers in meaningful ways. Moody's is transforming how the world sees risk. As a global leader in ratings and integrated risk assessment, we're advancing AI to move from insight to action—enabling intelligence that not only understands complexity but responds to it. We decode risk to unlock opportunity, helping our clients navigate uncertainty with clarity, speed, and confidence.If you are excited about this opportunity but do not meet every single requirement, please apply! You still may be a great fit for this role or other open roles. We are seeking candidates who model our values: invest in every relationship, lead with curiosity, champion diverse perspectives, turn inputs into actions, and uphold trust through integrity.Skills and CompetenciesPh.D. in Computer Science, Machine Learning, Natural Language Processing, Statistics, or a related quantitative field; or Master's degree with 2-3 years of experience in machine learning evaluation or a related areaStrong foundations in statistical methods, experimental design, and hypothesis testingExperience evaluating machine learning or NLP models, including designing experiments and interpreting resultsFamiliarity with LLM evaluation benchmarks and methodologiesStrong programming skills in Python or RExcellent communication skills in English (both written and verbal)Preferred:Experience evaluating LLMs or generative AI systemsExperience with production machine learning systemsExposure to cloud platforms such as AWS, GCP, or AzurePublications or demonstrated work in model evaluation, benchmarking, or related areasEducationPh.D. in Computer Science, Machine Learning, Natural Language Processing, Statistics, or a related quantitative field; or Master's degree with 2-3 years of experience in machine learning evaluation or a related areaResponsibilitiesEvaluate and validate large language models for production-grade analytical and decision-support systemsDesign and implement evaluation frameworks for assessing LLM performance in credit analytics and decision-support contextsDevelop metrics and benchmarks to measure model robustness, reliability, consistency, and output qualityAnalyze model behavior across diverse inputs, identifying failure modes, edge cases, and areas for improvementCollaborate with model development and deployment teams to integrate validation processes into the model lifecycleConduct systematic assessments of model stability over time and across updatesEvaluate model outputs for bias, fairness, and economic relevance to credit risk applicationsDevelop and maintain documentation for evaluation methodologies, findings, and recommendationsContribute to the advancement of best practices for LLM evaluation within the Credit COEAbout the TeamOur Credit Center of Excellence (COE) team is responsible for maintaining and enhancing our industry-leading credit analytics and predictive modelling capabilities. We work closely with various teams including product management, commercial strategy, and go-to-market leaders to ensure the delivery of high-quality credit risk assessments and solutions. By joining our team, you will be part of exciting work in credit analytics with a global team spread across all US time zones, GMT, and GMT+1.