Lead Data Scientist

Lead Data Scientist

The Association of American Medical Colleges is a not-for-profit association dedicated to transforming health care by supporting the entire spectrum of medical education, medical research, and patient care conducted by our member institutions. We are dedicated to the communities we serve and steadfast in our goal to improve the health of all.

At the AAMC, we are committed to supporting our employees with a comprehensive and competitive benefits package designed to promote well-being, professional development, and work-life balance.

Additional information can be found on our website.

Lead Data Scientist is responsible for leading and conducting research and analysis to evaluate, enhance, or create products and services. Collaborates with internal colleagues across the organization and external partners throughout research and development to achieve significant impact. Leads the design, application, and oversight of methodological frameworks, including advanced statistical modeling, analytic strategy, and evaluation plans; ensures rigor, reproducibility, and alignment with strategic research objectives. Analyzes data, prepares data visualizations and summarizes results and delivers reports describing results and recommendations for technical and non-technical audiences. Queries, extracts, and transforms data from various AAMC databases and repositories, such as the data warehouse and data mart; sets coding standards and develops reusable analytical assets and automation tools. Collaborates cross-functionally and provides scientific leadership, consulting on research direction, mentoring analysts, and partnering with stakeholders to shape products, dissemination strategies, and research roadmaps that advance organizational priorities.

Support and Conduct Research to Improve Residency Selection:

• Lead the design, execution, and scientific direction of research portfolios focused on residency selection, applicant behavior, and the evaluation of new selection tools; ensure methodological rigor, psychometric soundness, and clear research pathways from concept to implementation.
• Identify research questions and help to define research priorities and analytic strategy at scale, developing conceptual frameworks, research questions, methodological approaches, and statistical models to evaluate tool performance and policy impacts.
• Serve as a primary scientific liaison with internal teams, external vendors, advisory committees, and specialty stakeholders, anticipating risks, making methodological recommendations, and resolving complex technical issues.
• Oversee evidence synthesis and knowledge integration, guiding literature reviews, environmental scans, and validity frameworks to inform adoption and responsible use of emerging assessment and selection tools.
• Lead research governance and compliance processes, including IRB compliance, DUAs, NDAs, and legal agreements, ensuring ethical data use, risk mitigation, and alignment with organizational policies and related guidelines.
• Lead dissemination of findings through manuscripts, technical briefs, and presentations, ensuring transparent documentation of methodology and results, and mentor junior staff in scholarly writing and publication standards.

Data Management:

• Provide strategic direction for data architecture and management processes, developing and refining standards, governance frameworks, and workflows that support large-scale residency selection research and product development.
• Lead complex data extraction, integration, and transformation from AAMC repositories such as the data warehouse and data mart, optimizing query performance, automating repeatable workflows, and ensuring scalability and traceability across projects.
• Serve as primary data liaison with internal teams and/or external partners to align on data formats, schedules, and security considerations.
• Oversee operational and research data exchanges, including recurring and ad-hoc transfers, and ensure timeliness, accuracy, and compliance.
• Lead quality assurance, validation, and risk mitigation processes for incoming and outgoing data, proactively identifying anomalies and directing resolution with technical teams and stakeholders.
• Develop and reproducible analytical workflows, including dataset production, documentation standards, code versioning, and transparent lineage tracking.
• Perform data validation for data displayed in products or tools (e.g., Residency Explorer, Careers in Medicine) to ensure data is rendered accurately and in compliance with data display rules.
• Conduct quality assurance checks of data to identify red flags and coordinate with AAMC colleagues to investigate and resolve issues.
• Clean data, document syntax, and analyze external analytic data sources in collaboration with research partners and constituents.
• Analyze qualitative and quantitative data to answer questions about residency selection or assessments, as needed (e.g., advanced analytics and AI, regressions and multilevel modeling, correlations and descriptive analyses).
• Synthesize findings for leadership and external audiences, preparing executive-ready summaries, technical reports, and presentations, and mentor junior analysts in analytical reasoning and communication.

Develop Resources and Deliverables for External and Internal Audiences:

• Lead the definition of project scope, research objectives, and deliverables, ensuring alignment with strategic priorities, scientific rigor, and operational feasibility.
• Facilitate meetings, take notes, and follow up on action items. Facilitate and guide cross-functional collaboration, setting agendas, driving decision-making discussions, ensure action items translate into measurable research progress, and delegate tactical tasks where appropriate.
• Design and oversee creation of compelling data visualizations and evidence-driven narratives, ensuring accuracy, methodological transparency, and effective translation of complex analytical findings for varied audiences.
• Develop high-quality presentations, reports, and leadership materials for leadership briefings, advisory committees, and external stakeholders, coaching team members on evidence communication and messaging strategy.
• Serve as lead on manuscripts, conference submissions, and scholarly products, ensuring methodological clarity, alignment with publication standards, and dissemination of findings to the medical education community.

Relationship Management:

• Cultivate and steward high-level collaborations across ASRD, the Services, and other clusters aligning research priorities with enterprise strategy and enabling coordinated delivery of high-impact solutions.
• Establish relationships with constituents and colleagues in external organizations to co-create high-impact products and services that deliver meaningful value to the community.
• Represent the research portfolio in cross-organizational discussions, providing thought leadership, negotiating research scope and expectations, and ensuring stakeholder alignment throughout the project lifecycle.

Required Qualifications:

• Minimum Education: Master's degree in Psychology or related field
• Preferred Education: Masters degree in industrial-organizational psychology, educational measurement, economics, data science or related fields

Required Experience:

• 8-10 years of related work experience.
• 1-3 years of supervisory experience.

Preferred Experience:

• Research Experience: Hands-on research experience in selection, admissions, educational measurement or related field, with a demonstrated track record of producing high-quality applied research. This may include working on large, complex research projects, technical reports, publications in peer-reviewed journals, and expertise in research methodologies.
• Project Management: Experience in managing complex research projects, including budgeting, planning, and execution, as well as proficiency in statistical tools, research software, and other relevant technologies.
• Strategic Oversight: Experience in aligning research objectives with organizational goals, contributing to long-term strategic planning, and effectively communicating research findings to stakeholders at various levels (internal teams, external partners).
• Management Experience: Experience in a managerial role, such as managing research teams, overseeing projects, and providing mentorship.

Additional Information:

• Expert-level knowledge of personnel selection science, validation frameworks, and evidence-based assessment design, including predictive modeling, fairness evaluation, scoring, and implementation in high-stakes selection contexts.
• Deep expertise in research design, statistics, psychometrics, and measurement theory, with the ability to select, justify, and operationalize methodological approaches for large-scale, complex research initiatives.
• Leads project strategy and execution, establishing timelines, resource plans, analytic milestones, and risk-mitigation structures; ensures successful delivery across concurrent, high-visibility initiatives.
• Proactively identifies, resolves or escalates methodological, analytic, and operational issues, exercising sound judgment in determining escalation pathways, communicating trade-offs, and guiding decision-making.
• Demonstrates strategic prioritization and workload balancing, managing multiple research streams while maintaining scientific rigor, documentation quality, and timely delivery.
• Time-management skills with the ability to prioritize multiple, critical tasks while maintaining a high level of service and attention to detail and adhering to a strict schedule.
• Exceptional written and verbal communication skills, capable of translating complex analyses into clear narratives for leaders, researchers, product teams, and a wide variety of stakeholders, including AAMC staff and constituents.
• Effectively influences and leads within collaborative and consensus-driven environments, building alignment across diverse perspectives and guiding stakeholders toward evidence-based decisions.
• Maintains strict adherence to AAMC policies regarding confidentiality, data privacy, personally identifiable information and governance policies, ensuring compliant handling of sensitive data.
• Expert in advanced statistical and