Artificial Intelligence/Machine Learning Scientist

Location: Hybrid — (In-Person: Tuesday — Thursday) Department: Innovation and Competency-Based Education Reports to: Director, Assessment Innovation Research Exemption Status: Exempt Job Summary The American Board of Pediatrics (ABP) is seeking a highly skilled and innovative Artificial Intelligence (AI) /Machine Learning (ML) Scientist to join our team. This role is crucial in leveraging AI and ML to enhance the ABP’s assessment programs and services, including processes for initial certification and supporting the continuous professional development of pediatricians as part of continuing certification. The AI/ML Scientist will lead cross-functional efforts to integrate advanced AI techniques into ABP’s assessment design and implementation, enhancing ABP’s operations and innovation research. Essential Duties and Responsibilities Develop and maintain production-quality Python code leveraging libraries such as PyTorch, TensorFlow/JAX, Hugging Face Transformers, scikit-learn, spaCy, NumPy, pandas, and related ML/NLP toolkitsto analyze large-scale text and response data, extract actionable insights from statistical analysis, and inform assessment design and decision-making. Build, evaluate, and deploy AI agents and model-based systems (e.g., LLM-powered agents, retrieval-augmented generation pipelines, automated scoring and feedback agents) to support assessment development, review, and longitudinal learning processes. Lead and collaborate with cross-functional teams (e.g. psychometricians, content experts, IT experts, vendors, etc. ) to conduct research projects and integrate AI solutions into operational assessment systems; serve as a subject matter expert in ML and AI system design for ABP leadership and stakeholders. Identify and prototype opportunities for innovation across ABP’s assessment portfolio, proposing and implementing AI-driven, Python-based solutions that are scalable, interpretable, and aligned with governance and validity requirements. Communicate and disseminate effectively, including presenting and publishing work on AI/ML methodologies and agent-based systems to internal and external audiences such as the ABP Senior Leadership Team, Board of Directors, and the broader pediatric and ABMS communities. Other Functions Supervise additional staff as needed. Attend relevant ABP meetings as needed Perform other duties as assigned. Supervisory Responsibilities There are no current direct supervisory responsibilities. Required Qualifications Bachelor’s degree in statistics/machine learning, NLP/computational linguistics, computer science, bioinformatics, clinical informatics, artificial intelligence, or a related field. For those without an advanced degree (master’s and/or PhD), minimum 5 years of experience in AI/machine learning, with a strong background in developing and implementing AI solutions. Proficiency in programming languages such as Python, R, Java, and cloud platforms such as AWS, Azure, or Google Cloud. Expertise in machine learning, deep learning, and NLP techniques. Innovative thinker, collaborative team player, and passionate about leveraging AI to improve measurement science, assessment of clinical skills of physicians, and pediatric health care. Experience conducting research studies and publishing findings at conferences and journals in AI/ML or related fields. Experience in working on cross-departmental teams and working with external stakeholders. Preferred Qualifications Hands-on experience in building ML pipelines or AI agentic design. An advanced degree (master’s and/or PhD) is highly preferred. Travel Required Travel is expected to be approximately 5%, primarily for national conferences. #J-18808-Ljbffr American Board of Pediatrics