Chief Assessment Officer (The ABA)

Chief Assessment Officer

Raleigh, North Carolina
Hybrid schedule with a minimum of two days per week on-site
Reports to the Executive Director

Help shape the future of physician assessment and certification.

The American Board of Anesthesiology (ABA) seeks a Chief Assessment Officer (CAO) to lead the science, strategy, and innovation behind its assessment, credentialing, and certification programs. This is an opportunity for an accomplished psychometric expert and executive leader to advance how physician knowledge and performance are evaluated by combining scientific rigor, emerging technology, and a commitment to high-quality patient care.

Since 1938, the ABA has served the anesthesiology profession through certification. Today, the organization is building on that foundation with an innovative approach to assessment and continuous professional development. Its mission is to advance the highest standards of anesthesiology practice and help physicians provide the best possible care for every patient, every day.

The Opportunity

As a member of the Executive Leadership Team, the CAO will help shape enterprise strategy while overseeing the development, validation, delivery, and continuous improvement of the ABA’s assessment portfolio. The role leads approximately 33 staff through three direct reports across Assessment Development, Psychometrics and Research, Credentialing, and Assessment Delivery.

The incoming leader will help guide the ABA toward a digital-first assessment ecosystem by exploring artificial intelligence, adaptive approaches, simulation, and assessment methods that extend beyond traditional multiple-choice examinations. Success requires recognized credibility in measurement science and the executive judgment to translate innovation into valid, reliable, fair, and accessible certification programs.

This leader must think beyond departmental priorities, collaborate across the organization, and build trust with physician leaders, governing board members, and an experienced multidisciplinary team.

What You Will Lead

• Enterprise strategy and assessment innovation. Partner with the Executive Leadership Team and Board of Directors to shape long-range priorities, evolve certification models, and identify approaches that strengthen the ABA’s mission and physician experience.
• Assessment and psychometric excellence. Oversee the complete assessment lifecycle, including examination design, development, administration, psychometric analysis, scoring, and reporting.
• Credentialing and continuing certification. Lead the credentialing and continuing certification enterprise, ensuring that eligibility, certification status, pathway requirements, and diplomate services are integrated with assessment strategy and program operations.
• Research leadership. Direct validation and research efforts that evaluate program effectiveness, inform organizational decisions, and advance the ABA’s contributions to assessment and medical education.
• Responsible technology advancement. Partner with technology leaders to establish platform requirements and investment priorities, including the validation, governance, and ethical use of artificial intelligence in assessment development and scoring.
• Operational and financial performance. Oversee budgets, vendors, resource planning, quality measures, and process improvements that support secure, scalable, and efficient assessment delivery.
• Talent and organizational leadership. Develop high-performing leaders and teams, guide change thoughtfully, and foster a culture of collaboration, inclusion, accountability, and continuous learning.

The Leader We Seek

The successful candidate will bring deep expertise in psychometrics and high-stakes assessment, along with the ability to lead at the enterprise level. This person can explain complex scientific concepts clearly, build productive partnerships across disciplines, and distinguish promising innovation from approaches that require further evidence. The ideal candidate will be equally comfortable advising a governing board, developing people, and translating strategy into operational results.

Qualified candidates will bring:

• A doctoral degree, such as a Ph.D., Ed.D., Psy.D., or equivalent, in Educational Measurement, Psychometrics, Educational Psychology, Industrial and Organizational Psychology, Statistics, Measurement and Evaluation, or a closely related discipline.
• At least 10 years of progressively responsible leadership experience in assessment, credentialing, certification, licensure, educational measurement, or a related field.
• Executive leadership experience overseeing complex, high-stakes assessment programs, with demonstrated expertise in psychometric theory, assessment design, research, and statistical methodologies.
• Experience leading multidisciplinary teams and significant organizational transformation.
• Experience partnering with technology teams to advance assessment platforms and digital products, with knowledge of artificial intelligence, advanced analytics, and emerging assessment technologies.
• Strong strategic, financial management, communication, and organizational development capabilities.

Experience working with physician volunteers, governing boards, or professional associations is strongly preferred.

Working at the ABA

The ABA is a fast-paced organization that continuously innovates to meet the evolving needs of anesthesiologists and the profession. The organization offers a welcoming and inclusive environment that values professional growth, flexibility, collaboration, and continuous learning. The ABA embraces change as an opportunity to solve meaningful challenges and improve how it serves physicians.

This position requires an on-site presence in Raleigh, North Carolina, at least two days per week. Occasional travel of up to 10%, including overnight stays, is also required.

Our Commitment to Diversity, Equity, Inclusion & Belonging

Nonprofit HR is committed to fostering and maintaining a work environment where diversity, equity, and inclusion (DEI) are fully integrated into everything we do for the benefit of our employees and the clients that we serve. To fully realize our goal, we prioritize our understanding of the complexities of DEI within our workforce to inform our approach to talent management. We believe that this guides how we do our work, advise our clients to operationalize DEI and position our content and educational opportunities help strengthen the talent management capacity of the social impact sector.

Continue reading our about our commitment at nonprofithr.com/deinow.