Psychometric Externship

Job Description

Job Description

Education is ever-evolving. And so are we.

We seek passionate people from diverse backgrounds who are committed to helping improve the work of schools as a trusted partner in advancing learning. By joining the Cognia team, you will be immersed in a community of energetic and resourceful people dedicated to a common vision to impact and inspire education providers to advance and enable pathways of success for all learners. At Cognia we have developed a holistic approach to education that empowers schools to create real outcomes and optimize learning for students of all backgrounds, servicing more than 36,000 institutions, 5 million teachers, and 25 million students in 85 countries. When you join Cognia, you join a team of experts passionate about our mission and embracing our core values in everything we do.

Cognia is granting up to two (2) externships (i.e., remote internships) for students working toward PhDs in appropriate graduate study programs. The primary focus of the program centers around an appropriate research project that each student chooses in consultation with Cognia psychometricians and researchers.

The externships are targeted for students whose interests focus on psychometric or measurement issues such as equating, item calibration, statistical models, response time, dimensionality issues, and scaling. For example, recent years have included projects on evaluating raters of open-response items, aberrant response analyses, diagnosing student misconceptions on open-response items, modeling/research related to technology enhanced items or response time behavior, detecting unusual item response behavior in alternate assessments, detection of item parameter drift, and accuracy of routing in multistage tests.

An additional important component of the externship program is to help students gain in-depth knowledge and understanding of the daily activities of Cognia psychometricians via a series of seminars that give a systematic deep dive into our psychometric practices.

Externship Overview

The 14-week externship program will encompass:

- A program designed around accomplishing a research project or application, eligible for submission for presentation at a national conference.

- Regular Zoom meetings with mentors around goal setting, monitoring progress, and ensuring project completion.

- A series of seminars from both Cognia psychometricians and other related groups will provide an in-depth knowledge of how psychometrics is applied in a variety of ways in the educational testing industry. A significant proportion of these seminars will occur during a one-week in-person visit to Cognia offices in Portsmouth, NH.

- Externs will work on projects about 20 hours per week (some weeks more, some less).

- Externs will be provided with access to all needed psychometric resources by use of secure file-sharing sites or by use of a Cognia laptop.

Dates, Locations, and Compensation

The 2027 Cognia Externship is a primarily remote program, apart from the ‘Extern Festival Week’ during which externs will travel to Cognia’s offices in Portsmouth, New Hampshire to engage in seminars, presentations, and in-person networking with each other and Cognia personnel.

Important Dates

- Application deadline: Monday, November 16th, 2026

- Interview window: November 30 – December 8, 2026

- Decision date: On or before December 18th, 2026

- Externship start date: Monday, January 25th, 2027.

• The duration is 14 weeks with an end date of April 30th, 2027

- Extern Festival Week: One week duration in March 2027 when externs travel to Cognia offices for in-person networking and seminars. Exact dates will be determined based on participants’ availability.

Compensation

The full compensation package includes the following:

- A stipend in the amount of $7,000.

- Costs associated with the Extern Festival Week. This will include flights/transportation to and from Portsmouth, New Hampshire, as well as hotel accommodation and all meals during the week.

- Costs associated with equipment. Cognia will set up and ship a computer and any associated hardware to you and arrange for shipment back to Cognia at the conclusion of the externship.

Who should apply?

Candidates enrolled in a psychometric, educational measurement, or other related doctoral program (e.g., quantitative psychology, learning sciences, etc.) at a fully accredited university/college in the United States are encouraged to apply.

Eligibility Requirements:

- Current enrollment in a relevant doctoral program.

- Completion of at least 2 years of coursework toward the doctorate prior to the externship program start date.

- Currently residing in the United States.

What to include in the application

Applications to the Externship program should include (1) a resume or CV and (2) a one-page research interest statement.

The research interest statement should include a description of your research interests, which could include topics you have some experience with but could also describe research interests you have not yet had the opportunity to pursue. Descriptions of some projects of interest to Cognia are listed below for reference. You are also encouraged to describe your own research interests; we are open to a variety of projects, including emerging areas such as applications of generative AI in educational measurement.

Incomplete applications will not be considered.

Project Descriptions

For the 2027 Cognia Externship program, our psychometrics team has special interests in research related to the following areas:

- Standard Setting: Research on the ID Matching method, such as building a cognitive model of how panelists reach their judgments or statistically modeling those judgments, to better understand and strengthen the method.

- Cognitive Diagnostic Models (CDMs):

• Developing a framework for designing formative assessments for use with CDMs, from defining attributes and building the Q-matrix through item development.
• Applying CDMs to existing formative assessments in science and evaluating how useful the resulting diagnostic information is.

- Aberrant Response Analyses: Using innovative methods to analyze large-scale response data to detect and flag unusual responses or response patterns at the individual and/or group level.

- Technology Enhanced Items (TEIs): Investigating how TEIs function psychometrically, including misfitting TEIs, ways to model them, and comparisons with other item types (e.g., multiple-choice items).

-Calibration and Equating:

• Calibrating items using smaller sample sizes (e.g., calibrating items with the 2PL model based on a sample of around 300).
• Using non-representative samples for calibration and equating, and how this might affect ability estimates.
• Evaluating scale drift across administrations, including how changes in item position or context (e.g., when items are reused across forms or programs) affect item parameter estimates and scale stability.

- Measurement Invariance: Studying measurement invariance across different student populations.

- Multistage Testing: Psychometric evaluation of multistage tests.

- Dimensionality: Analyzing and evaluating multidimensionality, especially with smaller sample sizes.

If one or more of the projects above match your interests, say which ones in your research interest statement and add some details about why they interest you or how you think you can contribute.

Note: Applicants are NOT required to select any of the above projects. If none of them interest you, please use your research interest statement to describe your other research interests.

#LI-Remote