Psychometrician

Role Description

Under the guidance of the Director of Psychometrics and Quantitative Research, the Psychometrician is responsible for:

• Designing and conducting psychometric analyses of test performance.
• Conducting psychometric analyses related to test construction, validity, reliability, equating, and scoring.
• Preparing technical reports.
• Developing, maintaining, and applying quality control procedures.
• Providing quantitative guidance to professional research staff at CAL.
• Coordinating external quantitative resources on behalf of the organization.

The Psychometrician will also:

• Provide quantitative guidance to other researchers at CAL.
• Coordinate external quantitative resources on behalf of the organization.

Essential Duties:

• Design and conduct psychometric analyses for the WIDA ACCESS program.
• Plan, conduct, document, and present psychometric and statistical analyses using classical test theory and the Rasch model with Winsteps and Facets software, including analyses related to:
  • Item quality
  • Rater performance
  • DIF
  • Reliability and classification consistency
  • Standard setting
  • Equating
  • Scoring and reporting
• Investigate and evaluate new approaches to conducting psychometric analyses in language testing.
• Establish and monitor quality control procedures related to quantitative analyses, from data cleaning to final technical reports, ensuring the reliability and integrity of results.
• Provide guidance to professional researchers on proposal design and quantitative analytical approaches upon request and participate in writing quantitatively-oriented portions of proposals as needed.
• Support outreach and collaboration, including research, development, and client development activities with external partners, academia, innovators in education and language, and the public as needed.

Qualifications

• A Ph.D. in Educational Measurement, Psychometrics, or related field.
• A minimum of five years of demonstrated operational experience in designing and conducting statistical and psychometric analyses for high stakes testing programs.
• Strong demonstrated experience in applying the Rasch model to a variety of psychometric issues.
• Experience in conducting analysis on performance and polytomously-scored tasks.
• Demonstrated proficiency in programming in R or Python.
• Proficiency in using SPSS, Winsteps, Facets, and other statistical and psychometric software.

The ideal candidate for this role has:

• Experience working within a psychometric program in a K-12 U.S. Educational Context, and preferably in language testing.
• Experience communicating effectively in English with clients and internal and external stakeholders in high-stakes contexts.
• Experience working as a member of a team in producing timely error-free deliverables.

Requirements

• Create and apply quality control procedures in creating and maintaining databases, conducting statistical and psychometric analyses, conducting other quantitative research, and producing technical reports.
• Program, document procedures, and instruct junior members of the team in statistical software such as R, Python, SPSS, Winsteps, Stata, and SAS.
• Create replicable procedures for routine psychometric work.
• Write narrative to accompany psychometric analyses for technical reports and other documentation.

The ideal candidate for this role has:

• Demonstrated ability to work with a diverse, multi-disciplinary professional team of researchers and language testers.
• Strong demonstrated experience with computerized adaptive assessments.
• Experience in conducting standard setting study and/or alignment studies.

Travel Requirements

• This position requires occasional travel (1-3 times per year).

Benefits

• Minimum Salary: $101,970.00; Benefits eligible.

Physical Demands

The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. While performing the duties of this job, the employee is regularly required to effectively communicate. The employee may be required to sit or stand in a stationary position for 50% of the time. This position constantly operates a computer and other office productivity equipment. CAL adheres to ADA compliance and guidelines as a common practice.

Additional Information

This job description summarizes the main duties of the job. It neither prescribes nor restricts the exact tasks that may be assigned to carry out these duties. This document should not be construed in any way to represent a contract of employment. Management reserves the right to review and revise this document at any time.

Affirmative Action/Equal Opportunity Employment Statement

The Center for Applied Linguistics (CAL) provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity, disability, or genetics. In addition to federal law requirements, CAL complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.

CAL expressly prohibits any form of workplace harassment based on race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. Improper interference with the ability of CAL employees to perform their job duties may result in discipline up to and including discharge.