Psychometrician

Description

NCCER is seeking an experienced Psychometrician to partner with internal team members and external customers to execute the full range of systems and processes for test design, development, assembly, and validation. Desired candidates will have experience and expertise in working with external customers to conduct job/task analyses, item development, and standard setting workshops. In addition, they should also be able to collaborate with internal team members to develop test specifications and conduct psychometric analyses.  

Major Duties

1. Execute the full range of systems and processes related to test design, development, assembly, and validation
2. Adhere to the recognized standards for educational and psychological testing for all systems and processes
3. Execute research studies to help solve job-related problems, examine the effectiveness of systems and processes, and evaluate new approaches and analytical techniques
4. Constantly update knowledge of the changes in the field of educational measurement and related sub-fields such as psychometrics, computational psychometrics, and assessment engineering
5. Collaborate and coordinate with internal team members and SMEs to conduct job/task analyses
6. Collaborate with internal team members to develop test specifications
7. Collaborate with internal team members and SMEs to develop performance level descriptors
8. Collaborate and coordinate with internal team members, SMEs, and external partners to execute item development workshops (in-person and virtual)
9. Coordinate with internal team members, SMEs, and external partners to execute in-person standard setting workshops
10. Coordinate with internal team members to assemble test forms (manually or automated)
11. Conduct standard and advanced psychometric analyses (classical test theory and item response theory) to evaluate item and test functioning, possibly including differential item and test functioning, where possible
12. Execute procedures for assessment engineering 

Requirements

• M.S. or Ph.D. in quantitative psychology; industrial/organizational psychology; research evaluation and methodology; psychometrics; assessment and measurement; or other fields related to educational measurement 
• Experience that allows for the ability to successfully execute the major duties listed above
• Knowledge of classical test theory and item response theory (dichotomous; polytomous models is a bonus competency)
• Knowledge of standard statistical analyses such as (but not limited to) ANOVA, chi-square, various types of correlations, and various types of regression (e.g., multiple or logistic)
• Knowledge of machine learning techniques for pattern identification and clustering (bonus competency)
• Knowledge of methods for estimating longitudinal growth (bonus competency)
• Familiarity with the Standards for Educational and Psychological Testing
• High level of detail-orientation
• Excellent time management skills
• Ability to communicate complex concepts and procedures in simple, understandable ways to internal team members and external partners

Software Skills

• Microsoft Office 365 (Word, PPT, Outlook, Excel, and Teams)
• Ability to write code for database queries, psychometric analyses, and visualization in R (preferred) or Python