Senior Psychometrician & Psychometrician II

OVERVIEW OF POSITIONS

NBME®'s Psychometrics and Data Analysis (PADA) unit is seeking a Psychometrician and a Senior Psychometrician to join their team. Interested candidates when applying should include a cover letter to indicate which position they are applying for and their interest in and fit for that role. These positions are expected to make contributions in four primary areas:

• Applied research and development in support of modifications to examination designs, and analysis methods and procedures.
• Planning and execution of psychometric analyses, scoring, and score reporting for operational examination programs.
• Contribute to the educational measurement community via professional presentations and publications related to independent and organizational research agendas.
• Evaluate existing procedures for improvement opportunities, identify new procedures to increase accuracy and efficiency of exam processing.

The Senior Psychometrician is expected to make primary contributions to one additional area:

• Designing and evaluating data visualization and graphical representation of data for stakeholder feedback, examinee feedback reports, and interactive data tools.

Both positions must effectively collaborate with a variety of internal staff including data analysts, test developers, information technologists, and program managers. In addition, external facing communication and engagement with clients and subject matter experts from a wide range of medical and health care fields is also necessary.

Operational examination programs at NBME include the United States Medical Licensing Examination®, an extensive medical school examination program that supports the assessment needs of US and international medical schools, and numerous client examinations primarily in support of medical specialties. The variety of assessment programs creates a wealth of data and opportunities for applied and theoretical research that can impact practice.

Both roles have been designated as primarily remote, which means they generally do not require onsite work more than an average of once per quarter, though may require additional prescheduled onsite work. Options to work in person in Philadelphia are also available.

Diversity, Equity, and Inclusion Statement

At NBME, we continue to innovate and improve how we fulfill the evolving needs of the health care community. This commitment starts and ends with the people at NBME. By recruiting and empowering talented individuals from various disciplines and backgrounds, which includes professionals with diverse life experiences, abilities, and perspectives, NBME can take a well-informed, robust approach to advancing medical education and assessment for years to come. We also continue to focus on ensuring that our DEI work is impactful and ingrained in everything we do, including with our staff, workplace culture, products and services, the Philadelphia community and the broader medical education landscape.

QUALIFICATIONS

Skills and Abilities

• Strong background in psychometric theory and application, including knowledge of IRT, CTT, statistics, test development procedures, and research design.
• Excellent computer skills and a working knowledge of statistical software (e.g., Calibration software, SAS, Python, R).
• Excellent oral and written communication skills.
• Ability to identify errors and solve problems.
• Excellent interpersonal skills, including the ability to work closely and successfully with individuals from a variety of professional and educational backgrounds.
• Knowledge and/or experience in natural language processing, automated item generation, process data analytic methods, formative assessment approaches is desirable.

Additionally, for the Senior Psychometrician:

• Advanced knowledge of applied measurement topics, such as equating, scaling, generalizability theory, Bayesian methodology, and/or standard setting.
• Demonstrated strength in presenting assessment data graphically to facilitate user comprehension, support correct inferences, and guide subsequent actions.

Experience

For the Senior Psychometrician, a minimum of 5 years of experience in a Psychometrician/Measurement Scientist role is required for consideration.

For the Psychometrician, a minimum of 1 year of experience is required for consideration.

Education

Doctorate degree in Educational/Psychological Measurement, Statistics, or a related field is required.

About NBME:

NBME offers a versatile selection of high-quality assessments and educational services for students, professionals, educators, regulators and institutions dedicated to the evolving needs of medical education and health care. To ensure our assessments meet the highest standards of quality, stay relevant and align to the current curriculum in medical schools and training programs, we rely on a wide network of collaborators. These include the volunteers who help develop our exam questions, the committees and panels who represent various groups within the medical education community, external researchers and health profession organizations.

We are committed to meeting the needs of educators and learners globally with assessment products and expert services such as NBME® Subject Examinations, Customized Assessment Services, Self-Assessments, the International Foundations of Medicine® Program and Item Writing Workshops. Together with the Federation of State Medical Boards, NBME develops and manages the United States Medical Licensing Examination®, which measures the ability to apply knowledge and skills that form the basis of safe and effective patient care. Our Competency-based Assessment unit is focused on new methods as well as the optimization of assessment in the workplace and education.

As a result of leadership in ongoing research, innovative measurement practices and the exploration of forward-thinking assessment modalities and improvements, NBME advances assessment science. Our grant and funding opportunities further support this dedication to medical education and assessment science. We help develop the next generation of assessment professionals through our Summer Psychometric Internship Program. Through the Stemmler Fund, Strategic Educators Enhancement Fund and Latin America Grants Program, researchers and educators can continue to improve the assessment of health care professionals around the world.

NBME views diversity, equity and inclusion (DEI) as foundational and enduring to our strategy and vision. We continue to focus on ensuring that our DEI work is impactful and ingrained in everything we do, including with our staff, culture, products and services, the Philadelphia community and the broader medical education landscape. Our commitment manifests in our hiring and staff development, recruitment for committees, grants programs, design and review of our assessments, and involvement in our local and national communities.

Learn more about NBME at NBME.org.

#LI-Remote

#Senior