AWS Certification Psychometrician

As cloud technologies continue to transform businesses, skilled individuals are in high demand. At AWS Training and Certification (T&C), we are passionate about revolutionizing the way people advance their cloud skills and careers. We equip diverse builders of today and tomorrow with the knowledge they need to leverage the power of the AWS Cloud. Join our dynamic, fast-growing team and help us empower our customers to build cloud skills.

The AWS Certification team is seeking a Psychometrician with experience working with criterion-referenced assessment programs to support a large global AWS Certification program. In this role, you will support all psychometric aspects of exam development and operation, including job analyses, standard setting, automated test assembly, item and test analyses, optimal item bank design, quality assurance, and project planning.

You will work closely with a team of psychometricians, subject matter experts, certification exam program managers, publishing, delivery, security, product management, and translation/localization teams to support ongoing analyses of exam data. To be successful in this position, you must be highly motivated, creative, detail oriented, and a self-starter who is able to think big, execute, ensure high quality, yet stay focused on the details.

Key job responsibilities
- Conduct Job Task Analysis (JTA) workshops and post-JTA survey analyses to define the blueprint and test specifications for new certifications or updates to existing certifications
- Conduct standard setting studies to set the passing score for an exam
- Run item analysis to evaluate quality and performance of exam items
- Use automated test assembly procedures to assemble forms or item pools
- Work with content development to optimize the health of item banks
- Support the development of a cloud-based analytics and reporting system
- Interpret and clearly communicate the results of analyses to stakeholders through written and oral reports
- Follow the accreditation standards set by ISO/IEC:2012 17024 and the National Council for Certifying Agencies (NCCA) as they relate to valid psychometric practices
- Contribute to the development and execution of the strategic goals regarding the AWS certification program
- Consult with leadership, internal staff, external consultants, and industry leaders regarding advancement of current offerings

Basic Qualifications

- PhD in Psychometrics, Quantitative Psychology, Educational measurement, or I/O Psychology
- 3+ years’ post-doctoral direct experiences working with high stakes certification or licensure programs
- Experience conducting job task analysis and standard setting
- Experience in equating, item analysis, and form assembly
- Experience using item response theory
- Proficiency in R or Python languages

Preferred Qualifications

- Experience in testing for global audiences
- History of conference presentations and publications
- Experience with different exam designs and delivery (e.g. LOFT, CAT)
- Experience using AI methods (e.g., Generative AI, machine learning)
- Experience with data forensics methods

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $136,000/year in our lowest geographic market up to $212,800/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.