Senior Data Scientist (Psychometrics)

About us

At Udemy, we’re on a mission to transform lives through learning. Through our intelligent skills platform and a global community of instructors, we’ve helped over 70 million learners and 16,000 organizations achieve their goals. Come join us in ensuring everyone, everywhere has access to the skills they need to unlock their potential and create possibilities for themselves and others.

About you 

You’re an analytical problem-solver ready to put your skills toward purposeful work that has a global impact. You want to lead the way in innovation, exploring the latest technologies and finding new solutions. You thrive in a collaborative environment and are eager to work with and learn alongside the best in Product, Design, and Engineering.

About this role 

In the Data organization at Udemy, we’re passionate about transforming the future of education using data!  Psychometrics plays an important role in Udemy’s growing skills validation strategy. We’re looking for fun, collaborative, self-motivated psychometricians with an insatiable sense of curiosity and a knack for asking the right questions.

Our ideal candidate is an analytically creative thinker who is unafraid of ambiguity, has a track record of tackling difficult technical challenges, and has a pragmatic and relentless focus on driving meaningful business impact. 

We believe in building diverse teams, collaborative teams in which each person brings unique skills, knowledge, and experience to the table. If this opportunity sounds exciting to you, then read on!

As a psychometrician at Udemy, you will be partnering with product managers, designers, and engineers to build algorithmic and model-driven solutions to power skill assessments. This role will require a combination of applied measurement experience, data-driven decision making, and the data science and coding chops to dive headfirst into development on production systems. A successful candidate will have knowledge of best industry practices for growing and maintaining a large set of assessment products, while being courageously experimental in finding new solutions to ongoing issues in test development and maintenance.

What you'll be doing

• Collaborate with other psychometricians, data scientists, PMs, designers, and engineers to maintain and grow our skill assessment platform
• Effectively communicate and present to both technical and non-technical audiences
• Critically evaluate and implement the state-of-the-art research literature in educational assessment
• Develop production-grade data pipelines and continuously improve the quality and accessibility of data
• Develop psychometric models for skill assessment through all phases, including prototyping, productionization, and experimental evaluation
• Define metrics and build dashboards to track the adoption and health of skill assessments
• Collaborate with content developers and instructional designers to develop and refine item writing and review processes

What you’ll have

• 3+ years of hands-on industry experience in a psychometric or data science-oriented role
• Domain experience in educational assessment and psychometrics (e.g., Classical Test Theory (CTT), Item Response Theory (IRT), Computerized Adaptive Testing (CAT), equating and scaling) 
• Experience creating or applying novel measurement models (Diagnostic Classification, Knowledge Tracing, MIRT) and their associated estimation algorithms
• Strong SQL ability and proficiency with 1+ programming languages (e.g., R or Python)
• End-to-end experience organizing and analyzing large data sets for operational purposes
• Ability to communicate effectively with non-technical stakeholders
• Strong knowledge of applied statistics and experimental design
• Ability to work autonomously, relentlessly prioritize, and collaborate with team members across time zones

We understand that not everyone will match each of the above qualifications. However, we also realize that everyone has unique experiences that can add value to our company. Even if you think your background might not perfectly align, we'd love to hear from you!

We Are More Excited About You Because You Have:

• 5+ years of relevant industry experience
• Experience with machine learning and causal inference
• Familiarity with standard software engineering practices (e.g., Scrum)
• Experience using big data technologies (e.g., Hadoop, Spark)

Life at Udemy

We aspire to be as vibrant and dynamic as the communities we serve, as inquisitive as those who use our platform, and as revolutionary as the future we strive to open for everyone. Here are some of the things we love about life at Udemy:

We’re invested in creating an inclusive environment that welcomes a diverse range of backgrounds and experiences. From creating employee resource groups, ensuring we’re a Fair Pay Workplace, and building a flexible work culture, our belonging, equity, diversity, and inclusion (BEDI) initiatives always put our people first. We want you to be able to bring your authentic self to work because when we all do, we’re better for it.

Learning is what we do – inside and out. Our Learning & Development team is second to none, helping ensure your journey is one of continuous progression. You’ll also have unlimited access to Udemy courses, monthly UDays (meeting-free professional development days), and a generous annual professional development stipend.

Our reason to exist is to revolutionize learning – that calls for taking risks and learning from failures. Whether it’s our hackathons (a company-wide effort to envision new possibilities for our product) or sharing our prototypes, we see experimentation as a crucial step on the path to success.

We’re committed to creating world-class employee experiences and are proud of the recognition of this by Great Place to Work. 

Of course, the best thing about being part of Udemy is knowing your work makes a difference for people and organizations around the world. You’ve got the skills; why not use them to help others develop theirs?

At Udemy, we value diversity and inclusion and consider qualified applicants without regard to race, color, religion, sex, national origin, ancestry, age, genetic information, sexual orientation, gender identity, marital or family status, veteran status, medical condition, or disability. We will consider for employment qualified applicants with arrest and conviction records.

Our Benefits Start With U

Our benefits package at Udemy “starts with U” and is grounded in mission-driven benefits that intentionally align with our core values. Here’s a sneak peek at just a few highlighted benefits if you’re a full-time Udemate based in the United States:

Core Benefits: Comprehensive medical, dental, vision, life/AD&D, and disability coverage

HSA/FSA/Commuter: Pre-tax savings/spending plans, including HSA employer contributions

Transgender Benefits: Expanded medical coverage for gender-affirming care

Parental Benefits: Financial assistance for fertility (including egg-freezing), adoption, and surrogacy expenses; twelve weeks of paid family bonding for birth, surrogacy, adoption, and foster placement (plus up to six to ten weeks of paid disability leave for birthing parents) as well as a two-week “ease-back” program that enables parents to return part-time at full pay

Whole U: Fully paid memberships for meditation/mindfulness, therapy/coaching visits, financial planning, fertility and family building, maternity and newborn care, parenting and pediatrics, menopause care, breastmilk shipping, and caregiving support

Learning: Free access to Udemy and Udemy Business plus a yearly stipend to spend on professional development

Retirement: Pre-tax and Roth 401(k) plan with an employer contribution

Vacation: Flexible vacation for salaried Udemates and fifteen days per year for hourly

Holidays: Twelve paid holidays throughout the year

Information regarding data privacy is available within the Udemy Careers Privacy Notice.