Senior Analyst, Product Experimentation

Company Description

We create world-class content, which we distribute across our portfolio of film, television, and streaming, and bring to life through our theme parks and consumer experiences. We own and operate leading entertainment and news brands, including NBC, NBC News, MSNBC, CNBC, NBC Sports, Telemundo, NBC Local Stations, Bravo, USA Network, and Peacock, our premium ad-supported streaming service. We produce and distribute premier filmed entertainment and programming through Universal Filmed Entertainment Group and Universal Studio Group, and have world-renowned theme parks and attractions through Universal Destinations & Experiences. NBCUniversal is a subsidiary of Comcast Corporation.

Comcast NBCUniversal has announced its intent to create a new publicly traded company ('SpinCo') comprised of most of NBCUniversal's cable television networks, including USA Network, CNBC, MSNBC, Oxygen, E!, SYFY and Golf Channel along with complementary digital assets Fandango, Rotten Tomatoes, GolfNow, GolfPass, and SportsEngine. The well-capitalized company will have significant scale as a pure-play set of assets anchored by leading news, sports and entertainment content. The spin-off is expected to be completed during 2025.

As a company uniquely positioned to educate, entertain and empower through our platforms, Comcast NBCUniversal stands for including everyone. Our Diversity, Equity and Inclusion initiatives, coupled with our Corporate Social Responsibility work, is informed by our employees, audiences, park guests and the communities in which we live. We strive to foster a diverse, equitable and inclusive culture where our employees feel supported, embraced and heard. Together, we'll continue to create and deliver content that reflects the current and ever-changing face of the world.

Job Description

We are seeking a Senior Analyst of experimentation to help support our A/B/N testing efforts across multiple Product teams (Viewing, Commerce, Personalization, and Editorial). With the vast amount of content available from NBCUniversal and the ever-evolving and fast-paced product development of Peacock, we will need a smart and capable individual to support the A/B and multivariate testing lifecycle, spanning from sifting through large amounts of data for insights into experimentation, generating informed hypotheses and objectives for data-informed prioritization, designing and conducting tests for statistical significance in continuous and iterative optimization, to analyzing testing results and presenting data-driven actions to a multifunctional team of stakeholders in business and technology.

The candidate should have technical experience and statistical skills with statistical modeling (e.g., hypothesis testing, multivariate statistics) and randomized controlled experimental design (e.g., KPIs, bucketing, sample size, power, statistical significance), A/B/N testing toolsets (e.g., Optimizely, Apptimize, Adobe Target, Split), a programming language for data analysis (e.g., SQL, Python, R), and full-stack streaming devices (e.g., Xfinity, tvOS, iOS, Android mobile/TV, Roku, Play Station, Xbox, Amazon Fire TV).

This role will sit within the larger Decision Sciences team. The goal of the Decision Sciences team is to deliver and champion intelligent use of data that drives a best-in-class customer experience and fuels product growth. The Senior Analyst, Product Experimentation, therefore, should be able to thrive in a collaborative working environment by partnering with other Decision Sciences teams, including Data Science, Data Engineering, Data Measurement, and other Peacock business units across products, content and marketing.

Responsibilities include, but are not limited to:

• Driving A/B/N experiments in order to directly impact the company's optimization strategy for user acquisition, conversion, engagement, retention, and continuous CLV (customer lifetime value) growth
• Partnering with Product and Marketing teams, as well as Engineering stakeholders, in order to translate business requirements into experiment designs of robust statistical and technical soundness, and ultimately actionable insights into continuous optimization
• Designing, conducting, and analyzing A/B/N tests administered to treatment groups against control and holdout groups via statistical modeling in order to ensure analytical power and significance
• Performing segmentation analyses and statistical modeling on user attributes, content personalization, marketing attribution to inform further testing opportunities
• Developing reporting dashboards and visualizations in order to streamline the experimentation process and to present results to key stakeholders in an elegant fashion

Qualifications

• BS in statistics, mathematics, analytics, or related field, or equivalent technical work experience
• At least two years of experience in an A/B and multivariate testing role, preferably in digital media or video streaming
• Statistical skills in applying hypothesis testing, calculating critical value, defining p-value, and using z-test, t-test, ANOVA, or chi-square test
• Proficiency of at least one programming language for data analysis (i.e., Python, R) and cloud-based relational SQL database management systems (e.g., Google Cloud Platform BigQuery)
• Hands-on experience setting up tests with A/B/N testing tools (e.g., Optimizely, Apptimize, Adobe Target, Split), or demonstrated mastery of experimental design and practical application of same techniques
• Working knowledge of digital analytics platforms (e.g., Adobe Analytics, Google Analytics), data management platforms (e.g., mParticle), and visualization tools (e.g., Tableau, Domo)
• Experience querying data in data warehouse systems (e.g., AWS, Snowflake, BigQuery)
• Ability to translate statistical concepts and probability theories to a general audience
• A basic understanding of complex machine learning frameworks (e.g., neural network, gradient boosting, etc.) is preferred, but not required

Additional Requirements:

Hybrid: This position has been designated as hybrid, generally contributing from the office a minimum of three days per week.

This position is eligible for company sponsored benefits, including medical, dental and vision insurance, 401(k), paid leave, tuition reimbursement, and a variety of other discounts and perks. Learn more about the benefits offered by NBCUniversal by visiting the Benefits page of the Careers website.

Salary range: $85,000 - $115,000

Additional Information

As part of our selection process, external candidates may be required to attend an in-person interview with an NBCUniversal employee at one of our locations prior to a hiring decision. NBCUniversal's policy is to provide equal employment opportunities to all applicants and employees without regard to race, color, religion, creed, gender, gender identity or expression, age, national origin or ancestry, citizenship, disability, sexual orientation, marital status, pregnancy, veteran status, membership in the uniformed services, genetic information, or any other basis protected by applicable law.

If you are a qualified individual with a disability or a disabled veteran and require support throughout the application and/or recruitment process as a result of your disability, you have the right to request a reasonable accommodation. You can submit your request to AccessibilitySupport@nbcuni.com.