Senior Data Engineer

Job Description

We are seeking a skilled and experienced Senior Data Engineer for our Data and AI team to contribute to the development of innovative AI-powered projects at The Washington Post. As a key member of our team, you will be responsible for designing, building, and deploying data solutions that power cutting-edge AI technologies, including Generative AI, personalization, and recommendation systems, as well as revenue science applications. This role involves gathering requirements, writing code, testing, deploying, and maintaining software using best practices in development. The engineer will work collaboratively on large-scale projects, contribute to the creation of new features, products, and tools, and mentor junior developers.

Motivation:

• You are passionate about solving complex technical challenges and driving impactful projects that shape the user experience for a news media platform.

• You'll thrive in our team if you're a natural problem-solver, a strong communicator, and a proactive and curious individual who is always looking to learn and grow.

• You'll be motivated by the opportunity to tackle complex technical challenges, lead and influence a team, and collaborate with cross-functional teams to drive user-focused solutions.

• You'll also have the freedom to explore emerging technologies and trends, and to identify new opportunities for growth and innovation.

• If you're passionate about delivering high-quality solutions that meet the needs of our users, we want to hear from you.

Responsibilities:

• Technical Leadership: Lead data engineering efforts using Python, Spark, Flink and other data processing technologies, delivering scalable, secure, and reliable data pipelines, and contribute to the architecture and decision-making processes of large, complex projects.

• Project Planning and Execution: Collaborate on gathering and analyzing project requirements to define technical solutions, and work with cross-functional teams to integrate front-end and back-end solutions, ensuring seamless user experiences.

• Software Development: Write clear, maintainable code, following best practices in version control, code reviews, and testing, and ensure that all software solutions are scalable, secure, and meet the needs of our users.

• Collaboration and Communication: Collaborate with cross-functional teams, including data scientists, product managers, and designers, to identify and prioritize project requirements, and communicate technical solutions to stakeholders.

• Mentorship and Coaching: Mentor and coach junior engineers, offering technical guidance and reviewing code to ensure quality, and contribute to the growth and development of our engineering team.

• Project Management: Collaborate with Product, Design, Newsroom and business stakeholders to manage and optimize project timelines, proactively identifying blockers and risks to ensure timely delivery.

• Technical Expertise: Stay current with industry trends, suggesting improvements to enhance performance, security, and scalability, and apply knowledge of data engineering to drive innovation and improvement in our products and services.

• Documentation and Knowledge Sharing: Contribute to the development of technical documentation and design specifications for team members and share knowledge and expertise with the team to promote a culture of innovation and collaboration.

Requirements

• Education and Experience: Bachelor's degree in Computer Science, Software Engineering, or a related field is acceptable, but a Master's degree is preferred. Equivalent professional experience will also be considered. 3+ years of overall experience in ETL/ELT, Data modeling, and querying and analyzing large datasets using SQL.

• Technical Skills: Extensive experience with ETL development using Python and PySpark. Experience with AWS Data Lakehouse technologies including Redshift, Athena, EMR, and Glue catalog. Strong understanding of Pub/Sub and event-driven architecture. Able to design and deploy DAGs using Apache Airflow. Experience with Apache Iceberg, Postgres, MySQL, DynamoDB, Tableau, and Superset also highly preferred.

• Cloud Infrastructure: Strong proficiency with cloud infrastructure, including experience with AWS services (e.g., EC2, S3, RDS, Lambda, Batch) for building and deploying scalable, reliable, cloud-based applications. Strong skills with Docker and ECS are highly preferred.

• Software Development: Experience with software development best practices, including coding standards, code reviews, testing, and CI/CD pipelines.

• Performance Monitoring: Experience with performance monitoring and observability tools (e.g., Splunk, Datadog) for maintaining application health and performance.

• Leadership and Communication: Proven ability to mentor junior engineers, influence team-wide practices, and communicate technical concepts to both technical and non-technical stakeholders.

• Problem-Solving: Excellent problem-solving skills, with the ability to troubleshoot and debug issues across the stack.

• Autonomy and Decision-Making: Ability to handle complex projects with limited supervision and manage technical decision-making.

Compensation and Benefits

Wherever you are in your life or career, The Washington Post offers comprehensive and inclusive benefits for every step of your journey:

• Competitive medical, dental and vision coverage

• Company-paid pension and 401(k) match

• Three weeks of vacation and up to three weeks of paid sick leave

• Nine paid holidays and two personal days

• 20 weeks paid parental leave for any new parent

• Robust mental health resources

• Backup care and caregiver concierge services

• Gender affirming services

• Pet insurance

• Free Post digital subscription

• Leadership and career development programs

Benefits may vary based on the job, full-time or part-time schedule, location, and collectively bargained status.

The salary range for this position is:

The actual salary within this range will depend on individual skills, experience, and qualifications as they relate to specific job requirements. This position may be eligible for a bonus or incentive program, and a member of the Talent Acquisition team will discuss bonus payment terms and conditions during the interview process.

The Post strives to provide its readers with high-quality, trustworthy news and information while constantly innovating. That mission is best served by a diverse, multi-generational workforce with varied life experiences and perspectives. All cultures and backgrounds are welcomed. 

The innovation doesn’t end in the Newsroom – dozens of teams power The Washington Post.  We are now hiring the next innovator – how will you Impact Tomorrow?

#washpostlife