Data Engineer-Airflow, Python,Pyspark, Databricks

Job Description

Data Engineer, Chennai

We’re seeking a highly motivated Data Engineer to join our agile, cross-functional team and drive end-to-end data pipeline development in a cloud-native, big data ecosystem. You’ll leverage ETL/ELT best practices and data lakehouse paradigms to deliver scalable solutions. Proficiency in SQL, Python, Spark, and modern data orchestration tools (e.g. Airflow) is essential, along with experience in CI/CD, DevOps, and containerized environments like Docker and Kubernetes. This is your opportunity to make an impact in a fast-paced, data-driven culture.

Responsibilities

• Responsible for data pipeline development and maintenance.
• Contribute to development, maintenance, testing strategy, design discussions, and operations of the team.
• Participate in all aspects of agile software development including design, implementation, and deployment.
• Responsible for the end-to-end lifecycle of new product features / components.
• Ensuring application performance, uptime, and scale, maintaining high standards of code quality and thoughtful application design.
• Work with a small, cross-functional team on products and features to drive growth.
• Learning new tools, languages, workflows, and philosophies to grow.
• Research and suggest new technologies for boosting the product.
• Have an impact on product development by making important technical decisions, influencing the system architecture, development practices and more.

Qualifications

• Excellent team player with strong communication skills.
• B.Sc. in Computer Sciences or similar.
• 3-5 years of experience in Data Pipeline development.
• 3-5 years of experience in PySpark / Databricks.
• 3-5 years of experience in Python / Airflow.
• Knowledge of OOP and design patterns.
• Knowledge of server-side technologies such as Java, Spring
• Experience with Docker containers, Kubernetes and Cloud environments
• Expertise in testing methodologies (Unit-testing, TDD, mocking).
• Fluent with large scale SQL databases.
• Good problem-solving and analysis abilities.

Requirements - Advantage

• Experience with Azure cloud services.
• Experience with Agile Development methodologies.
• Experience with Git.

Additional Information

Our Benefits

• Flexible working environment
• Volunteer time off
• LinkedIn Learning
• Employee-Assistance-Program (EAP)

About NIQ

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View™. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population.

For more information, visit NIQ.com

Want to keep up with our latest updates?

Follow us on: LinkedIn | Instagram | Twitter | Facebook

Our commitment to Diversity, Equity, and Inclusion

NIQ is committed to reflecting the diversity of the clients, communities, and markets we measure within our own workforce. We exist to count everyone and are on a mission to systematically embed inclusion and diversity into all aspects of our workforce, measurement, and products. We enthusiastically invite candidates who share that mission to join us. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability status, age, marital status, protected veteran status or any other protected class. Our global non-discrimination policy covers these protected classes in every market in which we do business worldwide. Learn more about how we are driving diversity and inclusion in everything we do by visiting the NIQ News Center: https://nielseniq.com/global/en/news-center/diversity-inclusion