Data Engineering Manager

Job Description Summary

We are looking for a skilled and enthusiastic Data Engineer with expertise and hands-on experience in any of the ETL tools, Databricks, Snowflake SQL, and PySpark to join our innovative team.
As a Data Engineer, you will be responsible for designing, implementing, and optimizing scalable data pipelines, ensuring data quality, and building robust data infrastructure.
You will collaborate closely with data scientists, domain experts, and other stakeholders to ensure the efficient and reliable flow of data and analytics across the organization.

 

Job Description

Position Title : Data Engineering manager

Location – Hyd |India| #LI Hybrid

About the role

We are looking for a skilled and enthusiastic Data Engineer with expertise in any of the ETL tools, Databricks, Snowflake SQL, and PySpark to join our innovative team.

As a Data Engineer, you will be responsible for designing, implementing, and optimizing scalable data pipelines, ensuring data quality, and building robust data infrastructure.

You will collaborate closely with data scientists, domain experts, and other stakeholders to ensure the efficient and reliable flow of data across the organization.

Your responsibility includes but not limited to:

• ETL Development: Design, implement, and maintain scalable ETL pipelines to extract, transform, and load data from various sources into data warehouses or data lakes.
• Databricks Utilization: Leverage Databricks to develop, optimize, and scale data engineering workflows.
• Data Modelling: Design and implement robust data models to support analytics and reporting needs.
• SQL Proficiency: Write, optimize, and maintain complex SQL queries for data extraction and transformation tasks.
• PySpark Development: Utilize PySpark for big data processing tasks, developing scalable and efficient solutions.
• Data Quality and Validation: Implement data quality checks and validation processes to ensure data accuracy and consistency.
• Data Integration: Integrate data from multiple sources, ensuring data consistency and reliability.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.

Essential Requirements:

• Documentation: Create and maintain comprehensive documentation for data pipelines, data models, and related processes. Performance Optimization: Optimize data processing performance by fine-tuning ETL pipelines and data storage solutions.
• Security and Compliance: Ensure compliance with data governance policies and security protocols to protect sensitive and confidential information.
• Continuous Improvement: Stay current with industry trends and emerging technologies, continuously improving data engineering practices.
• Strong problem-solving skills and attention to detail. Excellent communication and collaboration skills with senior stakeholders
• Experience with cloud platforms (e.g., AWS, Google Cloud, Azure) and their data services (Databricks). Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
• Knowledge of real-time data processing and streaming technologies.
• Proficiency in data visualization tools (e.g., Tableau, Power BI). Experience with DevOps practices and tools for CI/CD pipelines.

Desirable:

• Bachelor's or master’s degree in computer science, Engineering, Information Systems, or a related field.
• 6+ years’ experience in a Global company as a data steward, engineer, modeler or data scientist, with a strong focus on ETL tools, SQL, and PySpark.
• Business understanding of pharmaceutical industry and data standards. Domain experience in at least one of the following areas – a) Pharma R&D, b) Manufacturing, Procurement and Supply Chain and c) Marketing and Sales. Experience in working in Pharma / Life Science industry strongly preferred
• Strong proficiency in SQL and experience with database management systems (e.g., PostgreSQL, MySQL, Oracle). Hands-on experience with Databricks for developing and optimizing data workloads.
• Proficiency in PySpark for big data processing tasks.Knowledge of data warehousing solutions (e.g. Snowflake). Experience of working with large codebase / repos using Git / Bitbucket

Why Novartis

Our purpose is to reimagine medicine to improve and extend people’s lives and our vision is to become the most valued and trusted medicines company in the world. How can we achieve this? With our people. It is our associates that drive us each day to reach our ambitions. Be a part of this mission and join us!

Learn more here: https://www.novartis.com/about/strategy/people-and-culture

You’ll receive: You can find everything you need to know about our benefits and rewards in the Novartis Life

Handbook.

https://www.novartis.com/careers/benefits-rewards,

Join our Novartis Network: If this role is not suitable to your experience or career goals but you wish to stay connected to hear more about Novartis and our

career opportunities, Join the Novartis Network here:

https://talentnetwork.novartis.com/network

Commitment to Diversity & Inclusion:

Novartis embraces diversity, equal opportunity, and inclusion. We are committed to building diverse teams, representative of the patients and communities we serve, and we strive to create an inclusive workplace that cultivates bold innovation through collaboration and empowers our people to unleash their full potential.

Join our Novartis Network: If this role is not suitable to your experience or career goals but you wish to stay connected to learn more about Novartis and our career opportunities, join the Novartis Network here:

 

Skills Desired

Agility, Analytical Thinking, Brand Awareness, Building Construction, Business Analytics, Cross-Functional Collaboration, Digital Marketing, Marketing Strategy, Media Campaigns, Sales, Stakeholder Engagement, Stakeholder Management, Strategic Marketing, Waterfall Model