Data Engineering Lead

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. 

 Data Engineering Lead

Data Engineering Lead is responsible for collaborating with the Data Architect to design and implement scalable data lake architecture and data pipelines

Position Summary   

•     Design and implement scalable data lake architectures using Azure Data Lake services.
•     Develop and maintain data pipelines to ingest data from various sources.
•    Optimize data storage and retrieval processes for efficiency and performance.
•    Ensure data security and compliance with industry standards.
•    Collaborate with data scientists and analysts to facilitate data accessibility.
•    Monitor and troubleshoot data pipeline issues to ensure reliability.
•    Document data lake designs, processes, and best practices.
•    Experience with SQL and NoSQL databases, as well as familiarity with big data file formats like Parquet and Avro.

Roles and Responsibilities

        Must Have Skills

       Essential  

•  Azure Data Lake
•  Azure Synapse Analytics
•  Azure Data Factory
•  Azure DataBricks
•  Python (PySpark, Numpy etc)
• SQL
• ETL
•  Data warehousing
• Azure Devops
•  Experience in developing streaming pipeline using Azure Event Hub, Azure Stream analytics, Spark streaming
•  Experience in integration with business intelligence tools such as Power BI

Good to have skills

• Big Data technologies (e.g., Hadoop, Spark)
•  Data security

General Skills

•  Experience with Agile and DevOps methodologies and the software development lifecycle.
•  Proactive and responsible for deliverables
•  Escalates dependencies and risks
•  Works with most DevOps tools, with limited supervision
• Completion of assigned tasks on time and regular status reporting
• Should be able to train new team members
• Desired to have knowledge on any of the cloud solutions such as Azure or AWS with DevOps/Cloud certifications.
• Should be able to work with a multi culture global teams and team virtually
• Should be able to build strong relationship with project stakeholders

EY | Building a better working world 

 
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.  

 
Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.  

 
Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.