Senior Engineer (SA1 DES - Data Bricks, Data engineer AZURE)

Roles & responsibilities

Role Overview: The Senior Associate 1 - “Azure Data Engineer with Gen AI/Fabric skills” will be part of the GDC Technology Solutions (GTS) team, working in a technical role in the Audit Data & Analytics domain that requires developing expertise in KPMG proprietary D&A (Data and analytics)) tools and audit methodology. He/she will be a part of the team responsible for extracting and processing datasets from client ERP systems (SAP/Oracle/Microsoft Dynamics) or other sources to provide insights through data warehousing, ETL and dashboarding solutions to Audit/internal teams and be involved in developing solutions using a variety of tools & technologies

The Senior Associate 1 - “Azure Data Engineer” will be predominantly responsible for:

Data Engineering (Primary Skills)

·Azure data engineer expertise having experience working on all major Azure resources including expertise in development and debugging of programs using Databricks expertise. ·Utilize Azure Databricks notebooks to build and manage data transformations, create tables, and ensure data quality and consistency. Leverage Unity Catalog for data governance and maintaining a unified data view across the organization. ·Design, develop, and maintain scalable and efficient data pipelines to process large datasets from various sources using Azure Data Factory (ADF). ·Integrate data from multiple data sources and ensure data consistency, quality, and accuracy, leveraging Azure Data Lake Storage (ADLS). ·Develop and manage data warehouses to store and organize large volumes of structured and unstructured data using Azure Synapse Analytics and other relevant Azure services. ·Design and implement ETL (Extract, Transform, Load) processes to ensure seamless data flow across systems using Azure cloud platform ·Work experience on Microsoft Fabric experience is an added advantage ·Professional should be enthusiastic to learn, adapt and integrate Gen AI into the business process and should have experience working in Azure AI services ·Optimize data storage and retrieval processes to enhance system performance and reduce latency.

Execution

·Development of transformations using Azure Databricks, Azure Data Factory or Python. Able to handle any data mapping changes and customizations within Databricks using Pyspark code. ·Debug, optimize and performance tune and resolve issues, if any, with limited guidance, when processing large data sets and propose possible solutions ·Assist Audit engagement teams in data extraction and data transformation process by coordinating with the Client’s IT teams and other technical leads ·Handle and analyze enormous volumes of data using big data technologies like Azure Databricks and Apache Spark. In order to support data analytics, machine learning, and other data-driven applications, they need to create data processing workflows and pipelines. ·Able to integrate Azure Databricks with ERP systems or third part systems using APIs and build Python or Pyspark notebooks to apply business transformation logic as per the common data model ·Must have experience in concepts like Partitioning, optimization, and performance tuning for improving the performance of the process ·Experience working with General Ledger / Sub-Ledger analysis and developed risk assessment / substantive routines for Audit/Internal Audit (preferably from Big 4). ·Good to have knowledge on KQL (Kusto Query Language), Azure REST APIs ·Work with Audit engagement teams to interpret the results and provide meaningful audit insights from the reports ·Maintaining the accurate and up to date project status for self and any assigned team members ·Prepare and review required documents supporting the engagement with utmost attention to details ·Coach juniors on data processing best practices and knowledge transfer of low complexity engagements ·Enthusiastic to learn and use Azure AI services in business processes.

Work experience working on Unified data applications using Microsoft Fabric is an added advantage

Roles & responsibilities

Role Overview: The Senior Associate 1 - “Azure Data Engineer with Gen AI/Fabric skills” will be part of the GDC Technology Solutions (GTS) team, working in a technical role in the Audit Data & Analytics domain that requires developing expertise in KPMG proprietary D&A (Data and analytics)) tools and audit methodology. He/she will be a part of the team responsible for extracting and processing datasets from client ERP systems (SAP/Oracle/Microsoft Dynamics) or other sources to provide insights through data warehousing, ETL and dashboarding solutions to Audit/internal teams and be involved in developing solutions using a variety of tools & technologies

The Senior Associate 1 - “Azure Data Engineer” will be predominantly responsible for:

Data Engineering (Primary Skills)

·Azure data engineer expertise having experience working on all major Azure resources including expertise in development and debugging of programs using Databricks expertise. ·Utilize Azure Databricks notebooks to build and manage data transformations, create tables, and ensure data quality and consistency. Leverage Unity Catalog for data governance and maintaining a unified data view across the organization. ·Design, develop, and maintain scalable and efficient data pipelines to process large datasets from various sources using Azure Data Factory (ADF). ·Integrate data from multiple data sources and ensure data consistency, quality, and accuracy, leveraging Azure Data Lake Storage (ADLS). ·Develop and manage data warehouses to store and organize large volumes of structured and unstructured data using Azure Synapse Analytics and other relevant Azure services. ·Design and implement ETL (Extract, Transform, Load) processes to ensure seamless data flow across systems using Azure cloud platform ·Work experience on Microsoft Fabric experience is an added advantage ·Professional should be enthusiastic to learn, adapt and integrate Gen AI into the business process and should have experience working in Azure AI services ·Optimize data storage and retrieval processes to enhance system performance and reduce latency.

Execution

·Development of transformations using Azure Databricks, Azure Data Factory or Python. Able to handle any data mapping changes and customizations within Databricks using Pyspark code. ·Debug, optimize and performance tune and resolve issues, if any, with limited guidance, when processing large data sets and propose possible solutions ·Assist Audit engagement teams in data extraction and data transformation process by coordinating with the Client’s IT teams and other technical leads ·Handle and analyze enormous volumes of data using big data technologies like Azure Databricks and Apache Spark. In order to support data analytics, machine learning, and other data-driven applications, they need to create data processing workflows and pipelines. ·Able to integrate Azure Databricks with ERP systems or third part systems using APIs and build Python or Pyspark notebooks to apply business transformation logic as per the common data model ·Must have experience in concepts like Partitioning, optimization, and performance tuning for improving the performance of the process ·Experience working with General Ledger / Sub-Ledger analysis and developed risk assessment / substantive routines for Audit/Internal Audit (preferably from Big 4). ·Good to have knowledge on KQL (Kusto Query Language), Azure REST APIs ·Work with Audit engagement teams to interpret the results and provide meaningful audit insights from the reports ·Maintaining the accurate and up to date project status for self and any assigned team members ·Prepare and review required documents supporting the engagement with utmost attention to details ·Coach juniors on data processing best practices and knowledge transfer of low complexity engagements ·Enthusiastic to learn and use Azure AI services in business processes.

Work experience working on Unified data applications using Microsoft Fabric is an added advantage

Job Requirements

Education Requirements

B. Tech/B.E/MCA (Computer Science / Information Technology) 

Technical Skills

Primary Skills:

ØMinimum 4-6 years of experience in Data Engineering. ØProficiency in SQL, Python or Pyspark notebooks development. ØStrong knowledge of ETL tools and processes. ØHands-on experience with Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and. ØComprehensive knowledge of Azure cloud services. ØExperience with Databricks notebooks for building transformations and creating tables ØMicrosoft Fabric and Azure AI services is an added advantage

 Enabling Skills

·Excellent analytical, problem solving and troubleshooting abilities ·Critical thinking: able to look at numbers, trends and data and come to new conclusions based on findings ·Attention to detail and good team player ·Quick learning ability and adaptability ·Willingness and capability to deliver within tight timelines ·Effective communication skills ·Flexible to work timings and willingness to work in different projects/technologies ·Collaborate with business stakeholders to understand data requirements and deliver solutions