Internal Audit - Lead Data Engineer
Tasks
- Build ETL/ELT pipelines
- Build lakehouse bronze silver gold models
- Coordinate hiring and onboarding enablement
- Coordinate offshore and onshore execution
- Define testing and CI/CD standards
- Establish data quality monitoring and alerting
- Implement CDC incremental loads and full refresh
- Implement ingestion patterns including streaming
- Implement performance and cost optimization
- Integrate SAP ECC and SAP S4HANA data
- Lead architecture design reviews
- Lead end to end delivery planning
- Maintain lineage and audit evidence
- Manage metadata and schema validation frameworks
- Manage schema evolution and reconciliation
- Mentor engineers on Databricks engineering practices
- Own data engineering roadmap
- Publish governed datasets for BI
- Serve as internal audit data engineering SME
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Azure Data | Azure Data Factory | CDC | CI/CD | Clustering | Data Factory | Data Governance | Data Lakehouse | Data Modeling | Data Quality | Databricks | Databricks Jobs | Delta Lake | Delta Lake Merge | ELT | ETL | Lineage | Metadata Management | Monitoring and Alerting | Partitioning | PySpark | Python | Runbooks | SAP ECC | SAP S4HANA | SQL | Schema evolution | Spark SQL | Structured Streaming | Version control | Z-Order
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
ESI Digi Tech ATLAS 2 168203 INR 2500K-4500KAWS Lambda | AWS Step Functions | Amazon Simple Notification Service | Amazon Simple Queue Service | Amazon Web ServicesSenior-level Full TimeHyderabad, IND-TG, India5h ago
-
Senior-level Full TimeHyderabad, IND-TG, India5h ago
-
ESI Central BI 166816 INR 1500K-2146KAWS Redshift | Amazon Web Services | DBT | Data Lake | Data ModelingSenior-level Full TimeHyderabad, IND-TG, India5h ago
-
Databrick Solution Architect INR 800K-1500KBatch Processing | CD pipelines | CI/CD | CI/CD pipelines | Cluster managementSenior-level Full TimeCoimbatore, India9h ago
-
Lead Backend Engineer (MongoDB / NoSQL) INR 1600K-4000KAWS | Agile | Azure | CI/CD | Distributed SystemsSenior-level Full TimeIN-TN-Chennai10h ago
-
Senior-level Full TimeBengaluru, KA, India11h ago
-
Lead Data Scientist INR 2500K-2829KArtificial Intelligence | Classification | Cloud Computing | Clustering | ETLEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeVadodara, GJ, India11h ago
-
Software Developer Lead (AWS, Java/Python) INR 2695K-4000KAWS | AWS CloudFormation | Amazon S3 | Amazon SNS | Amazon SQSSenior-level Full TimeHyderabad, India11h ago
-
Lead Data Scientist INR 2516K-3356KArtificial Intelligence | Big Data | Classification | Cloud Computing | ClusteringEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeVadodara, GJ, India11h ago
-
Lead AI Engineer #3008 INR 2500K-4500KAgent Orchestration | Azure | Deepgram | Fine Tuning | LLM ObservabilityFlexible working hours | Global exposure | Professional development | Rewards and recognition | Work-life balanceSenior-level Full TimeChennai, India12h ago
-
Analytics Solution- Associate INR 2040K-2755KAgile | Atlassian | Atoti | Change Data Capture | Continuous DeploymentSenior-level Full TimeBengaluru, Karnataka, India13h ago
-
Senior-level Full TimePune, Maharashtra, India13h ago
-
Senior Manager, Data Engineering Service & SRE INR 2000K-3500KAnomaly Detection | Apache Airflow | Automated testing | BigQuery | CloudSenior-level Full TimeBengaluru, IN, 56010314h ago
-
Mid-level Full TimePune, Maharashtra, India14h ago
-
Mid-level Full TimeBengaluru, Karnataka, India15h ago
-
AWS | AWS Glue | Advanced SQL | Airflow | Amazon EMRMid-level Full TimeHyderabad, Telangana, India15h ago
-
Lead Software Engineer - Java, Big Data, Hybrid cloud INR 2050K-4000KAPIs | AWS | Agile | Apache Spark | Application ResiliencySenior-level Full TimeBengaluru, Karnataka, India R16h ago
-
Cloud Data Engineer INR 2000K-2755KAWS | AWS Athena | AWS Glue | AWS Lake Formation | AWS LambdaAgile team | Global collaboration | On-call rotationSenior-level Full TimePune, India16h ago
-
B2B SaaS | Dashboards | Data Pipelines | Data analytics | ETLCompany equipment | Fully remote work | Health coverage | Home office stipend | Inclusive collaborative cultureMid-level Full TimeIndia R16h ago
-
GenAI Engineer - Database INR 2520K-3880KData Governance | Data Lineage | Embeddings | Entity Resolution | Graph DatabasesEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimePune, MH, India17h ago
-
Deputy Manager, Analytics & Insights INR 1244K-1680KARR | Churn | DBT | Data Modeling | Data VisualizationEducation reimbursement | Health plans | Hybrid work | Parental leave | Retirement optionsMid-level Full TimeMohali, IND17h ago
-
GenAI Lead INR 2500K-4500KAzure OpenAI | Data Governance | Embeddings | Generative AI | LangchainHybrid workSenior-level Full TimeBengaluru, India17h ago
-
Principal I, Data Engineering INR 2000K-2040KApache Kafka | Apache Spark | Azure Data | Azure Data Factory | CI/CDSenior-level Full TimeBangalore, KA, India18h ago
-
Software Engineering Manager INR 3384K-5000KAI-assisted software development | AWS | Backend Development | Cloud Native | Code QualityCross-functional team projects | Learning Campus trainingSenior-level Full TimeMumbai, India18h ago
-
Sr Quality Engineer (SD/HD map, Spatial Data Engineering) INR 1800K-2400KAPI | AWS | Anomaly Detection | ArcGIS | ClassificationCross-team collaboration | Inclusive work environment | Mentorship | Professional developmentSenior-level Full TimeMumbai, India18h ago