Internal Audit - Data Engineer III
Tasks
- Automate ingestion with infrastructure as code and CI CD
- Create data quality checks anomaly detection and alerting
- Create maintain medallion data models bronze silver gold
- Design build maintain data pipelines
- Develop optimize ETL/ELT workflows
- Document data lineage validation reconciliation test coverage
- Establish metadata management schema validation versioning
- Implement Azure data lake architecture governance security cost
- Implement CDC incremental loads schema evolution
- Optimize performance and cost in Databricks
- Perform code reviews and promote engineering best practices
- Support audit execution with governed analytical datasets
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | Airflow | Avro | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | CDC | CI/CD | Clustering | Data Factory | Data Governance | Data Lake Storage | Data Lineage | Data Modeling | Data Quality | Data Reconciliation | Databricks | Delta Lake | Delta Live Tables | Delta Live) | Git | Incremental loading | Infrastructure as Code | JSON | Parquet | Partitioning | PySpark | Python | SQL | Schema Validation | Schema evolution | Semantic Modeling | Spark | Spark SQL | Structured Streaming | Time Travel | Unity Catalog | Z-Order | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Fullstack Data Engineer (Remote) INR 800K-2000KAPI Keys | AWS IAM | AWS S3 | AWS VPC | Apache SparkRemote work | UK shift scheduleSenior-level Full TimeMaharashtra, Pune, India R6h ago
-
Principal Consultant - GenAI Engineer INR 3000K-4144KAWS | Agentic AI | Azure | Azure OpenAI | Cloud ComputingSenior-level Full TimeIndia-Pune7h ago
-
Principal Consultant - GenAI Engineer INR 3000K-4725KAWS | Agentic AI | Azure OpenAI | Cloud platform | Deep learningSenior-level Full TimeIndia-Kolkata7h ago
-
Senior DevOps Engineer (AWS | MLOps | Kubernetes) INR 2500K-4000KAWS IAM | Amazon EKS | Amazon S3 | Amazon Web Services | Argo CDHybrid workSenior-level Full TimeMaharashtra, Pune, India7h ago
-
GenAI Azure INR 1800K-2500K.NET | API Management | ASP.Net Core | Access Control | Active DirectoryMid-level Full TimeIN-AP-Hyderabad7h ago
-
Senior-level Full TimeHyderabad, India8h ago
-
Staff Software Engineer, Cloud SQL Postgres INR 1500K-2400KC++ | Database Administration | PostgreSQL | Query Optimization | Relational databasesSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Actuarial - Data Science INR 1500K-3000KClassification | Data Preprocessing | Data Quality | Explainable AI | Feature EngineeringMid-level Full TimeBangalore, KA, India9h ago
-
Data Engineer II - Python, AWS & Pyspark INR 1500K-2200KAWS | Data Migration | Data Modeling | Data Warehouse | Data pipelineMid-level Full TimeHyderabad, Telangana, India11h ago
-
AWS | Alerting | Audit Logging | Azure | CI/CDSenior-level Full TimePune, MH, India11h ago
-
Consultant - Gen AI Engineer INR 550K-610KAPI Integration | Generative AI | Language Processing | Machine Learning | Natural LanguageEntry-level Full TimeBangalore, Karnataka, India12h ago
-
Apache Iceberg | Apache Spark | Avro | CI/CD | Cause analysisEntry-level Full TimeHyderabad, Telangana, India12h ago
-
Engineering-L2-Bengaluru-Vice President-Software Engineering INR 1500K-6000KApache Iceberg | Apache Spark | Avro | CI/CD | DatabricksExecutive-level Full TimeBengaluru, Karnataka, India13h ago
-
Software Engineer III - Data Engineer, AWS, ETL, Python INR 2000K-3500KAWS | CI/CD | Data Governance | Data Lake | Data ProcessingSenior-level Full TimeBengaluru, Karnataka, India13h ago
-
Gen AI Lead engineer INR 3000K-4800KAgentic Frameworks | CI/CD | Data Visualization | Git | LangchainSenior-level Full TimeChennai, Tamil Nadu, India13h ago
-
Engineering-L2-Bengaluru-Associate-Software Engineering INR 1800K-3200KANSI SQL | Apache Iceberg | Apache Spark | Avro | CI/CDSenior-level Full TimeBengaluru, Karnataka, India13h ago
-
Full Stack Developer :: Acon INR 2500K-3500KAI Search | AI Studio | Application Security | Arize Phoenix | Azure AISenior-level Full TimeIndia13h ago
-
Lead Assistant Manager INR 2500K-4500KASR | AWS EC2 | Amazon ECS | Amazon SageMaker | Amazon Web ServicesSenior-level Full TimeNoida, Uttar Pradesh, India14h ago
-
Data Engineer INR 2755K-3500KAPI Development | AWS | Amazon Redshift | Azure | BashHybrid working | Inclusive culture | Learning and development programmesSenior-level Full TimeHyderabad, India14h ago
-
Director of Software Engineering - Big Data, Cloud INR 2000K-6000KAI machine learning | AWS | Agile | Azure | Big DataExecutive-level Full TimeMumbai, Maharashtra, India14h ago
-
Assistant Manager-GDC INR 1500K-2000KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageMid-level Full TimeBangalore, Karnataka, India15h ago
-
Data Engineer- AWS+MS Fabric+ Purview INR 2000K-3500K.NET | AWS CloudFormation | AWS Lambda | Amazon EC2 | Amazon RDSSenior-level Full TimePune, Maharashtra, India16h ago
-
Software Engineer III - Databricks, SQL , Python INR 2000K-2800KApache Spark | Cloud technologies | Data Lake | Data Warehouse | DatabricksSenior-level Full TimeBengaluru, Karnataka, India16h ago
-
Senior-level Full TimeBangalore, Karnataka, India17h ago
-
Senior-level Full TimeBangalore, India19h ago