IT engineer data lakehouse
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Align designs with governance and metadata standards
- Apply master data management strategies
- Build bronze silver gold lakehouse layers
- Create curated data marts for reporting
- Define data quality rules and reporting
- Design scalable batch and streaming data pipelines
- Develop automated data validation tests
- Develop ingestion from structured and semi structured sources
- Document pipeline logic and data contracts
- Implement dimensional models star schema
- Implement domain driven 3NF data warehouse models
- Implement infrastructure as code with version control and CI/CD
- Monitor pipeline health and data quality
- Optimize cluster resource usage
- Profile and tune data transformation performance
- Refactor legacy pipelines for scalability
- Track data lineage and audit trails
Perks/Benefits
Skills/Tech-stack
3NF | APIs | Audit trails | Azure | Azure Databricks | Batch Processing | CI | CI Pipelines | CI/CD | Cluster optimization | Data Contracts | Data Governance | Data Ingestion | Data Lineage | Data Management | Data Modeling | Data Pipelines | Data Quality | Data Quality Rules | Data Validation | Dimensional Modeling | Domain-Driven Design | Infrastructure as Code | Integration Testing | Lakehouse | Master Data | Master Data Management | Performance Tuning | Power BI | Power Query | PySpark | Quality Rules | SAP | Scala | Slowly Changing Dimensions | Snowflake Schema | Star Schema | Streaming processing | Test-Driven | Test-Driven Development | Unit Testing | Unity Catalog | Version control | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
AWS DE- Gen AI INR 2000K-3300KAWS Glue | AWS Identity | AWS Identity and Access Management | AWS Identity and Access Management IAM | AWS LambdaSenior-level Full TimeIN-TN-Chennai1h ago
-
Software Engineer II INR 1000K-2000KApache Spark | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactoryMid-level Full TimeIndia, Telangana, Hyderabad5h ago
-
Sr. Specialist, Data Engineering INR 2771K-4000KAgile | Airflow | BigQuery | CI/CD | Cloud FunctionsRelocation assistance within countrySenior-level Full TimeMumbai, MH, IN6h ago
-
Senior-level Full TimeTelangana, India7h ago
-
API Integration | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryMid-level Full TimePune, Maharashtra, India8h ago
-
Azure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure SQLMid-level Full TimePune, Maharashtra, India8h ago
-
Azure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimePune, Maharashtra, India8h ago
-
Lead Software Engineer INR 2500K-5000KAWS | Asynchronous programming | Authentication | Authorization | AutogenSenior-level Full TimeRemote, India R10h ago
-
Senior Software Engineer - Ingestion INR 2040K-2800KBackup and Recovery | C++ | Data Capture | Database replication | Distributed SystemsSenior-level Full TimeBengaluru, India12h ago
-
Alteryx | Big Data | Data Mining | Data Modeling | Data VisualizationFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India14h ago
-
.NET | Agile | Azure | Azure Databricks | C#Senior-level Full TimeKolkata DN 57, India14h ago
-
Mid-level Full TimeBangalore, Karnataka, IN India, 56008714h ago
-
Forward Deployment Engineer (AI Solutions)-BK INR 2500K-4500KAI Agents | AWS Bedrock | Amazon Web Services | Angular | Azure OpenAIMid-level Full TimeBengaluru, Karnataka, India14h ago
-
Forward Deployment Engineer (AI Solutions) INR 2500K-5000KAI Agents | AWS Bedrock | Amazon Web Services | Angular | Azure OpenAIMid-level Full TimeBengaluru, Karnataka, India14h ago
-
DE-RCE-Fraud Analytics-Senior-GDSN02 INR 3264K-5000KAWS Glue | AWS Lambda | Amazon EventBridge | Amazon S3 | CI/CDSenior-level Full TimeBengaluru, KA, IN, 56001614h ago
-
Senior-level Full TimePune, MH, IN, 41100514h ago
-
Senior Data Engineer INR 2000K-2500KAutomated testing | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeIN KA Bengaluru, India14h ago
-
TechOps-CloudOps-DataOps-Senior INR 3000K-4000KAutomation | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeBengaluru, KA, IN, 56001614h ago
-
TechOps-CloudOps-DataOps-Staff INR 3000K-4000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Databricks | Azure DevOpsSenior-level Full TimeBengaluru, KA, IN, 56001614h ago
-
Data Platforms SRE INR 1500K-2500KAWS IAM | Access Management | Active Directory | Azure Active Directory | Azure DevOpsEntry-level Full TimeBangalore, India14h ago
-
Senior Software Engineer - Java Fullstack, Gen AI INR 3500K-4000KAgile | Angular | Artifactory | Azure | Azure GCPSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India14h ago
-
Mid-level Full TimeIN, BANGALORE, India14h ago
-
Machine Learning Engineer INR 980K-1800KAPI Design | AWS | Apache Airflow | Apache Spark | AzureDomestic travel 10 percent | Hybrid workMid-level Full TimeIN, BANGALORE, India14h ago
-
Senior DevOps Engineer INR 3000K-4000KAccess Management | Azure | Azure Data | Azure Data Factory | Azure DatabricksSenior-level Full TimePune, MH, IN, 41100514h ago
-
Software Development Engineer in Test (SDET) INR 2300K-4200KAWS | Azure | CI/CD | Cloud platform | CypressSenior-level Full TimeBengaluru, Karnataka, India14h ago