Senior Data Engineer

Tasks

• Alert on pipeline failures
• Architect data lakehouse
• Architect data warehouse
• Build data pipelines
• Catalog data assets
• Design data pipelines
• Develop ELT processes
• Develop ETL processes
• Enforce data security policies
• Implement data cleansing
• Implement data transformation
• Implement data validation
• Ingest structured data
• Ingest unstructured data
• Manage data lineage
• Monitor data pipelines
• Optimize data pipelines
• Optimize for cost efficiency
• Optimize for scalability
• Track system performance with monitoring tools
• Troubleshoot pipeline issues
• Tune pipeline performance
• Tune query performance

Perks/Benefits

• Hybrid work
• Professional development
• Remote work flexibility
• Work from home allowance

Skills/Tech-stack

Application Insights | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Databricks | Azure Event | Azure Event Hubs | Azure Monitor | Azure Purview | Cloud Cost Optimization | Cost Optimization | Data Cataloging | Data Cleansing | Data Lake Storage | Data Lakehouse | Data Lifecycle Management | Data Lineage | Data Processing | Data Security | Data Transformation | Data Validation | Data Warehouse | Data lifecycle | Distributed data | Distributed data processing | ELT | ETL | Event Hubs | Kafka | Lifecycle Management | Log Analytics | Real Time | Real-time Streaming