Lead Data Engineer
Tasks
- Build CI/CD pipelines
- Build data pipelines with Azure Data Factory and Databricks
- Collaborate with business for requirements
- Design data ingestion and transformation
- Develop Python based APIs on Azure Functions
- Develop testing and deployment workflows
- Implement Lakehouse and Data Warehouse solutions
- Implement data security with Key Vault and managed identities
- Lead data engineering projects
- Migrate on-premise data warehouses to Azure
- Operationalize data solutions
- Perform data masking and encryption
- Provide technical guidance to data engineers
- Support data visualization with Power BI
- Tune pipeline performance
Perks/Benefits
- N/A
Skills/Tech-stack
API Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Functions | Azure Key Vault | Azure Logic | Azure Logic Apps | Azure SQL | Azure SQL Database | CI/CD | Data Factory | Data Lake Storage Gen2 | Data Modeling | Data encryption | Data masking | Dimensional Modeling | Key Vault | Lakehouse | Logic Apps | Managed Identities | Microsoft Azure | Power BI | PySpark | Python | Relational modeling | SQL Database | Service principals
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Senior Data Engineer
Related jobs
-
Senior Full-Stack AI Engineer- 3185 (Remote) INR 2000K-4500KAI Foundry | AWS Bedrock | Agile | Anthropic | Azure AIFlexible working hours | Learning and development programs | Recognition rewards | Remote workSenior-level Full TimeChennai, India R9h ago
-
Full-Stack AI Engineer - 3184 (Remote) INR 2000K-4500K.NET | AI Foundry | Agile | Azure AI | Azure AI FoundryFlexible working hours | L and D programs | Professional development | Remote work | Rewards and recognitionSenior-level Full TimeChennai, India R9h ago
-
Data Platform Engineer/Consultant Specialist INR 1500K-2500KAmazon S3 | Ansible | Atlassian Confluence | Atlassian Jira | AutomationFlexible workingMid-level Full TimeHyderabad, Telangana, India R9h ago
-
Associate Engineer - Azure Data engineer INR 747K-1039KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksMid-level Full TimeBangalore - Karnataka, India - Kalyani …10h ago
-
Gen AI Developer INR 2000K-4500KData Engineering | Deep learning | Distributed Systems | GenAI | InferenceSenior-level Full TimeIN-MH-Pune10h ago
-
AWS | Agile | Amazon Web Services | Authentication | AuthorizationFlexible working | Inclusive work culture | Opportunities for growth | Professional developmentMid-level Full TimeHyderabad, Telangana, India R10h ago
-
Senior-level Full Timebengaluru, India12h ago
-
API | AWS | Agile | CI/CD | CSSMid-level Full TimeHyderabad, Telangana, India12h ago
-
Senior Gcp Data Engineer/Senior Consultant Specialist INR 1500K-2500KAccess Control | Apache Airflow | Apache Kafka | Apache NiFi | Audit LoggingSenior-level Full TimePune, Maharashtra, India R12h ago
-
Association | Classification | Data Modeling | Data Warehousing | ETLSenior-level Full TimeChennai, Tamil Nadu, India13h ago
-
Senior Software Engineer I INR 2200K-2829KAPI Design | Airflow | Alerting | Apache Spark | Batch ProcessingCareer growth | Comprehensive health and wellbeing support | Leadership development | Learning opportunitiesSenior-level Full TimeChennai, TN, India13h ago
-
Data Engineer_Sr_Python Specialist INR 1500K-2500KAPIs | Alerting | Data Augmentation | Data Engineering | Data LineageSenior-level Full Timebangalore, India13h ago
-
Senior Data Engineer I INR 2040K-3500KAirflow | Analytics hub | Authorized Views | BigQuery | BigQuery EditionsSenior-level Full TimeChennai, TN, India14h ago
-
Software Engineer II INR 1000K-1700KApache Spark | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactoryMid-level Full TimeHyderabad, TS, IN14h ago
-
Mid-level Full TimeIndia14h ago
-
Staff GenAI Engineer INR 2500K-4800KAWS | Agentic Workflows | Azure | CI/CD | Data PipelinesCollaborative team environment | Leadership development programs | Training sessionsSenior-level Full TimeMumbai, MH, India14h ago
-
Generative AI Engineer INR 1800K-3000KAI Foundry | AWS SageMaker | Amazon Bedrock | Anthropic | Azure AICollaborative team environment | Leadership development program | Training sessionsMid-level Full TimeMumbai, Maharashtra, India14h ago
-
Senior-level Full TimeMumbai, Maharashtra, India14h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India14h ago
-
Mid-level Full TimeMumbai, Maharashtra, India14h ago
-
Mid-level Full TimeIndia15h ago
-
Senior Software Engineer INR 2000K-4500K.NET | Agile | Artificial Intelligence | Azure | Azure FunctionsSenior-level Full TimeBengaluru, KA, IN; Hyderabad, TS, IN15h ago
-
Senior-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101315h ago
-
Airflow | As-a-Service | BigQuery | CI/CD | Cloud SQLContinuous professional development | Flexible working | Inclusive workplaceMid-level Full TimeHyderabad, Telangana, India R15h ago
-
Agile | CI/CD | Cloud Native | Cloud platform | Continuous DeliveryContinuous professional development | Flexible working | Inclusive and diverse environment | Opportunities for growthMid-level Full TimePune, Maharashtra, India R16h ago