Lead Data Engineer
Tasks
- Build CI/CD pipelines in Azure DevOps
- Build Python APIs on Azure Functions
- Build lakehouse and data warehouse solutions
- Design data ingestion and transformation
- Develop Power BI visualizations
- Develop PySpark pipelines
- Document risk controls and project activities
- Ensure architectural alignment
- Gather requirements with business stakeholders
- Handle sensitive data masking and encryption
- Implement Azure Logic Apps
- Lead data engineering projects
- Manage data in secured Azure environments
- Migrate data warehouses to Azure
- Operationalize data solutions
- Provide technical guidance
- Support SDLC testing and deployment
- Tune pipeline performance in ADF and Databricks
Perks/Benefits
- N/A
Skills/Tech-stack
API Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Functions | Azure Key Vault | Azure Logic | Azure Logic Apps | Azure SQL | Azure SQL Database | CI/CD | Data Factory | Data Lake Storage Gen2 | Data Modeling | Data Warehouse | Data masking | Dimensional Modeling | Encryption | Key Vault | Lakehouse | Logic Apps | Managed Identities | Microsoft Azure | Power BI | PySpark | Python | Relational modeling | SQL Database | Service principals
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Data Engineering | Data Engineering Manager | Engineer | Engineering Manager | Lead | Lead Data | Lead Data Engineer | Manager
Related jobs
-
Senior-level Full TimePune City, India3h ago
-
AI Engineer – Reliability & Warranty Analytics INR 1800K-3000KBacktesting | BigQuery | CI/CD | Calibration | Cloud RunMid-level Full TimeChennai, Tamil Nadu, India4h ago
-
Associate III Business Intelligence-SUPPORT SERVICES-Data & Analytics - In House Engineering INR 800K-1800KBig Data | Big data concepts | C++ | Data Visualization | Data conceptsMid-level Full TimeBangalore, Karnataka, India5h ago
-
AI Engineer – Retail/CPG Domain INR 1800K-2500KAWS | Azure | Data Governance | Data Modeling | Data SecurityMid-level Full TimeGurugram, Haryana, India5h ago
-
Associate Principal Engineer (India Office) INR 1500K-2000KADLS | Azure Data | Azure Data Factory | Azure DevOps | CDCMid-level Full TimeBangalore, Karnataka, India7h ago
-
Senior-level Full TimePune, Maharashtra, India7h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India7h ago
-
Applied AI ML Lead INR 2500K-4800KAI Agents | AWS Lambda | AWS S3 | AWS Step Functions | Agentic AISenior-level Full TimeHyderabad, Telangana, India7h ago
-
Agentic AI | Agile | DAX | Data Governance | Data ModelingSenior-level Full TimeBengaluru, Karnataka, India8h ago
-
Lead Software Engineer - Java/Python, LLM, AWS INR 3000K-5000KA/B | A/B Testing | AWS | Agile | Amazon AuroraSenior-level Full TimeBengaluru, Karnataka, India8h ago
-
Platform Engineering Professional - Data Engineer INR 2200K-3500KAWS | Amazon Kinesis | Apache NiFi | Apache Spark | CI/CDMid-level Full TimeBengaluru, IN, 5601038h ago
-
AWS Data Engineer INR 1500K-6000KAWS | Automation | Cloud Architecture | Data Governance | Data IngestionExecutive-level Full TimeHyderabad, Telangana, India8h ago
-
Senior-level Full TimeHyderabad, TG, India9h ago
-
Senior-level Full TimeNoida, Uttar Pradesh, India10h ago
-
FastAPI | Flask | GIS | Git | MultiprocessingMid-level Full TimeNoida, Uttar Pradesh, India10h ago
-
SDET- India (Chennai/Bangalore) INR 1500K-2000KAWS S3 | Apache Airflow | Apache Superset | BDD | Big DataMid-level Full TimeIndia11h ago
-
Amazon Web Services | Cloud platform | Data Modeling | Databricks | Distributed SystemsBroadband reimbursement | Continuous learning support | Global enterprise client exposure | Health insurance | Paid certificationsSenior-level Full TimeIndia11h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Career growth opportunities | Entrepreneurial-minded team | Friendly relaxed atmosphere | Remote/distributed workMid-level Full TimeBangalore, India11h ago
-
Analytics Engineer - Associate INR 1800K-2800KAirflow | Alerting | Azure Data | Azure Data Factory | DagsterSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous culture | Competitive compensation | Flexible management | Friendly atmosphereMid-level Full TimeMumbai, India12h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudMid-level Full TimeNoida, India12h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Friendly work environment | Impactful product | Professional development opportunitiesMid-level Full TimeAhmedabad, India12h ago
-
Software Engineer III - AI/ML, Prompt Engineer INR 2475K-4132KA/B | A/B Testing | API Integration | B testing | Chain-of-ThoughtSenior-level Full TimeMumbai, Maharashtra, India12h ago
-
Mid-level Full TimeBengaluru, Karnataka, India17h ago
-
Developer or Technical Lead (Niche or Regular), Cloud Consultants, QA Tester, QA Lead, Data Migratio INR 2516K-3356KAI Core | AI Launchpad | API first | API-first design | Access ControlSenior-level Full TimeBangalore, KA, IN17h ago