Lead Data Engineer
Tasks
- Build batch and real time data processing pipelines
- Build streaming data pipelines
- Design ETL/ELT pipelines
- Design scalable data models schemas and warehouse structures
- Develop backend processing components
- Ensure data quality and governance
- Implement CI/CD workflows
- Implement data warehousing solutions
- Improve distributed data processing best practices
- Integrate data from APIs and databases
- Maintain version control and automated deployments
- Optimize pipeline performance and cost
- Participate in architecture discussions
- Perform root cause analysis
- Set up monitoring and security controls
- Transform data for analytics
- Troubleshoot production issues
Perks/Benefits
- N/A
Skills/Tech-stack
Agile | Apache Beam | Automated Deployment | BigQuery | CI/CD | Cloud platform | Data Governance | Data Modeling | Data Quality | Data Warehousing | Dataflow | ELT | ETL | Git | Google Cloud | Google Cloud Platform | Java | Kafka | Monitoring | SQL | Security | Streaming | Version control
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Data Quality Analyst ( ETL/Data Warehouse ) INR 800K-1600KAgile | Anomaly Detection | Azure Data | Azure Data Factory | CI/CDMid-level Full TimeHome - India (HOMEIND)12h ago
-
Senior Data Engineer - Full Stack (US, Remote) INR 2000K-3000KAPI | AWS | CI/CD | CLI | Data GovernanceCollaborative team | Developer productivity focus | Remote workSenior-level Full TimeMaharashtra, Pune, India R12h ago
-
AWS Cloud Engineer INR 1500K-3000KAWS CloudFormation | AWS Fargate | AWS Glue | AWS Lambda | AWS Step FunctionsSenior-level Full TimeBangalore, India12h ago
-
Senior-level Full TimeBangalore, India12h ago
-
Architect (ATC) INR 1500K-3500KAutomation | Backup Restore Validation | Backup/Restore | CI/CD | Cause analysisSenior-level Full TimeIN-MH-Pune13h ago
-
Senior-level Full TimeIN-AP-Hyderabad13h ago
-
Senior-level Full TimeKolkata,West Bengal,India16h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India18h ago
-
Mid-level Full TimeHyderabad, IN18h ago
-
Lead Software Engineer (Cloud Native | Microservices | AWS) INR 3000K-4000K.NET | API | AWS | AWS CDK | Behavior-Driven DevelopmentEnhanced medical benefits | Hybrid work | Paid time off | Wellbeing benefits | Work-life balanceSenior-level Full TimeHyderabad, India18h ago
-
Global HR Reporting and Analytics Lead (HR/Workday reporting) INR 3000K-4000KData Architecture | Data Quality | Data Quality Management | Data Visualization | Data integrationCoaching and mentoring | On-the-job learning | Skills based technical training | Training and developmentSenior-level Full TimeDelhi, India18h ago
-
GenAI Engineer - Database INR 2000K-3465KData Governance | Data Privacy | Data Security | Embeddings | Graph AlgorithmsEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimePune, MH, India19h ago
-
Architect Data Products(M/F/D) INR 2400K-4000KAPIs | Automation | Azure | Azure Data | Azure Data FactoryCareer development | Flexible working hours | Inclusive work environment | International networking | Learning opportunitiesSenior-level Full TimeHyderabad, IN19h ago
-
Cloud Data Engineer(M/F/D) INR 1500K-2000KAPIs | Agile | Automation | Azure | Azure DataCareer development | Flexible working models | International networking | Learning opportunitiesMid-level Full TimeHyderabad, IN19h ago
-
Senior-level Full TimeBangalore, India20h ago
-
Sr Lead Software Engineer INR 3000K-5000KAI assisted code | AI-assisted code review | AI-assisted refactoring | API Versioning | AWSSenior-level Full TimeBengaluru, Karnataka, India21h ago
-
Senior-level Full TimeBangalore,Karnataka,India23h ago
-
Amazon Web Services | Apache Airflow | Apache Hadoop | Azure Data | Azure Data FactoryFlexible work arrangements | Mentorship | Wellbeing supportSenior-level Full TimeChennai - Menon Eternity, India1d ago
-
API Management | Azure Cognitive | Azure Cognitive Search | Azure Databricks | Azure FunctionsSenior-level Full TimeAiroli, India1d ago
-
Data Engineer (Audience Intelligence) INR 1500K-2500KAzure | Azure Data | Azure Data Factory | Azure DevOps | Azure FunctionsMid-level Full TimeHyderabad, Telangana, India1d ago
-
Embedded SW Engineer- AS Core INR 1800K-3000KASPICE | AUTOSAR | Automotive communication | CAN | Code reviewMid-level Full TimeIND - India Tech Center1d ago
-
Python Gen AI consultant INR 3000K-4000KAutogen | Automated testing | CI/CD | Cloud platform | Crew AIMid-level Full Timebangalore, KA, IN1d ago
-
Sr Support Engineer 1-Functional Support,WMS,Fulfillment INR 1500K-2500KAPI | Application logs | Automated Fulfillment Workflows | Cause analysis | DashboardsSenior-level Full TimeIND - Coimbatore (708), India1d ago
-
Mid-level Full TimeBangalore, KA, IN1d ago
-
Sr Technical Consultant-Cloud - Java,Microservices,Kafka,Nosql INR 2200K-3500KAKS | ARM Templates | Active Directory | Apache Spark | AzureSenior-level Full TimeBangalore, India1d ago