Lead Data Engineer
Tasks
- Build batch and real time data processing pipelines
- Build streaming data pipelines
- Design ETL/ELT pipelines
- Design scalable data models schemas and warehouse structures
- Develop backend processing components
- Ensure data quality and governance
- Implement CI/CD workflows
- Implement data warehousing solutions
- Improve distributed data processing best practices
- Integrate data from APIs and databases
- Maintain version control and automated deployments
- Optimize pipeline performance and cost
- Participate in architecture discussions
- Perform root cause analysis
- Set up monitoring and security controls
- Transform data for analytics
- Troubleshoot production issues
Perks/Benefits
- N/A
Skills/Tech-stack
Agile | Apache Beam | Automated Deployment | BigQuery | CI/CD | Cloud platform | Data Governance | Data Modeling | Data Quality | Data Warehousing | Dataflow | ELT | ETL | Git | Google Cloud | Google Cloud Platform | Java | Kafka | Monitoring | SQL | Security | Streaming | Version control
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Mid-level Full TimeHyderabad, India3h ago
-
Data Scientist (GenAI/Core DS) INR 2000K-3000KAWS | Data Analysis | Data Modeling | Data Visualization | LangchainSenior-level Full TimeGurugram, Haryana, India4h ago
-
Senior-level Full TimeIN-AP-Hyderabad4h ago
-
Senior-level Full TimeIN-AP-Hyderabad4h ago
-
Senior-level Full TimeIN-AP-Hyderabad4h ago
-
APIs | AWS Kinesis | ClickHouse | DBT | Data GovernanceCareer growth | Flexible collaboration | Inclusive culture | Innovation focus | Remote-friendly work setupSenior-level Full TimeIndia5h ago
-
Senior-level Full Timebangalore, India5h ago
-
Full Stack Engineer – MERN Stack INR 1300K-2500KAPIs | AWS | Azure | Bootstrap | CI/CDCollaborative culture | Growth opportunities | MentorshipMid-level Full TimeMaharashtra, Pune, India5h ago
-
Python AI Engineer INR 1800K-3500KAPI Development | AWS | AWS Lambda | Amazon Bedrock | Amazon SageMakerHybrid workSenior-level Full TimeIndia5h ago
-
Apache Kafka | Codebeamer | Data Validation | Event Driven | Event-driven architectureSenior-level Full Timebangalore, India6h ago
-
Mid-level Full Timebangalore, India6h ago
-
Senior Data Engineer (Python, PySpark, Kafka) INR 1500K-2040KApache Kafka | Data Warehousing | Dimensional Modeling | Flink | GitSenior-level Full Timebangalore, India6h ago
-
Software Engineer (SDE 2) - Full Stack (Frontend), Lakeflow INR 1800K-3000KAPI Integration | AWS | Accessibility | Azure | CSSMid-level Full TimeBengaluru, India6h ago
-
.NET | API Integration | Agents | Automation Anywhere | Azure FunctionsEntry-level Full TimeNashik, MH, India6h ago
-
TRQ27-0135-01-OR Consultant - Data & AI Practice INR 400K-600KCBC | CPLEX | Cloud Computing | Constraint Programming | Error HandlingEntry-level Full TimeIndia7h ago
-
Sr. AI Engineer, Backend (Altium) INR 2000K-4500KAPI Development | Agentic Systems | Algorithms | CI/CD | Data StructuresEmployee resource groups | Remote work optionSenior-level Full TimeBangalore, India8h ago
-
Software Engineering INR 1400K-1820KAzure Cosmos | Azure Cosmos DB | Azure Data | Azure Data Factory | Azure EventMid-level Full TimeHyderabad, TS, IN8h ago
-
AWS | AWS CloudWatch | Alerting | Backups | BashSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Applied Computer Vision Engineer - Data Driven Development INR 2000K-3630KAWS | Apache Spark | Azure | C++ | Convolutional Neural NetworksSenior-level Full Timebengaluru, India9h ago
-
Senior-level Full TimePune, Maharashtra, India10h ago
-
AWS | Amazon Bedrock | Anthropic | Azure | Azure AIFlexible work arrangements | Health insurance | Learning and development | Paid time off | Retirement benefitsSenior-level Full TimeIndia10h ago
-
AWS | Accessibility | Cloud Architecture | Code Reviews | Design PatternsAccess to AI tools | Continuous learning | Flexible paid time off | Global impact | Learning membershipSenior-level Full TimeIndia R10h ago
-
AI Engineer - Acon - BLR INR 2000K-4500KAI Search | AI Studio | Application Security | Arize Phoenix | AzureSenior-level Full TimeIndia10h ago
-
AWS | Anthropic API | C++ | CI/CD | DockerFully remote | Healthcare coverage | No Internal Meetings Fridays | Paid sabbatical | Paid time offSenior-level Full TimeIndia R10h ago
-
Software Engineer - Spark, scala, Python, SQL, Databricks INR 1500K-2500KAirflow | Apache Spark | Automated testing | Azure Data | Azure Data LakeEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India10h ago