Data Engineering Lead - Assistant Vice President 2
Tasks
- Apply data governance and catalog controls
- Build batch and streaming data pipelines
- Design data lake architectures for analytics and ML
- Develop ingestion patterns from RDBMS APIs and event streams
- Implement incremental loads and CDC patterns
- Implement partitioning and data compaction strategies
- Operate production data platforms in cloud environments
- Optimize Spark SQL queries and performance
- Use orchestration tools to schedule data workflows
Perks/Benefits
- Employee networks
- Flexible work/life support
- Inclusive development opportunities
- Paid volunteer days
Skills/Tech-stack
AWS | AWS Kinesis | Airflow | Apache Iceberg | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Event | Azure Event Hubs | CDC | CI/CD | Cause analysis | DBT | Data Factory | Data Lakehouse | Data pipeline | Databricks | Databricks Workflows | ETL | Event Hubs | IAM | Incident Response | Incremental loading | Kafka | Object storage | Partitioning | PySpark | Python | Query Optimization | Root Cause Analysis | Root cause | S3 | SQL | Spark SQL | Structured Streaming | Unity Catalog
Education
Related jobs
-
A/B | A/B Testing | Attention | B testing | CI/CDSenior-level Full TimeIndia1d ago
-
Lead OE Analytics INR 2000K-3500KAudit Support | Automation | Cause analysis | Cost Optimization | Data AnalysisSenior-level Full TimeKorba, Chhattisgarh, India1d ago
-
Data Analytics Lead INR 2516K-3380KAutomation | Data Cleansing | Data Engineering | Data Governance | Data ModelingCharity match program | Employee assistance program | Group medical insurance | Hybrid work flexibility | Paid time offSenior-level Full TimeGurgaon - Cyber Park, India1d ago
-
Data Engineering Operations Lead INR 3000K-4000KAgile | Apache Airflow | Apache Hive | Apache Spark | BigQueryAnnual leave | Away from keyboard time | Contributory pension scheme | Diversity forums and networking groups | Fitness Wellbeing ProgrammesSenior-level Full TimePune, India R1d ago
-
Senior-level Full TimeBangalore, In, India1d ago
-
Lead Data Analyst INR 2500K-3356KA/B | A/B Testing | Airflow | Artificial Intelligence | AutomationSenior-level Full TimeBangalore,India1d ago
-
Lead Software Engineer - Specialist INR 1800K-2300KAirflow | Apache Beam | BigQuery | Bigtable | CI/CDHealthcare package | Hybrid work setting | Online learning platform | Paid time offSenior-level Full TimeIND-Trivandrum-Equifax Analytics-PEC, India1d ago
-
Java Technical Lead- Career INR 2156K-3000KAWS | Agile | Airflow | Angular | Apache BeamHealthcare package | Hybrid work setting | Online learning platform | Paid time offSenior-level Full TimeIND-Trivandrum-Equifax Analytics-PEC, India1d ago
-
Senior Manager - SAP Analytics Cloud & Datasphere INR 2500K-3356KAnalytics Cloud | CI/CD | Cross-calculations | Data Blending | Data GovernanceSenior-level Full TimeIndia - Chennai1d ago
-
Senior Manager - SAP Analytics Cloud & Datasphere INR 2500K-3356KAgile | Analytics Cloud | CI/CD | Cross-calculations | Data BlendingSenior-level Full TimeIndia - Chennai1d ago
-
Java Gen Ai Tech Lead INR 2000K-3500KAI Search | AWS | Azure | Azure AI | Azure AI SearchDigital learning platform access | Elder care | Flexible work | Health checks | Health insuranceSenior-level Full TimePune, IN1d ago
-
Lead SQL Server Database Engineer INR 2542K-3500KAWS RDS | Agile | Always On | Always On Availability | Always On Availability GroupsFlexible hybrid work model | Health insurance | Life insurance | Paid time off | Pension and retirement benefitsSenior-level Full TimeHyderabad, India R1d ago
-
Snowflake COE Lead INR 1500K-2500KAPIs | Apache Airflow | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeIndia2d ago
-
Agile | Amazon Redshift | Batch Processing | BigQuery | CI PipelineCollaborative global work environment | Cross-functional collaboration | Flexible working arrangements | Fully remote friendly | Inclusive cultureSenior-level Full TimeIndia R2d ago
-
API | Batch Processing | Data Compliance | Data Governance | Data ModelingContinuous learning | Flexible work hours | Health and wellness support | Inclusive work culture | Paid time offSenior-level Full TimeIndia2d ago
-
Data Analytics Lead INR 2516K-3356KAI tools | Customer Analytics | Data Quality | Data analytics | Power BICharity match program | Employee Assistance Program (EAP) | Group medical insurance | Hybrid work flexibility | Paid time offSenior-level Full TimeGurgaon - Cyber Park, India2d ago
-
21 CFR | 21 CFR Part 11 | CDISC | CFR Part 11 | Clinical dataCareer development | Inclusive work environment | Mentorship and coaching | Succession planning | Training complianceSenior-level Full TimeIN022 Hyderabad, India2d ago
-
ITOPS Lead - Analytics INR 2500K-3200KAnalytics Cloud | Calculation views | Change Management | Configuration Management | Data ComplianceSenior-level Full TimeBangalore (Airbus), India2d ago
-
Lead, Data Analyst INR 3000K-4200KData Analysis | Data Architecture | Data Governance | Data Pipelines | Data TransformationBe Well programs | Certification opportunities | Coaching | Continuous feedback | Flexible work arrangementSenior-level Full TimeINMANBP Bangalore (INMANBP) Manyatha, India2d ago
-
Principal AI Architect INR 2500K-4500KAI Development | AI Testing | AI monitoring | Agent systems | Agentic FrameworksSenior-level Full TimeIND Bengaluru Karnataka 8th Floor, 359/49/9, …2d ago
-
Senior-level Full TimeKA Bangalore, India2d ago
-
Mid-level Full TimeHyderabad, Telangana, India2d ago
-
Mid-level Full TimeHyderabad, Telangana, India2d ago
-
QA Automation Lead (Data Engineering)- India INR 1632K-2829KAsynchronous processing | CI/CD | Code review | Data Pipelines | Data QualityHybrid work options | Remote work optionsSenior-level Full TimeIndia R2d ago
-
Lead Data Engineer INR 3000K-4000KELT | ETL | JSON | MongoDB | MySQLCareer growth opportunities | Flexible work location | Generous time off | Remote-first work environment | Wellbeing initiativesSenior-level Full TimeIndia3d ago