Senior Data Engineer - Senior Manager
Tasks
- Automate Databricks job deployments
- Build batch and streaming ETL/ELT pipelines
- Build pipeline monitoring and alerting
- Design and develop data ingestion pipelines
- Develop CI/CD pipelines for data workflows
- Implement Apache Iceberg table management
- Implement Lakehouse architecture
- Implement data governance and access control
- Implement data quality checks and reconciliation
- Manage data pipeline orchestration
- Optimize Spark jobs for cost and performance
Perks/Benefits
- Flexible work/life support
- Inclusive development opportunities
- Paid volunteer days
- Vibrant employee networks
Skills/Tech-stack
Apache Airflow | Apache Iceberg | Batch Processing | CI/CD | Data Governance | Data Lineage | Data Quality | Data Validation | Databricks | Databricks Jobs | ELT | ETL | Git | PySpark | RBAC | SQL | Spark | Streaming | Terraform | Unity Catalog | Workflow Orchestration
Education
N/A
Related jobs
-
Senior-level Full TimeIN-KA-Bangalore22h ago
-
Mid-level Full TimeIN-KA-Bangalore22h ago
-
Java - Senior Developers INR 1500K-3000KAPI Design | AWS | CI/CD | Design Patterns | DevSecOpsContinuous improvement | Cross-functional collaboration | MentorshipSenior-level Full TimeIndia23h ago
-
Senior-level Full TimeNoida, UTTAR PRADESH, India1d ago
-
A/B | A/B Testing | Attention | B testing | CI/CDSenior-level Full TimeIndia1d ago
-
Associate III Software Engineering-SUPPORT SERVICES-Data & Analytics - In House Engineering INR 1300K-2500KAWS | Apache Kafka | Azure | Docker | GCPImmediate joiningMid-level Full TimeHyderabad, Telangana, India1d ago
-
Associate III Software Engineering-SUPPORT SERVICES-Data & Analytics - In House Engineering INR 1300K-2500KAWS | Azure | CI/CD | Docker | GCPMid-level Full TimeHyderabad, Telangana, India1d ago
-
Associate III Software Engineering-SUPPORT SERVICES-Data & Analytics - In House Engineering INR 1300K-2500KAWS | Agile | Algorithms | Azure | CI/CDMid-level Full TimeHyderabad, Telangana, India1d ago
-
Data Quality Engineer, Enterprise Data Platform - Data Quality & Data governance) , AWS , SQL & Python , Cloud, Power BI INR 1500K-2000KAWS Glue | AWS Glue Catalog | Amazon Athena | Amazon EMR | Amazon RedshiftMid-level Full TimeBengaluru, KA, India1d ago
-
Senior-level Full TimeBengaluru, India1d ago
-
AWS | Azure | CD pipelines | CI/CD | CI/CD pipelinesCareer growth | Flexible engagement | Fully remoteSenior-level Full TimeIndia R1d ago
-
Data Governance | Data Modeling | Data Quality | Data Security | Data WarehousingCareer growth | Collaborative environment | Continuous learning | Flexible day shift | Fully remote workSenior-level Full TimeIndia R1d ago
-
Mid-level Full TimeBengaluru, India1d ago
-
Software Development Engineer 4 INR 1500K-2800KAI Foundry | API Gateway | AWS Step Functions | Airflow | Amazon EventBridgeSenior-level Full TimeBangalore, India R1d ago
-
AWS | Apache Kafka | Azure | Azure Data | Azure Data BricksCareer development opportunities | Coaching and feedback | Flexible work arrangements | Training and learning supportMid-level Full TimeBengaluru, KA, IN, 5600161d ago
-
Sr. Data Engineer INR 2800K-4000KAgile | Apache Airflow | Apache Hive | Apache Spark | Batch ProcessingAnnual leave | Contributory pension scheme | Diversity forums | Hybrid work | Mindfulness app accessSenior-level Full TimePune, India R1d ago
-
Data Engineering Operations Lead INR 3000K-4000KAgile | Apache Airflow | Apache Hive | Apache Spark | BigQueryAnnual leave | Away from keyboard time | Contributory pension scheme | Diversity forums and networking groups | Fitness Wellbeing ProgrammesSenior-level Full TimePune, India R1d ago
-
Senior Enterprise Cloud Software Engineer (Cloud Operations, AI Engineering, Azure & AWS) INR 2500K-4000KAKS | APM | ARM | AWS SDK | Access ManagementSenior-level Full TimeIND - Chennai, Neville Towers, India1d ago
-
Senior-level Full TimeBengaluru, KA, IN, 5600161d ago
-
Senior Computer Scientist 1 INR 2475K-3800KAI orchestration | CI/CD | Cloud Platforms | Docker | Integration TestingSenior-level Full TimeNoida, India R1d ago
-
Solution Engineer (ETL, Cloud) INR 1500K-2400KAWS | Azure | Cassandra | DB2 | Data GovernanceWork from Anywhere cultureEntry-level Full TimeIndia R1d ago
-
Technology Architect- AI & DATA INR 2000K-4800KAPI Integration | AWS | Architecture governance | Artificial Intelligence | Cloud ComputingSenior-level Full TimeBangalore,Karnataka,India1d ago
-
Lead SQL Server Database Engineer INR 2542K-3500KAWS RDS | Agile | Always On | Always On Availability | Always On Availability GroupsFlexible hybrid work model | Health insurance | Life insurance | Paid time off | Pension and retirement benefitsSenior-level Full TimeHyderabad, India R1d ago
-
API Security | Application Security | Automation | Bash | CI/CDMid-level ContractHyderabad, India1d ago
-
Mid-level Full TimeIN-MH-Mumbai1d ago