Senior Data Engineer - Senior Manager
Tasks
- Build CI/CD pipelines for data workflows
- Build batch and streaming ETL/ELT pipelines
- Design scalable data ingestion pipelines
- Develop Apache Iceberg table management
- Ensure data governance access control and audit compliance
- Implement Lakehouse architecture
- Implement data quality validation rules
- Monitor end to end ingestion workflows
- Optimize Spark jobs for cost performance and scalability
- Orchestrate pipeline scheduling and retries
- Set up monitoring for SLA and failures
Perks/Benefits
Skills/Tech-stack
AWS | Apache Airflow | Apache Iceberg | CI/CD | DBT | Data Lineage | Data Quality | Databricks | ELT | ETL | Git | PySpark | RBAC | SQL | Terraform | Unity Catalog
Education
N/A
Related jobs
-
Senior-level Full TimeBangalore4h ago
-
Senior-level ContractHyderabad, India5h ago
-
Mid-level Full Timekerala, India5h ago
-
Manager - MIS & Reporting INR 650K-1500KData Governance | Data Visualization | Data Warehousing | Data analytics | Database ManagementAnnual health checkup | Dental benefits | Gender neutral policy | Group term life insurance | Inclusive work cultureMid-level Full TimeGurugram, India6h ago
-
Lead Software Engineer - Java, Spring boot, AWS Cloud & AI/ML (REST APIs, System Design, SDLC) INR 3000K-5000KAPI Gateway | AWS Glue | AWS Lambda | AWS Step Functions | AgileSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Senior-level Full TimeIN-MH-Pune6h ago
-
Data Engineer INR 1500K-2500KAWS | AWS Glue | AWS Lake Formation | AWS Step Functions | Amazon AthenaSenior-level Full TimeIN-MH-Pune6h ago
-
Snowflake Developer INR 1500K-2500KApache Airflow | DBT | Data Sharing | Data Warehousing | Dynamic TablesSenior-level Full TimeIN-TN-Chennai6h ago
-
Data & AI Project Manager INR 3000K-4500KAgile | Analytics | Business Intelligence | Change Management | ComplianceSenior-level Full TimeIN-TN-Chennai6h ago
-
Technologist, Data Analytics Engineering INR 3000K-4500KAI orchestration | API Management | Artificial Intelligence | Automation | CI/CDSenior-level Full TimeBengaluru, KA, India7h ago
-
GenAI Specialist INR 2000K-5000KAI Studio | Agent systems | Agentic AI | Azure AI | Azure AI StudioLeadership development | Training and development programsSenior-level Full TimeBengaluru, India7h ago
-
Mid-level Full TimeIndia9h ago
-
GenAI Engineer INR 1500K-3000KAmazon Web Services | Apache Spark | Azure | Chunking | Data StructuresMid-level Full TimeIndia9h ago
-
DE&A - Core - Cloud Data Engineering - Azure Data Engineering INR 3000K-4000KAWS | Agile methodology | Azure | Azure Data | Azure Data FactorySenior-level Full TimePune, Maharashtra, India11h ago
-
Anthropic SDK | Embeddings | Fine Tuning | Git | GitHub ActionsCollaborative values led team | Exposure to global teams and customers | Generous holiday allowance | Hybrid work environment | Individual career developmentEntry-level Apprenticeship Full TimeWhitefield, Bangalore, India11h ago
-
ML-Ops INR 3500K-5000KAsyncio | Benchmarking | CI/CD | CProfile | ConcurrencyLeadership development programs | Training sessionsSenior-level Full TimeMumbai, Maharashtra, India11h ago
-
Staff GenAI Engineer INR 2500K-5000KAWS | Agentic Systems | Azure | CI/CD | Distributed SystemsCollaborative team environment | Leadership development programs | Training sessionsSenior-level Full TimeMumbai, MH, India11h ago
-
Engineering Manager – AI Platforms INR 3000K-5000KAWS Bedrock | AWS SageMaker | Agile | Azure AI | CI/CDCollaborative team environment | Global consulting experience | Leadership development programs | Training sessionsSenior-level Full TimeMumbai, MH, India11h ago
-
Mid-level Full TimeMumbai, Maharashtra, India11h ago
-
Senior-level Full TimeMumbai, Maharashtra, India12h ago
-
Generative AI Engineer INR 1800K-3300KAI Studio | AWS SageMaker | Amazon Bedrock | Anthropic API | Azure AIGlobal consulting projects | Leadership trainingMid-level Full TimeMumbai, Maharashtra, India12h ago
-
Software Development Senior Associate INR 2000K-4500KAmazon Web Services | Artificial Intelligence | CI/CD | Data Analysis | DockerCareer growth opportunities | Certification support | Collaborative work environment | MentorshipSenior-level Full TimeChennai, TN, IN12h ago
-
Mid-level Full TimePune, India12h ago
-
Lead Data Engineer (OIC+Python) INR 2817K-4064KAPI Management | Apache Airflow | Cloud Composer | Cloud infrastructure | Cloud platformSenior-level Full TimeHyderabad, India12h ago
-
Data Engineer INR 1500K-2500KAccess Control | Airflow | Apache Beam | Apache Kafka | Artifact RegistrySenior-level Full TimeChennai, Tamil Nadu, India13h ago