Senior Data Engineer - Senior Manager
Tasks
- Automate Databricks job deployments
- Build batch and streaming ETL/ELT pipelines
- Build pipeline monitoring and alerting
- Design and develop data ingestion pipelines
- Develop CI/CD pipelines for data workflows
- Implement Apache Iceberg table management
- Implement Lakehouse architecture
- Implement data governance and access control
- Implement data quality checks and reconciliation
- Manage data pipeline orchestration
- Optimize Spark jobs for cost and performance
Perks/Benefits
- Flexible work/life support
- Inclusive development opportunities
- Paid volunteer days
- Vibrant employee networks
Skills/Tech-stack
Apache Airflow | Apache Iceberg | Batch Processing | CI/CD | Data Governance | Data Lineage | Data Quality | Data Validation | Databricks | Databricks Jobs | ELT | ETL | Git | PySpark | RBAC | SQL | Spark | Streaming | Terraform | Unity Catalog | Workflow Orchestration
Education
N/A
Related jobs
-
Data Quality Engineer, Enterprise Data Platform - Data Quality & Data governance) , AWS , SQL & Python , Cloud, Power BI INR 1500K-2000KAWS Glue | AWS Glue Catalog | Amazon Athena | Amazon EMR | Amazon RedshiftMid-level Full TimeBengaluru, KA, India1d ago
-
Data Governance | Data Modeling | Data Quality | Data Security | Data WarehousingCareer growth | Collaborative environment | Continuous learning | Flexible day shift | Fully remote workSenior-level Full TimeIndia R1d ago
-
Technology Architect- AI & DATA INR 2000K-4800KAPI Integration | AWS | Architecture governance | Artificial Intelligence | Cloud ComputingSenior-level Full TimeBangalore,Karnataka,India1d ago
-
API Security | Application Security | Automation | Bash | CI/CDMid-level ContractHyderabad, India1d ago
-
Mid-level Full TimeIN-MH-Mumbai1d ago
-
Pyspark Developer INR 800K-1500KData Modeling | Dimensional Modeling | PySpark | Relational databases | SQLMid-level Full TimeIN-TN-Chennai1d ago
-
AWS Data Engineer- Airflow INR 1500K-2400KAWS Glue | AWS Lambda | Amazon Redshift | Amazon S3 | Apache HadoopSenior-level Full TimeIN-AP-Hyderabad1d ago
-
Software Engineer - AI/ML with Python INR 2500K-3500KAWS | Airflow | Amazon SageMaker | Apache Airflow | Apache SparkAccess to cutting-edge technologies | Collaborative work environment | In office 5 days per week | Professional development | Training and education opportunitiesSenior-level Full TimeBangalore - Karnataka, India - Kalyani …1d ago
-
MS Fabric / Azure Data Engineer ( 5- 7 yrs ) - Immediate Joiners INR 1500K-2500KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | Data ArchitectureImmediate JoinersSenior-level Full TimeBengaluru, KA, India1d ago
-
Staff Software Engineer, Analytics INR 2520K-4000KAlgorithms | Analysis and Design | Big Data | C++ | Data IngestionSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
None Full TimeNoida, Uttar Pradesh, India1d ago
-
None Full TimeNoida, Uttar Pradesh, India1d ago
-
Snowflake COE Lead INR 1500K-2500KAPIs | Apache Airflow | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeIndia2d ago
-
Senior Software Engineer(AI/ML), Trust INR 4500K-6500KA/B | A/B Testing | APIs | Algorithms | Anomaly DetectionSenior-level Full TimeBangalore, India2d ago
-
None Full TimeIndia2d ago
-
Data Architect INR 3200K-3880KAWS Glue | AWS S3 | Access Control | Amazon Athena | Anomaly DetectionSenior-level Full TimeBengaluru, Karnataka, India2d ago
-
AI Software Engineer III - Platform INR 2000K-4000KAPI Gateway | APIs | AWS EC2 | AWS Lambda | Amazon APISenior-level Full TimeHyderabad, Telangana, India2d ago
-
Senior Big Data Engineer (R-19335) INR 2400K-3380KAccess Control | Apache Airflow | Apache Spark | Attribute-Based Access Control | Attribute-based accessSenior-level Full TimeHyderabad - India2d ago
-
Senior AI Engineer INR 1800K-2800KAWS | Algorithms | Anthropic API | Azure | CI/CDCareer Development Programs | Open source contribution support | Team knowledge sharingSenior-level Full TimeBangalore; Gurgaon, India2d ago
-
Senior-level Full TimeBangalore; Gurgaon, India2d ago
-
Audit Logging | Azure | Azure Data | Azure Data Factory | Azure Data LakeCareer advancement opportunities | Group health insurance | Paid Maternity Leave | Paid leaves annually | Paid time offSenior-level Full TimeIndia2d ago
-
Azure Data Engineer INR 1500K-2500KAzure Data | Azure Data Factory | Azure Data Lake | Azure Functions | Azure SynapseMid-level Full TimeIndia2d ago
-
Mid-level Full TimeBengaluru, Karnataka, India2d ago
-
Associate Process Manager INR 800K-1500KAWS | Apache Airflow | Apache Hadoop | Apache Spark | AzureMid-level Full TimePune, Maharashtra, India2d ago
-
Website Analytics Architect INR 2520K-3880KA/B | A/B Testing | AEMaaCS | Adobe Analytics | Adobe ExperienceSenior-level Full TimeBangalore, Karnataka, IN, 5601002d ago