DE&A - Sr. Data Engineer - Databricks
Pune, Maharashtra, India
INR 1500K-2800K (estimate) Senior-level Full Time
Tasks
- Build and maintain Delta Live Tables pipelines
- Design ETL/ELT pipelines on Databricks
- Design medallion lakehouse models
- Implement CI CD for Databricks with Asset Bundles and Git
- Implement data governance and access control with Unity Catalog
- Implement data quality testing and observability frameworks
- Integrate Databricks with cloud storage streaming and security services
- Mentor engineers and lead code reviews
- Optimize Spark jobs and Databricks clusters
- Orchestrate pipelines with Databricks Workflows Jobs
- Troubleshoot production pipeline issues and perform root cause analysis
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | AWS S3 | Asset bundles | Auto Scaling | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | CI/CD | Cloud Storage | Data Factory | Data Lake Storage | Databricks | Databricks (asset) | Databricks Asset Bundles | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | Event Hubs | Git | Google Cloud | Google Cloud Storage | Great Expectations | IAM | Kafka | Kinesis | Lakehouse | Lakehouse Monitoring | Medallion Architecture | Photon | PySpark | Python | Scala | Serverless computing | Spark SQL | Structured Streaming | Terraform | Unity Catalog | Z-Order
Education
N/A
Roles
Related jobs
-
Senior-level Full TimeDelhi, All4h ago
-
Senior-level Full TimeBangalore, Karnataka, India7h ago
-
Assure AI Engineer INR 3000K-4000KAI Act | AI RMF | Adversarial prompting | Audit evidence | Bias TestingSenior-level Full TimeBangalore, Karnataka, India7h ago
-
Software Engineer - AI/ML INR 2500K-3500KAI Search | AWS | Agent systems | Agentic AI | AutogenAccess to cutting-edge technologies | Collaborative work environment | Professional growth and development | Training and developmentSenior-level Full TimeBangalore - Karnataka, India - Kalyani …7h ago
-
Senior Data Engineer (Python, SQL, Snowflake, Devops, Azure/ AWS, Co-pilot, Kubernetes, Docker) INR 2520K-3500KADF | AWS | Airflow | Automated Data Profiling | AzureEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India8h ago
-
Applied AI ML Associate Senior INR 2000K-4000KAWS | Agent systems | Anomaly Detection | Attention | BERTMid-level Full TimeBengaluru, Karnataka, India9h ago
-
Applied AI/ML Engineer INR 3000K-5000KAWS | Amazon SageMaker | BERT | Causal Inference | ClassificationSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Lead Software Engineer - Terraform, Python, Kubernetes INR 3000K-5000KAWS | Amazon S3 | Automated testing | Bedrock | CI/CDSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
BigQuery | Cloud platform | Data Engineering | Data Modeling | Data PipelinesMid-level Full TimeIndia9h ago
-
Mid-level Full TimeTelangana, Hyderabad, India10h ago
-
Principal Software Engineer -Forward Deployment Engineer Reframe INR 2500K-4000KAWS S3 | CI/CD | Catalog | Cloud Native | Consent ManagementSenior-level Full TimeHyderabad, Telangana, India10h ago
-
Software Engineer (AI/ML) INR 500KAWS | Attention Mechanisms | Azure | Cloud platform | Deployment PipelinesBirthday day off | Career advancement | Casual leave | Employee assistance program | Learning and development opportunitiesMid-level Full TimeBengaluru, Onsite10h ago
-
Senior-level Full TimeBangalore, IN11h ago
-
Technical Lead INR 2000K-5000KAzure Machine Learning | CI/CD | Copilot Studio | Data Pipelines | DjangoSenior-level Full TimeChennai, Tamil Nadu, India11h ago
-
Mid-level Full TimeKarnataka, India11h ago
-
Senior-level Full TimeHyderabad, TS, India12h ago
-
Sr AI Engineer (Generative AI & Pharmacovigilance) INR 3264K-5000KAWS | Agentic AI | Azure OpenAI | CI/CD | CrewAISenior-level Full TimePune, MH, India12h ago
-
Backend Engineer (Cloud & AI Integration) INR 1500K-3000KAWS | Git | NoSQL | Production Support | PythonMid-level Full TimeBengaluru, KA, India12h ago
-
Senior Software Engineer - Full Stack & GenAI INR 3242K-4680KAPI Security | AWS | Amazon RDS | Amazon S3 | AngularHybrid work | Professional developmentSenior-level Full TimeBengaluru, KA, India R12h ago
-
Principal Member Technical Staff - (MTS 4) - Machine learning INR 3200K-4580KAWS | Anomaly Detection | Azure | Cassandra | Cosmos DBSenior-level Full TimeBengaluru, India12h ago
-
DevOps Track Sr.Engineer INR 1500K-3500KAccess Management | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Full TimeIndia13h ago
-
Senior Data Engineer INR 1500K-2600KDBT | Data Marts | Data Modeling | Data Quality | Data ReconciliationSenior-level Full TimeBangalore, KA, IN, 56007613h ago
-
Senior Data Engineer INR 3000K-4000KAmazon Web Services | Cloud platform | Continuous Delivery | Data Engineering | Data GovernanceFlexible work arrangements | Learning and development programs | MentorshipSenior-level Full TimeBangalore, India; Chennai, India; Hyderabad, India; …14h ago
-
Lead, AI Engineer INR 2500K-5000KAWS Bedrock | Agentic Workflows | Anthropic Claude | Automated Regression | Automated regression testingCareer growth | Company-sponsored certifications | Mentorship | Recognition rewards | SnacksSenior-level Full TimeHyderabad, TS, India14h ago
-
Senior Software Engineer, Machine Learning INR 3600K-5400KA/B | A/B Testing | Amazon S3 | Apache Airflow | B testingDental insurance | Financial wellness support | Health insurance | Mental health support | Paid time offSenior-level Full TimeBengaluru, India R14h ago