Data Engineer Sr (Databricks)
Tasks
- Collaborate with Snowflake and dbt teams on data models and data contracts
- Develop Databricks notebooks jobs and workflows
- Fix defects and support SIT UAT and post go live stabilization
- Follow coding standards, version control, and CI/CD practices
- Implement Delta Lake tables and patterns
- Implement incremental loads CDC patterns and batch schedules
- Integrate Databricks with AWS S3 or Azure ADLS ADF Synapse Key Vault and Snowflake
- Optimize Databricks clusters jobs and queries for performance and cost
- Perform data validation and reconciliation
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | AWS S3 | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure Synapse | Azure Synapse Analytics | CDC | CI/CD | Change Data Capture | DBT | Data Capture | Data Factory | Data Lake Storage | Databricks | Delta Lake | ELT | ETL | Git | JDBC | Key Vault | Medallion Architecture | PySpark | Python | SQL | Scala | Schema evolution | Serverless | Slowly Changing Dimensions | Snowflake | Spark SQL | Synapse Analytics | Time Travel | Unity Catalog | Upsert
Education
N/A
Roles
Related jobs
-
Technical specialist- Data Engineer, AS INR 2475K-3465KAgile | Airflow | Apache Beam | Apache Hadoop | Apache SparkAccident insurance | Best in class leave policy | Certification sponsorship | Childcare assistance reimbursement | Employee assistance programMid-level Full TimePune - Margarpatta, India9h ago
-
Technical specialist- Data Engineer, AS INR 2475K-3465KAgile | Airflow | Apache Beam | Apache Spark | Cause analysisAccident and term life insurance | Childcare assistance reimbursement | Employee assistance program | Flexible leave policy | Gender neutral parental leavesMid-level Full TimePune - Margarpatta, India9h ago
-
Data Platform Engineer INR 1000K-2000KAWS | Data Modeling | Data Warehouse | Data integration | ETLMid-level Full TimeBengaluru, BDC7A, India9h ago
-
Mid-level Full TimeBengaluru, BDC14A, India9h ago
-
Mid-level Full TimeKolkata, KDC1A, India9h ago
-
Mid-level Full TimeKolkata, KDC1A, India9h ago
-
Infra Tech Support Practitioner INR 1500K-2500KAKS | API Gateway | AWS | Ansible | Application Lifecycle ManagementMid-level Full TimeKolkata, KDC1A, India9h ago
-
Infra Tech Support Practitioner INR 1000K-1820KAKS | AWS CloudFormation | AWS Lambda | Amazon EC2 | Amazon RDSMid-level Full TimePune, PDC3C, India9h ago
-
Data Platform Operations Engineer (Databricks, DBT, Snowflake, AWS, Aurora) - Mumbai, MH - AS A INR 500K-1500KAWS | Amazon Aurora | Amazon S3 | Backup and Recovery | CDCMid-level Full TimeMumbai, MH17h ago
-
Junior GenAI Developer INR 400K-480KAgentic Workflows | Embeddings | Git | Hybrid search | LLM integrationHealth insurance | Language courses | Mentorship | Professional development opportunities | Relocation programEntry-level Full TimeAll Cities, India R17h ago
-
Amazon Aurora | Amazon RDS | Amazon S3 | Amazon Web Services | CI/CDMid-level Full TimeDahisar West, Mumbai City, Maharashtra17h ago
-
Data and Digital Initiatives INR 3000K-4000KAI | API Integration | Cloud Computing | Credit Risk | DashboardingFlexible working | Inclusive work environment | Opportunities to grow | Professional developmentSenior-level Full TimeMumbai, Maharashtra, India19h ago
-
AI Engineer with RAG / MCP INR 1500K-3000KDocument processing | LLM | Microservices | Model Context Protocol | Prompt engineeringMid-level Full TimeIN-AP-Hyderabad20h ago
-
SDET ETL Testing INR 1500K-2500KAWS | AWS Command Line Interface | AWS Data | AWS Data Catalog | AWS GlueSenior-level Full TimeIN-HR-Gurgaon20h ago
-
AWS Data Engineer- Gen AI INR 1500K-2500KAWS CDK | AWS Glue | AWS Step Functions | Amazon RDS | Amazon RedshiftSenior-level Full TimeIN-TN-Chennai20h ago
-
AWS Data Engineer- Airflow INR 1500K-2000KAWS Glue | AWS Lambda | Amazon Redshift | Amazon S3 | Apache HadoopMid-level Full TimeIN-AP-Hyderabad20h ago
-
Junior GenAI Developer INR 400K-480KAgentic Workflows | Embeddings | Git | Hybrid search | LLM APIHealth insurance | Language courses | Mentorship | Professional development opportunities | Relocation programEntry-level Full TimeAll Cities, India R20h ago
-
MLOps Engineer INR 1200K-1680KAWS | Bedrock | CI/CD | Data Versioning | DockerEmployee Share Savings Plan | Hybrid work | Learning and mentoring | Paid volunteer day | Personal coachingMid-level Full TimeBengaluru22h ago
-
Senior-level Full TimePune, Maharashtra, India22h ago
-
Senior-level Full TimePune, Maharashtra, India22h ago
-
Staff Machine Learning Engineer INR 4000K-5000KC# | C++ | Deep learning | Fine Tuning | HadoopDental insurance | Equity awards | Health insurance | Hybrid work | Vision insuranceSenior-level Full TimeBengaluru, Karnataka, India R22h ago
-
Mid-level Full TimeIndia23h ago
-
Lead Software Engineer - Data Engineering INR 3500K-5000KAWS | Airflow | Authentication | Authorization | CI/CDSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
Senior-level Full TimeMumbai, Maharashtra, India23h ago
-
Senior Analyst - AI/ML INR 3000K-5000KAlgorithms | Artificial Intelligence | Automl | Data Science | Data StructuresMentorshipSenior-level Full TimePune, Maharashtra, IND, 4110141d ago