Databrick Solution Architect
Tasks
- Build scalable data engineering pipelines using Databricks
- Configure clusters jobs workflows and repos
- Define medallion architecture
- Design Databricks Lakehouse architecture
- Design batch and streaming pipelines
- Develop PySpark transformations
- Develop SQL based transformations
- Implement CI CD pipelines for Databricks deployments
- Implement Delta Lake based data models
- Implement Unity Catalog governance model
- Lead code reviews and architecture reviews
- Optimize Databricks performance and cost
- Provide hands on support during development
Perks/Benefits
- N/A
Skills/Tech-stack
Batch Processing | CD pipelines | CI/CD | CI/CD pipelines | Cluster management | Data Engineering | Data Pipeline Architecture | Data pipeline | Databricks | Delta Lake | Lakehouse Architecture | Pipeline architecture | PySpark | SQL | Streaming | Unity Catalog
Education
N/A
Related jobs
-
Senior Manager, Data Engineering Service & SRE INR 2000K-3500KAnomaly Detection | Apache Airflow | Automated testing | BigQuery | CloudSenior-level Full TimeBengaluru, IN, 5601038h ago
-
Director - Data Engineering and ML INR 1500K-6000KArtificial Intelligence | Batch Processing | Data Architecture | Data Compliance | Data EngineeringHybrid work environmentExecutive-level Full TimeBengaluru, INDIA, India13h ago
-
IFC Data - Data Architect INR 2000K-3500KCloud Computing | Data Architecture | Data Governance | Data Lineage | Data ModellingSenior-level Full TimePune, Gera Commerzone SEZ, India17h ago
-
Senior Data Engineer INR 2755K-3500KAPI Development | AWS | Algorithms | Amazon Redshift | AzureHybrid working | Learning and development programmesSenior-level Full TimeHyderabad, India23h ago
-
Senior-level Full TimeIndia - Pune1d ago
-
GCP Data Architect INR 1500K-2450KBigQuery | CI/CD | Cloud Build | Cloud Composer | Cloud MonitoringSenior-level Full TimeIndia - Pune1d ago
-
Senior Director, Engineering INR 3100K-4500KAWS | Analytics | Batch Processing | Data Architecture | Data EngineeringSenior-level Full TimeBengaluru1d ago
-
Engineering Lead - Data & Analytics INR 3000K-4800KAWS | Agentic Analytics | Agile | Artificial Intelligence | Automated DeploymentSenior-level Full TimePune, MAHĀRĀSHTRA, India1d ago
-
Data & Analytics Cloud Manager/Architect INR 2500K-4000KAPIs | Automated retraining | BigQuery | CI/CD | Cloud AISenior-level Full TimeIndia1d ago
-
Senior-level Full Timebengaluru , India1d ago
-
Senior-level Full Timebengaluru , India1d ago
-
Data Architect INR 2200K-3500KArchitecture Documentation | Batch Processing | Cloud Composer | Cloud Dataflow | Cloud DataprocSenior-level Full TimePune, Maharashtra, India1d ago
-
Databricks & GCP Data Platform Architect INR 1500K-2040KAlerting | Apache Spark | Audit Logging | BigQuery | CI/CDSenior-level Full TimeHyderabad, //TS, India1d ago
-
EY - GDS Consulting - AI And DATA - AWS Databricks - Senior INR 2400K-3500KAWS | AWS Glue | AWS Lambda | Amazon DynamoDB | Amazon EC2Career development opportunities | Coaching and feedback | Flexible workingSenior-level Full TimeHyderabad, TG, IN, 5000811d ago
-
Data Architect INR 2000K-3000K.Net Core | Agile | C# | Continuous Deployment | Continuous integrationSenior-level Full TimePune, PDC2C, India1d ago
-
Senior-level Full TimePune, PDC2C, India1d ago
-
Data Modeler INR 2520K-4000KData Governance | Data Modeling | Data Quality | Data Warehouse | Data integrationSenior-level Full TimeBengaluru, BDC14A, India1d ago
-
Data & Analytics Lead INR 2520K-3380KAPI Integration | Analytics Roadmapping | Automation | Azure Data | Azure Data LakeSenior-level Full TimeBengaluru - India1d ago
-
Senior-level Full TimeBengaluru, India1d ago
-
Enterprise Data & Analytics Lead, Data Engineering INR 2800K-4000KAWS | Agile | Airbyte | Airflow | CI/CDEmployee resource groups | Learning and development | Paid Company Holidays | Paid vacation | Volunteer timeSenior-level Full TimeIN025 Embassy Manyata Business Park, India1d ago
-
Lead Data Architect INR 2400K-3500KAPI Design | Azure Fabric | Azure Stream | Azure Stream Analytics | Data ModelingHybrid workSenior-level Full TimeIN025 Embassy Manyata Business Park, India1d ago
-
Lead Data Architect INR 2500K-4000KAmazon S3 | Amazon Web Services | Batch Processing | CI/CD | DBTSenior-level Full TimePune DIA, India1d ago
-
Senior-level Full TimeIN KA Bengaluru, India1d ago
-
Business Operations Specialist, Senior INR 3000K-4000KAPI Integration | Agile | Airflow | Azure DevOps | Batch integrationSenior-level Full TimeBangalore Office, India1d ago
-
Anaplan | Azure | DAX | Data Governance | Data ModelingMid-level Full TimeBangalore Karle Town SEZ, India1d ago