Databrick Solution Architect
Tasks
- Build scalable data engineering pipelines using Databricks
- Configure clusters jobs workflows and repos
- Define medallion architecture
- Design Databricks Lakehouse architecture
- Design batch and streaming pipelines
- Develop PySpark transformations
- Develop SQL based transformations
- Implement CI CD pipelines for Databricks deployments
- Implement Delta Lake based data models
- Implement Unity Catalog governance model
- Lead code reviews and architecture reviews
- Optimize Databricks performance and cost
- Provide hands on support during development
Perks/Benefits
- N/A
Skills/Tech-stack
Batch Processing | CD pipelines | CI/CD | CI/CD pipelines | Cluster management | Data Engineering | Data Pipeline Architecture | Data pipeline | Databricks | Delta Lake | Lakehouse Architecture | Pipeline architecture | PySpark | SQL | Streaming | Unity Catalog
Education
N/A
Related jobs
-
ESI Central BI 166816 INR 1500K-2146KAWS Redshift | Amazon Web Services | DBT | Data Lake | Data ModelingSenior-level Full TimeHyderabad, IND-TG, India1h ago
-
Analytics Solution- Associate INR 2040K-2755KAgile | Atlassian | Atoti | Change Data Capture | Continuous DeploymentSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Senior Manager, Data Engineering Service & SRE INR 2000K-3500KAnomaly Detection | Apache Airflow | Automated testing | BigQuery | CloudSenior-level Full TimeBengaluru, IN, 56010310h ago
-
Director - Data Engineering and ML INR 1500K-6000KArtificial Intelligence | Batch Processing | Data Architecture | Data Compliance | Data EngineeringHybrid work environmentExecutive-level Full TimeBengaluru, INDIA, India15h ago
-
IFC Data - Data Architect INR 2000K-3500KCloud Computing | Data Architecture | Data Governance | Data Lineage | Data ModellingSenior-level Full TimePune, Gera Commerzone SEZ, India19h ago
-
Data Engineering Advisor - HIH - Evernorth INR 1500K-1700KAWS IAM | AWS Lambda | AWS Step Functions | Agile | Amazon EC2Mid-level Full TimeHIH - Hyderabad, India19h ago
-
Software Engineering Lead Analyst - HIH - Evernorth INR 2000K-2040KAWS | Apache Airflow | Azure | Data Lake | Data QualitySenior-level Full TimeHIH - Hyderabad, India19h ago
-
Senior Data Engineer INR 2755K-3500KAPI Development | AWS | Algorithms | Amazon Redshift | AzureHybrid working | Learning and development programmesSenior-level Full TimeHyderabad, India1d ago
-
Senior-level Full TimeIndia - Pune1d ago
-
GCP Data Architect INR 1500K-2450KBigQuery | CI/CD | Cloud Build | Cloud Composer | Cloud MonitoringSenior-level Full TimeIndia - Pune1d ago
-
Senior Director, Engineering INR 3100K-4500KAWS | Analytics | Batch Processing | Data Architecture | Data EngineeringSenior-level Full TimeBengaluru1d ago
-
Engineering Lead - Data & Analytics INR 3000K-4800KAWS | Agentic Analytics | Agile | Artificial Intelligence | Automated DeploymentSenior-level Full TimePune, MAHĀRĀSHTRA, India1d ago
-
Data & Analytics Cloud Manager/Architect INR 2500K-4000KAPIs | Automated retraining | BigQuery | CI/CD | Cloud AISenior-level Full TimeIndia1d ago
-
Senior-level Full Timebengaluru , India1d ago
-
Senior-level Full Timebengaluru , India1d ago
-
Data Architect INR 2200K-3500KArchitecture Documentation | Batch Processing | Cloud Composer | Cloud Dataflow | Cloud DataprocSenior-level Full TimePune, Maharashtra, India1d ago
-
Databricks & GCP Data Platform Architect INR 1500K-2040KAlerting | Apache Spark | Audit Logging | BigQuery | CI/CDSenior-level Full TimeHyderabad, //TS, India1d ago
-
EY - GDS Consulting - AI And DATA - AWS Databricks - Senior INR 2400K-3500KAWS | AWS Glue | AWS Lambda | Amazon DynamoDB | Amazon EC2Career development opportunities | Coaching and feedback | Flexible workingSenior-level Full TimeHyderabad, TG, IN, 5000811d ago
-
Data Architect INR 2000K-3000K.Net Core | Agile | C# | Continuous Deployment | Continuous integrationSenior-level Full TimePune, PDC2C, India1d ago
-
Senior-level Full TimePune, PDC2C, India1d ago
-
Data Modeler INR 2520K-4000KData Governance | Data Modeling | Data Quality | Data Warehouse | Data integrationSenior-level Full TimeBengaluru, BDC14A, India1d ago
-
Data & Analytics Lead INR 2520K-3380KAPI Integration | Analytics Roadmapping | Automation | Azure Data | Azure Data LakeSenior-level Full TimeBengaluru - India1d ago
-
Senior-level Full TimeBengaluru, India1d ago
-
Enterprise Data & Analytics Lead, Data Engineering INR 2800K-4000KAWS | Agile | Airbyte | Airflow | CI/CDEmployee resource groups | Learning and development | Paid Company Holidays | Paid vacation | Volunteer timeSenior-level Full TimeIN025 Embassy Manyata Business Park, India1d ago
-
Lead Data Architect INR 2400K-3500KAPI Design | Azure Fabric | Azure Stream | Azure Stream Analytics | Data ModelingHybrid workSenior-level Full TimeIN025 Embassy Manyata Business Park, India1d ago