Senior Databricks Solution Architect

Tasks

• Build batch and real time ingestion frameworks
• Collaborate with data teams
• Configure access controls
• Create architecture documentation and conduct design reviews
• Define Databricks CI CD best practices
• Define DevOps and environment management
• Define workspace architecture
• Design Databricks lakehouse solutions
• Design Delta Lake architecture
• Design cluster strategies
• Implement Databricks governance and security
• Implement cost optimization
• Implement data lineage and auditing
• Implement data pipelines
• Implement data quality frameworks
• Implement schema evolution and optimization
• Lead Databricks implementations
• Lead migration to Databricks
• Optimize clusters and Spark jobs
• Perform Databricks performance tuning
• Set up Unity Catalog
• Tune queries

Perks/Benefits

• N/A

Skills/Tech-stack

Amazon Web Services | Apache Spark | Audit Logging | Azure | Cluster management | Data Governance | Data Lakehouse | Data Lineage | Databricks | Databricks Machine Learning | Databricks Repos | Databricks Workflows | Delta Lake | ELT | ETL | Feature Store | Google Cloud | Jobs Orchestration | Kafka | MLOps | MLflow | Machine Learning | Model Deployment | Notebooks | PySpark | SQL | Scala | Schema evolution | Structured Streaming | Unity Catalog | Web Services