Azure Databricks Engineering Lead
Tasks
- Architect end-to-end data pipelines
- Ensure performance scalability security and governance best practices
- Implement Delta Lake and data modeling
- Implement data quality frameworks
- Lead design development and optimization of Azure Databricks data platforms
- Lead requirements discussions solution walkthroughs and architecture reviews
- Mentor data engineers review designs code and deployments
- Provide stakeholder status updates manage expectations and flag risks
- Support release planning sprint ceremonies and delivery governance
- Tune Spark performance optimize costs manage clusters
Perks/Benefits
Skills/Tech-stack
Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure Governance | Azure SQL | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Lake Storage Gen2 | Data Modeling | Delta Lake | DevOps | ELT | ETL | Encryption | PySpark | RBAC | SQL | Scala | Synapse Analytics
Education
N/A
Related jobs
-
Partnership Analytics Lead - 14 Months FTC GBP 62K-80KAzure | Copilot Studio | DBT | Microsoft Fabric | Power AutomateHybrid work modelSenior-level ContractLondon, England, United Kingdom R20h ago
-
Data Engineering Manager GBP 45K-56KBigQuery | CI/CD | DBT | Data Vault | Data meshFlexible benefits | Flexible working | Learning and development budget | Life assurance | Matched contribution pension schemeMid-level Full TimeLondon, England, United Kingdom R22h ago
-
Lead Data Engineer GBP 45K-60KAPIs | Analytics | Ansible | Azure | Cloud platformHybrid work environment | Learning and developmentSenior-level Full TimeUK - Hook, United Kingdom1d ago
-
Engineering - Data Engineer - Vice President - London GBP 121K-146KApache Iceberg | Apache Spark | Avro | CI/CD | ContainerizationExecutive-level Full TimeLondon, Greater London, England, United Kingdom1d ago
-
Lead AI Analyst GBP 64K-80KAPI Integration | Anomaly Detection | Azure AI | Code review | Computer VisionSenior-level Full TimeLeicester, Leicestershire, United Kingdom1d ago
-
Senior Manager, Data Engineering and Operations GBP 78K-102KAgile delivery | Automated testing | Compliance | Continuous Delivery | Continuous integrationAccess to private GP 24 7 | Company car scheme | Discounted products | Holiday purchase scheme | Paid carer’s leaveSenior-level Full TimeLondon, GB, E1 8EP2d ago
-
Lead Data Scientist GBP 46K-52KAgile | Computer Vision | Data Engineering | Data Visualization | Ethical AIFlexible work schedule | Hybrid working | Professional developmentSenior-level Full TimeLeeds, United Kingdom2d ago
-
Lead Data Engineer GBP 80K-95KAWS | Agile | Apache Spark | Azure | CI/CDAnnual leave | Cycle to work scheme | Discounted gym membership | Employee clubs and events | Employee feedback and performance reviewsSenior-level Full TimeEdinburgh2d ago
-
Lead Data Analyst GBP 60K-70KAnalytics Strategy | Business Intelligence | Dashboard Design | Data Governance | Data StorytellingSenior-level Full TimeCoventry Central Office, United Kingdom2d ago
-
Lead Data Analyst - Treasury (FX Markets) GBP 75K-100KDBT | Data Modeling | Data Visualization | Experimental Design | PythonHybrid working | Paid annual leave | Paid sabbatical | Paid sick leave | Parental leaveSenior-level Full TimeLondon, United Kingdom2d ago
-
Lead Data Scientist GBP 78K-100KAWS | Agentic AI | Azure | Data Pipelines | EmbeddingsDiverse team culture | Growth opportunities | Inclusive culture | International careersSenior-level Full TimeLondon2d ago
-
Data Science Manager, Supply Marketplace GBP 45K-57KAllocation | Causal Inference | Experimentation | Forecasting | Liquidity modelingBenefits | Remote work within UKMid-level Full TimeLondon or Remote within UK R2d ago
-
Coach - Data Engineer Level 5 GBP 55K-70KAWS | Apache Airflow | Apache Spark | Azure | Batch dataCertification support | Flexible working | Fully remote | Professional development | Travel as requiredSenior-level Full TimeLondon, United Kingdom R2d ago
-
Lead Data Engineer GBP 90K-106KApache Spark | Azure | Data Migration | Data Modeling | Data cleaningSenior-level Full TimeUK - London3d ago
-
AWS Lead Data Engineer GBP 86K-106KAWS Glue | AWS Lambda | AWS S3 | Agile | Amazon CloudWatchAnnual leave | Corporate discounts | Flexible working hours | Hybrid work | Private dental insuranceSenior-level Full TimeLondon - 10 Triton Street, United …3d ago
-
Bordereaux | Data Governance | Data Modeling | Data Quality | InsuranceCycle scheme | Discounts and offers | Income protection insurance | Learning opportunities | PensionSenior-level Full TimeLondon, England, United Kingdom3d ago
-
Revenue Operations Data Engineering Partner GBP 60K-80KAutomation | B2B SaaS | B2B SaaS Revenue Mechanics | CRM Integrations | CRM hygieneSenior-level Full TimeLondon, United Kingdom4d ago
-
Senior-level Full TimeLondon, United Kingdom4d ago
-
Lead Data Scientist GBP 85K-85KAmazon SageMaker | Artificial Intelligence | Azure ML | CI/CD | ContainersContributory pension scheme | Development days | Discount on insurance products | Flexible buy sell holiday options | Flexible working optionsSenior-level Full TimeDatchet, GB, SL3 9LL4d ago
-
Lead Digital Analytics Engineer (Google) GBP 70K-90KAttribution Modelling | BigQuery | Consent Management | CookiePro | Data LayerHybrid work | International mobility | Learning opportunities | Professional development | Remote workSenior-level Full TimeLondon5d ago
-
Lead Machine Learning Engineer GBP 70K-85KAI Pipelines | BigQuery | CI/CD | Cloud Build | Cloud ComposerDental cover | Employee wellbeing programme | Enhanced parental leave | Hybrid work | Learning budgetSenior-level Full TimeLondon5d ago
-
Lead Data Analyst - Operations GBP 75K-100KCapacity Planning | Cause and Effect | Cause and effect analysis | Data Modeling | Data PipelinesSenior-level Full TimeLondon, United Kingdom7d ago
-
Lead Digital Analyst – Customer Analytics GBP 49K-64KAWS | Adobe Analytics | Adobe Target | Behavioral Analysis | Customer AnalyticsAdditional holiday purchase | Annual holiday allowance | Colleague discount | Cycle to work scheme | Discounts from retailersSenior-level Full TimeLondon, London, United Kingdom7d ago
-
Lead Digital Analyst – Marketplace Analytics GBP 52K-64KAWS | Adobe Analytics | Adobe Target | Analytics Insight Generation | Data StorytellingAnnual holiday allowance | Buy additional holiday | Colleague discount | Cycle to work scheme | Discounts from retailersSenior-level Full TimeLondon, London, United Kingdom7d ago
-
Lead Data Analyst - Wise Platform Pricing GBP 85K-110KDBT | Data Modeling | Data Visualization | Python | SQLSenior-level Full TimeLondon, United Kingdom7d ago