Lead Data Engineer (Cloud)
Tasks
- Build ingestion pipelines into lakehouse
- Develop Spark transformation jobs
- Implement manifest driven transformations
- Maintain Delta Lake tables
- Orchestrate workflows in Airflow
- Troubleshoot pipeline reliability and performance
Perks/Benefits
- Annual incentive
- Employee share plans
- Flexible working arrangements
- Green Car Scheme
- Health wellbeing and lifestyle benefits
- Hybrid work arrangements
- Pension scheme
- Private health plans
- Shopping discounts
Skills/Tech-stack
Airflow | Apache Spark | Azure | CosmosDB | Data Governance | Data Quality | Data pipeline | Data pipeline orchestration | Databricks | Delta Lake | JSON | Lakehouse | Metadata Management | MongoDB | Pipeline Orchestration
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Analytics & Data Engineer GBP 47K-84KAutomation | Azure | Azure Machine Learning | Cloud Computing | Data GovernanceIn person collaboration once or twice monthly | Remote work flexibility | Security clearance supportMid-level Full TimeGloucester, England, United Kingdom1d ago
-
Senior Big Data Engineer (Java Focus) GBP 70K-87KAnsible | Apache Airflow | Apache Flink | Apache Hadoop | Apache IcebergHybrid workSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Senior Analytics Engineer GBP 70K-100KAmazon Redshift | BigQuery | Cloud Data | Cloud data warehousing | CubeCorporate discounts | Dog-friendly office | Flexible hours | Referral bonuses | Tech equipmentSenior-level Full TimeUnited Kingdom Remote R1d ago
-
Data Engineer GBP 45K-65KAccess Control | Apache Airflow | Apache Spark | Azure Data | Azure Data FactoryMid-level Full TimeLondon1d ago
-
Data Analyst - Energy GBP 26K-26KBigQuery | CI/CD | Data Modeling | Data Pipelines | Data QualityFlexible working | Four day week option | Gym membership allowance | Inclusive culture | Learning and development budgetMid-level Full TimeLondon, England, United Kingdom R1d ago
-
Senior Analytics Manager - FinCrime GBP 75K-100KAnalytics Strategy | Business Intelligence | Data Analysis | Data Modeling | Data QualitySenior-level Full TimeLondon, United Kingdom1d ago
-
Associate Data Developer GBP 55K-65KAnomaly Detection | Artificial Intelligence | Classification | Data Mapping | Data TransformationDeath in service funds | Gym membership discount | Health cashback plan | Income protection scheme | Purchase holiday schemeMid-level Full TimeLondon, England1d ago
-
ML Engineer (Binding) GBP 74K-100KAWS | Azure | Cloud Computing | Computational Drug Discovery | Deep learningCycle to work scheme | Option plan | Pension scheme | Private medical insurance | Remote WorkingSenior-level Full TimeLondon, England, United Kingdom1d ago
-
AI Consultant (Principal AI Engineer) GBP 81K-109KAPI Design | AWS | Agent Orchestration | Agile | AzureSenior-level Full TimeLondon, England (Angel Lane), United Kingdom1d ago
-
Data Engineer, Vice President GBP 111K-146KAWS | Apache Airflow | Azure | CI/CD | DBTEducation reimbursement | Family support programs | Flexible time off | Health and wellness resources | Hybrid work modelExecutive-level Full TimeL14 - Verde Building, 10 Bressenden … R1d ago
-
AI Engineer GBP 59K-66KAWS | Azure | Data Governance | Data Science | DatabricksAnnual leave | Critical illness cover | Flexible working arrangements | Life assurance | Paid parental leaveMid-level Full TimeKildean, United Kingdom1d ago
-
Lead Platform Engineer GBP 70K-85KAWS Step Functions | Airflow | Amazon EKS | Amazon SageMaker | CI/CDSenior-level Full TimeGlasgow Campus, United Kingdom1d ago
-
Head of AI Strategy GBP 80K-95KAI Governance | AI ethics | Artificial Intelligence | Data Governance | Data SecurityAccess to discounts | Birthday day off | Car allowance | Discount shopping portal | Holiday 25 days plus Bank HolidaysExecutive-level Full TimeSamsung House 2000 Hillswood Drive, Chertsey, …1d ago
-
Arrears Management | Behavioral analytics | Customer Segmentation | Data Analysis | Data GovernanceBuy/sell annual leave | Cycle to work | Electric vehicle leasing | Flexible working | Gym partnershipsMid-level Full TimeCardiff, Wales, United Kingdom1d ago
-
Data Engineer - Products GBP 49K-52KAWS S3 | Agile | Amazon Athena | Apache Spark | JiraCharity volunteering support | Coaching | Cycle to work scheme | Flexible work schedule | Free parkingMid-level Full TimeManchester, United Kingdom1d ago
-
Cloud Operations Analyst GBP 36K-54KAWS | Azure | Bash | Docker | EC2Company equity | Health insurance | Hybrid work | Life insurance | Mental health supportMid-level Full TimeManchester1d ago
-
Software Engineer -System Performance, Robot Software GBP 117K-157KBottleneck analysis | C++ | CPU caches | Computer Architecture | Computer Performance BenchmarkingCore working hours | Hybrid work | Work from home flexibilitySenior-level Full TimeLondon1d ago
-
Data Engineer II GBP 50K-60KAzure DevOps | Cause analysis | Data Architecture | Data Modeling | Data ObservabilityMid-level Full TimeNetherlands, Remote; Spain, Remote; United Kingdom, … R2d ago
-
AI Engineer - Retail (LH) GBP 65K-80KAI Foundry | Azure | Azure AI | Azure AI Foundry | Azure ContainerColleague discount | Employee assistance programme | Flexible working | Gym membership discounts | Holiday allowanceSenior-level Full TimeLondon, United Kingdom2d ago
-
Senior-level Full TimeUnited Kingdom, London; United Kingdom, Remote R2d ago
-
Senior-level Full TimeFrance, Remote; Germany, Remote; Netherlands, Remote; … R2d ago
-
Senior Analytics Engineer GBP 65K-81KAzure Fabric | Conformed Dimensions | DAX | Data Lake | Data Lineage25 days annual leave | Buy and sell holidays | Confidential Employee Assistance Programme | Flexible benefits | Further education and training supportSenior-level Full TimeLondon2d ago
-
Lead Data Architect GBP 86K-106KData Architecture | Data Engineering | Data Governance | Data Lineage | Data ModelingHybrid workingSenior-level Full TimeNewbury, England, United Kingdom2d ago
-
Access Control | Authorization | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Contract Full Time TemporaryBirmingham, England, United Kingdom2d ago
-
Graduate Machine Learning Engineer GBP 32K-32KAgile | Azure Machine Learning | Databricks | DevOps | DockerExpert supervision | Hybrid workingEntry-level Full TimeManchester, Greater Manchester, United Kingdom2d ago