Lead Data Engineer
Tasks
- Build and maintain CI/CD workflows
- Build and operate Python batch and streaming workflows
- Define and apply data quality observability and documentation standards
- Design and implement AWS data solutions
- Develop internal tools and self serve workflows
- Ensure data protection and access controls
- Improve platform resilience with recovery retry and backfill
- Lead design build and evolution of data pipelines
- Own data engineering stack improvements
- Review designs and code to raise data engineering quality
- Translate requirements into trusted datasets and data models
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Athena | Aurora PostgreSQL | Backfill | Batch Processing | CI/CD | Data Modeling | Data Quality | Database permissions | Deployment | Glue | IAM | Idempotent Processing | Incident Resolution | Infrastructure as Code | Lambda | Monitoring | Observability | Orchestration | Performance optimization | Python | Redshift | Replay | Retry and Recovery | S3 | Schema Design | Secrets Handling | Secure Secrets Handling | Secure secrets | Streaming Data | Testing | “as-code”
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Senior Data Engineer
Related jobs
-
Sr Staff AI Software Development Engineer GBP 55K-61KAWS | Artificial Intelligence | Azure | Databricks | Direct Preference OptimizationAccrued Paid Vacation | Commuter benefits | Dental insurance | Employee assistance program | Employee resource groupsSenior-level Full TimeCambridge, United Kingdom R12h ago
-
Forward Deployed AI Engineer GBP 111K-125KAPI Integration | Agent Orchestration | Conversational AI | Hugging Face | Langchain401k company match | Community involvement and volunteering events | Dental insurance | Disability benefits | Flexible paid time offMid-level Full TimeLondon, England, United Kingdom16h ago
-
Senior Machine Learning Engineer GBP 100K-115KAmazon SageMaker | Bayesian Modeling | Cost Optimization | Data Versioning | Deep learningDog-friendly office | Enhanced parental leave | Fertility support | Flexible working | Free mealsSenior-level Full TimeLondon (Shoreditch) R18h ago
-
Analytics Engineer GBP 73K-80KDBT | Data Modeling | Data Pipelines | Data Quality | MonitoringCycle to work scheme | Employer pension contribution | Home office setup budget | Hybrid working | Learning budgetMid-level Full TimeLondon18h ago
-
Analytics & Data Engineer GBP 47K-84KAutomation | Azure ML | Data Governance | Data Lineage | Data ModelingFlexible office collaboration | Remote work flexibilityMid-level Full TimeGloucester, England, United Kingdom18h ago
-
Lead Analytics Engineer GBP 90K-110KAccess Control | Amazon Redshift | Apache Superset | CI/CD | ClickHouseSenior-level Full TimeBristol, UK20h ago
-
Lead MLOps Engineer GBP 54K-54KCI/CD | GitLab CI | GitLab CI/CD | Grafana | KubernetesEmployee share plans | Flexible working | Hybrid working | Pension scheme | Private health plansSenior-level Full TimeGB-ENG-LAN-Preston22h ago
-
Lead Data Engineer (Cloud) GBP 55K-55KAirflow | Apache Spark | Azure | CosmosDB | Data GovernanceAnnual incentive | Employee share plans | Flexible working arrangements | Green Car Scheme | Health wellbeing and lifestyle benefitsSenior-level Full TimeGB-ENG-LAN-Preston22h ago
-
Forward Deployed Engineer III, GenAI, Google Cloud GBP 100K-130KAI APIs | Agent systems | Cloud platform | CrewAI | Data PipelinesSenior-level Full TimeLondon, UK; Paris, France23h ago
-
Distributed Systems | JAX | Machine Learning | Performance Analysis | ProfilingSenior-level Full TimeLondon, UK23h ago
-
Infrastructure Engineer (Data & Automations) GBP 105K-140KAI Agents | CI/CD | Cloud infrastructure | Container Orchestration | DBTAnnual company offsite | Co-working stipend | Learning and development stipend | Social travel stipendMid-level Full TimeUnited Kingdom R1d ago
-
Data Quality & Governance Engineer GBP 72K-80KAWS | AWS CodeBuild | AWS CodePipeline | AWS Glue | AWS LambdaEmployer pension contribution | Gym discounts | Health insurance | Hybrid working | Income protection insuranceMid-level Full TimeLondon.1d ago
-
Senior Applied Scientist GBP 205K-205KBlack | Cloud Computing | Complex Planning | Deep learning | GitHybrid work | Mentoring and coaching | Share optionsSenior-level Full TimeLondon1d ago
-
Staff Applied Scientist GBP 225K-340KAI architectures | Cloud Computing | Data Analysis | Deep learning | GitHybrid work scheme | Mentoring and coaching | Professional development | Stock optionsSenior-level Full TimeLondon1d ago
-
GenAI Engineer Global Link - Assistant Vice President GBP 195K-280KAgile | CI/CD | Clean Code | Cloud Native | Cloud Native ArchitectureEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysExecutive-level Full TimeLondon, England, United Kingdom1d ago
-
Senior GenAI Engineer Global Link ,Vice President GBP 85K-109KAgentic Frameworks | Agile | Autonomous Agents | CI/CD | Cloud NativeEmployee resource networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Mid-level Full TimeLondon, United Kingdom1d ago
-
API Data Engineer GBP 55K-72KAPI Integration | Apache Spark | Automation | Cloud Data | Cloud Data PlatformsHybrid workMid-level Full TimeEdinburgh, United Kingdom1d ago
-
Analytics & Data Engineer GBP 47K-84KAutomation | Azure | Azure Machine Learning | Cloud Computing | Data GovernanceIn person collaboration once or twice monthly | Remote work flexibility | Security clearance supportMid-level Full TimeGloucester, England, United Kingdom1d ago
-
Lead Data Analyst - Treasury (FX Markets) GBP 75K-100KDBT | Data Analysis | Data Modeling | Data Visualization | Data WarehousingHybrid working | Paid annual holiday | Paid sick days | Parental leave | SabbaticalSenior-level Full TimeLondon, United Kingdom1d ago
-
Senior Big Data Engineer (Java Focus) GBP 70K-87KAnsible | Apache Airflow | Apache Flink | Apache Hadoop | Apache IcebergHybrid workSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Senior Analytics Engineer GBP 70K-100KAmazon Redshift | BigQuery | Cloud Data | Cloud data warehousing | CubeCorporate discounts | Dog-friendly office | Flexible hours | Referral bonuses | Tech equipmentSenior-level Full TimeUnited Kingdom Remote R1d ago
-
Data Engineer GBP 45K-65KAccess Control | Apache Airflow | Apache Spark | Azure Data | Azure Data FactoryMid-level Full TimeLondon1d ago
-
Data Science Lead - AML Risk GBP 75K-100KExperiment design | Machine Learning | OOP | Precision-recall | PythonSenior-level Full TimeLondon, United Kingdom1d ago
-
Staff Software Engineer, Games, Inception, DeepMind GBP 225K-255KAI Model Training | AI model | Agentic Workflows | Compliance | Data ManagementSenior-level Full TimeLondon, UK1d ago