Lead Data Engineer
Tasks
- Build and maintain CI/CD workflows
- Build and operate Python batch and streaming workflows
- Define and apply data quality observability and documentation standards
- Design and implement AWS data solutions
- Develop internal tools and self serve workflows
- Ensure data protection and access controls
- Improve platform resilience with recovery retry and backfill
- Lead design build and evolution of data pipelines
- Own data engineering stack improvements
- Review designs and code to raise data engineering quality
- Translate requirements into trusted datasets and data models
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Athena | Aurora PostgreSQL | Backfill | Batch Processing | CI/CD | Data Modeling | Data Quality | Database permissions | Deployment | Glue | IAM | Idempotent Processing | Incident Resolution | Infrastructure as Code | Lambda | Monitoring | Observability | Orchestration | Performance optimization | Python | Redshift | Replay | Retry and Recovery | S3 | Schema Design | Secrets Handling | Secure Secrets Handling | Secure secrets | Streaming Data | Testing | “as-code”
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Senior Data Engineer
Related jobs
-
AI Research Engineer GBP 110K-200KC# | CUDA | Deep learning | Machine Learning | PyTorchHybrid Remote | Remote Interview AccommodationMid-level Full TimeHybrid (UK) R8h ago
-
CSS | Data Modeling | ETL | HTML | JavaScriptSenior-level Contract TemporaryRemote or Hybrid (Finchley, North London … R9h ago
-
AI Data Engineer GBP 111K-133KAgent memory | Amazon Web Services | Audio Processing | Batch Processing | Cloud infrastructureAccess to AI tools | Equity | OwnershipMid-level Full TimeLondon13h ago
-
Lead Data Platform Engineer GBP 84K-110KAmazon Web Services | Continuous Delivery | Continuous Integration/Continuous Delivery | Continuous integration | DatabricksSenior-level Full TimeLondon, UK, United Kingdom14h ago
-
Senior Analytics Engineer GBP 60K-70KCommon table expressions | DBT | Data Quality | Data Testing | Data WarehousingOn-going training | Skills for lifeSenior-level Full TimeLondon, GB14h ago
-
Principal Data Engineer GBP 70K-89KAWS | AWS Glue | Airflow | ArgoCD | AvroAutonomy | Influence | Mentorship | Professional growthSenior-level Full TimeLondon, England, United Kingdom14h ago
-
Insights Product Manager - Analytics Engineering GBP 50K-68KAmplitude | Anomaly alerting | CI/CD | DBT | Data CatalogAnnual leave | Counselling access | Employee assistance program | Free Economist content access | Moving home supportMid-level Full TimeLondon - Commercial R16h ago
-
Agent Orchestration | Authentication | Backend Development | CI/CD | Frontend DevelopmentBPSS clearance | Hybrid workSenior-level Contract Full TimeSheffield, England, United Kingdom19h ago
-
Senior Solutions Engineer - Qatar & S.Africa Fly-in GBP 70K-100KAI Agents | AWS | Apache Spark | Apache Spark architecture | Artificial IntelligenceHybrid work schedule | Travel for customer visits and events | Workshops seminars and community buildingSenior-level Full TimeLondon, United Kingdom; Paris, France R22h ago
-
Analytics Senior Lead - London GBP 60K-70KA/B | A/B Testing | Amazon Redshift | Attribution | B testingAI enterprise license | Annual bonus | Commuter benefits | Fertility and family support | HealthcareSenior-level Full TimeLondon, UK23h ago
-
Data Engineer GBP 87K-111KApache Airflow | DBT | Data Modeling | Data Pipeline Monitoring | Data QualityPaid 'me days' | Paid parental leave | Paid paternity leave | Paid sabbatical leave | RSU equitySenior-level Full TimeLondon, United Kingdom23h ago
-
APIs | AWS | Automation Pipelines | Azure | Backend DevelopmentAnnual leave | Company lunches | Learning and development budget | Monthly socials | Public holidaysEntry-level Full TimeLondon1d ago
-
Alerting | Cause analysis | Completeness Accuracy Timeliness Consistency | Data Governance | Data MonitoringSenior-level Full TimeLondon1d ago
-
Cloud Computing | Distributed Systems | Go | Infrastructure as Code | Kubernetes100% remote | Annual leave | Bonus | Equity | Shutdown daysSenior-level Full TimeUnited Kingdom (Remote) R1d ago
-
Data Science Lead - AML Risk GBP 75K-115KData Pipelines | Experiment design | Language Models | Language Processing | Large Language ModelsSenior-level Full TimeLondon, United Kingdom1d ago
-
Data Engineer GBP 45K-50KApache Airflow | Batch Data Processing | Batch data | BigQuery | Cloud platformCompany performance bonus | Employee assistance | Enhanced family leave | Flexible working | Gym membershipMid-level Full TimeLondon, England, United Kingdom1d ago
-
Infrastructure and MLOps Engineer GBP 91K-110KAWS | CI/CD | Docker | Kubernetes | LinuxAnnual leave | Barista bar | Dental plan | Employee assistance programme | Flexible workingSenior-level Full TimeBristol, UK1d ago
-
DevOps Engineer GBP 45K-70KAWS | AWS SSM | Ansible | Bash | CI/CDFlexible scheduling for sprint cycles | Hybrid work environmentMid-level Full TimeUnited Kingdom1d ago
-
Senior Machine Learning Engineer GBP 55K-75KApache Spark | CI/CD | Canary deployments | Data Engineering | Data PipelinesFlexible working | Hybrid workingSenior-level Full TimeBristol, UK1d ago
-
AI Research Engineer - Computer Vision GBP 65K-90KAdversarial Attacks | Computer Vision | Deep learning | Edge Computing | Hardware optimizationEnhanced parental leave | Gym membership | Learning allowance | Mental health support | Paid Family Emergency LeaveSenior-level Full TimeBerlin; London; Munich R1d ago
-
AWS | Apache Airflow | Apache Kafka | Apache Spark | CI/CDCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Flexible remote workSenior-level Full TimeManchester, United Kingdom1d ago
-
Quantitative Developer HFT GBP 150K-200KAlgorithms | C# | C++ | Data Structures | GoConference budget | Free meals | Pension scheme contributions | Private health insurance | Training budgetMid-level Full TimeLondon, England, United Kingdom1d ago
-
Senior AI Engineer GBP 70K-90KAPI Development | AWS | Agent Orchestration | Apache Spark | CI/CDCentral London office | Enhanced parental leave | Enhanced pension | Free meals and snacks | Life insuranceSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Bilby | CAMB | Class | DOLFINx | FenicsPart-time freelance | Project based workSenior-level FreelanceUnited Kingdom - Remote R1d ago
-
Data Engineering Technical Lead GBP 60K-70KAd Hoc Queries | Ad-Hoc | Data Integrity | Data Modeling | Data ProcessingHybrid work model | Mentorship | Professional developmentSenior-level Full TimeGBR - Manchester, United Kingdom1d ago