Lead Data Engineer
Tasks
- Build and maintain CI/CD workflows
- Build and operate Python batch and streaming workflows
- Define and apply data quality observability and documentation standards
- Design and implement AWS data solutions
- Develop internal tools and self serve workflows
- Ensure data protection and access controls
- Improve platform resilience with recovery retry and backfill
- Lead design build and evolution of data pipelines
- Own data engineering stack improvements
- Review designs and code to raise data engineering quality
- Translate requirements into trusted datasets and data models
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Athena | Aurora PostgreSQL | Backfill | Batch Processing | CI/CD | Data Modeling | Data Quality | Database permissions | Deployment | Glue | IAM | Idempotent Processing | Incident Resolution | Infrastructure as Code | Lambda | Monitoring | Observability | Orchestration | Performance optimization | Python | Redshift | Replay | Retry and Recovery | S3 | Schema Design | Secrets Handling | Secure Secrets Handling | Secure secrets | Streaming Data | Testing | “as-code”
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Senior Data Engineer
Related jobs
-
AI Engineer GBP 45K-50KAI Search | Azure AI | Azure AI Search | Azure OpenAI | DatabricksAbility to purchase additional annual leave | Annual leave | Discretionary bonus scheme | Employee assistance program | Employee share purchase planMid-level Full TimeLondon, United Kingdom7h ago
-
Reporting & Analytics Lead GBP 61K-61KAnalytics Cloud | Analytics Strategy | Data Governance | Data Quality | DatabricksAccident insurance | Annual leave | Family-friendly leave | Flexible benefits | Life assuranceSenior-level Full TimeLancaster House, United Kingdom16h ago
-
Mid-level Full TimeCanary Wharf, 1 Churchill Place, United …16h ago
-
Software Engineer - Data Infrastructure GBP 45K-52KAWS | Amazon SQS | Batch Processing | BigQuery | Data IngestionMid-level Full TimeLondon19h ago
-
API Management | Apache Spark | Azure API | Azure API Management | Azure DataSenior-level Contract Full TimeLeicester, Leicestershire, United Kingdom1d ago
-
Mid-level Full TimeTrimble Offices, Morley1d ago
-
Manager-Data Analytics GBP 57K-68KData Governance | Data Visualization | Data integration | Digital analytics | ForecastingMid-level Full TimeLONDON, LONDON, United Kingdom1d ago
-
AI Engineer GBP 70K-96KAccess Control | Azure | Azure AI | CI/CD | ClaudeCareer follow-up | Certification support | Continuous learning opportunities | Flexible schedule | Remote working policyMid-level Full TimeLondon, England, United Kingdom1d ago
-
Senior Data Engineer - Infra GBP 68K-85KApache Airflow | Apache Kafka | Apache Spark | ClickHouse | GrafanaSenior-level Full TimeLondon, England, GB1d ago
-
Data Steward Lead: Sales & Marketing GBP 70K-70KCRM data | CRM data management | Data Governance | Data Management | Data MigrationFlexible accommodations | Hybrid workSenior-level Full TimeLondon, London, United Kingdom1d ago
-
Mid-level Full TimeLondon, United Kingdom1d ago
-
Entry-level Full TimeGlasgow Campus, United Kingdom1d ago
-
Senior-level Full TimeGlasgow Campus, United Kingdom1d ago
-
Data Quality Remediation Lead GBP 72K-91KAI | Cause analysis | Data Governance | Data Quality | Data RecordsSenior-level Full TimeNorthampton, Barclays Campus, Pavilion Drive, United …1d ago
-
Data Engineer GBP 39K-39KAmazon Redshift | Amazon Web Services | Apache Spark | CI/CD | Cloud platformEducation grant for private schooling | Flexible working arrangements | Free shuttle bus to work | Generous time off | Hybrid workingMid-level Full TimeHinxton, United Kingdom1d ago
-
Clinical Data Analytics Lead GBP 65K-75KAnomaly Detection | Automation | Clinical data | Clinical data management | Clinical programmingHybrid work arrangement | Onsite expectation 12 days per month | Remote work optionSenior-level Full TimeLondon (The Westworks), United Kingdom1d ago
-
Senior Data Platform Engineer GBP 65K-80KAutomation | Azure | Azure CLI | Azure IaaS | Azure PaaS401k | Company discounts | Employee assistance program | Health care benefits day 1 | Paid time offSenior-level Full TimeQVC - Knowsley, United Kingdom1d ago
-
DevOps Engineer GBP 80K-109KAWS | Ansible | CloudFormation | Containers | Continuous DeploymentBlameless postmortems | On-call rotationSenior-level Full TimeLondon, United Kingdom1d ago
-
Clinical Data Analytics Lead GBP 65K-75KAnomaly Detection | Clinical Trial Data | Clinical trial | Data Governance | Data QualityHybrid work | Remote eligible based on eligibility criteriaSenior-level Full TimeLondon (The Westworks), United Kingdom1d ago
-
Senior Data Engineer GBP 70K-80KAWS | Agile | CCPA | CI/CD | Cloud platformCharity involvement | Discretionary bonus schemes | Health and well-being support | Learning hub access | Sustainability initiativesSenior-level Full TimeLuton, GB, LU2 9TN1d ago
-
SAP Data & Analytics Architect Senior Manager GBP 60K-80KAI and Machine Learning | AI enablement | APIs | Agile | Analytics CloudSenior-level Full TimeLondon1d ago
-
Senior Data Engineer -Quantexa GBP 70K-74KApache Spark | Azure DevOps | CI/CD | Contextual Data Modeling | Contextual dataFlexible hours | Office work on demand | Part-time options | Work from homeSenior-level Full TimeLondon1d ago
-
AI Data Engineer GBP 50K-60KBigQuery | Cloud Storage | Cloud platform | Docker | ETLFlexible benefits | Life assurance | Medical cover | Parental leave | Season ticket loansMid-level Full TimeLondon, England, United Kingdom1d ago
-
Mid-level Full TimeKnutsford, Radbroke Hall, United Kingdom1d ago
-
Lead Platform Engineer GBP 75K-85KAI Safety | AWS Step Functions | Amazon EKS | Amazon SageMaker | Apache AirflowSenior-level Full TimeGlasgow Campus, United Kingdom1d ago