Data Engineer
Manchester, NorthWest M1, United Kingdom
GBP 50K-60K (estimate) Mid-level Full Time
Tasks
- Build and enhance data pipelines with Spark
- Collaborate with analysts developers and stakeholders on data requirements
- Continuously improve pipeline performance reliability and maintainability
- Define and implement data quality rules and validations
- Design Gold layer data models for analytics
- Develop automated data quality checks and exception reports
- Develop transformation enrichment and aggregation logic
- Document pipelines data models and architectural decisions
- Ensure data access controls and security measures
- Implement Medallion Architecture layers
- Implement data governance standards and metadata management
- Maintain Databricks data platform
- Maintain data lineage documentation
- Monitor troubleshoot pipeline failures and data issues
Perks/Benefits
- Annual discretionary bonus scheme
- Cash back healthcare scheme
- Company contributory pension scheme
- Cycle to work scheme
- Employee assistance programme
- Enhanced family-friendly policies
- Hybrid working
- Life assurance
- Staff referral scheme
- Subsidised gym membership
Skills/Tech-stack
API | Apache Spark | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Key Vault | Azure SQL | Azure SQL Database | Data Governance | Data Lake Storage | Data Lineage | Data Modeling | Data Quality | Data orchestration | Databricks | Delta Lake | Key Vault | Power BI | Python | SQL | SQL Database
Education
N/A
Roles
Related jobs
-
APIs | AWS | Automation Pipelines | Azure | Backend DevelopmentAnnual leave | Company lunches | Learning and development budget | Monthly socials | Public holidaysEntry-level Full TimeLondon6h ago
-
Alerting | Cause analysis | Completeness Accuracy Timeliness Consistency | Data Governance | Data MonitoringSenior-level Full TimeLondon8h ago
-
AWS | Apache Airflow | Apache Kafka | Apache Spark | CI/CDCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Flexible remote workSenior-level Full TimeManchester, United Kingdom20h ago
-
Senior AI Engineer GBP 70K-90KAPI Development | AWS | Agent Orchestration | Apache Spark | CI/CDCentral London office | Enhanced parental leave | Enhanced pension | Free meals and snacks | Life insuranceSenior-level Full TimeLondon, England, United Kingdom20h ago
-
Bilby | CAMB | Class | DOLFINx | FenicsPart-time freelance | Project based workSenior-level FreelanceUnited Kingdom - Remote R20h ago
-
Data Engineering Technical Lead GBP 60K-70KAd Hoc Queries | Ad-Hoc | Data Integrity | Data Modeling | Data ProcessingHybrid work model | Mentorship | Professional developmentSenior-level Full TimeGBR - Manchester, United Kingdom20h ago
-
Data Engineer (Integration) GBP 60K-70KAWS Lambda | AWS Secrets | AWS Secrets Manager | AWS Systems Manager | AWS Systems Manager Parameter StoreCollaborative culture | Flexible collaboration hours | Fully remote across Europe | Work-life balanceSenior-level Full TimeLondon, United Kingdom R1d ago
-
Senior Data Engineer — Ingestion & Platform GBP 70K-80KAPI Key | Apache Airflow | Authentication | CI/CD | DBTFlexible collaboration hours | Fully remote work | Ownership and innovation culture | Work-life balanceSenior-level Full TimeLondon, United Kingdom R1d ago
-
Principal Machine Learning Engineer GBP 78K-90KAIOps | AWS | Airflow | Anomaly Detection | Batch inferenceAdditional leave | Hybrid working | Move annual leave carry over | Option to Buy Sell Annual Leave | Private healthcareSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Lead Data Engineer GBP 70K-87KAdobe Experience | Adobe Experience Platform | Apache Airflow | Azure | CI/CDHybrid workSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Software engineer, generative AI (UK) GBP 80K-90KAWS | Agentic Workflows | Asyncio | Azure | CI/CDCompany offsites | Dental insurance | Fertility support | Learning and development stipend | Medical insuranceMid-level Full TimeLondon, UK1d ago
-
AI engineer (UK) GBP 80K-90KAWS | Bias detection | Cloud platform | Deep learning | ExplainabilityCompany off-sites | Dental insurance | Fertility and family planning support | Learning and development stipend | Medical insuranceMid-level Full TimeLondon, UK1d ago
-
AI Engineer USD 100K-250KLLM Evaluation | Language Models | Large Language Models | Machine Learning | PostgreSQLFree meals | Private medical insurance | Visa sponsorshipMid-level Full TimeLondon, England, GB1d ago
-
Senior-level Full TimeLondon1d ago
-
Lead Data Engineer GBP 90K-106KBasel 3.1 | Cause analysis | Data Governance | Data Lineage | Data ValidationHybrid workingSenior-level Full TimeEdinburgh, Scotland, United Kingdom1d ago
-
Research Engineer, Responsibility Engineering, DeepMind GBP 250K-340KAutomated testing | Evaluation Frameworks | Inference pipeline | JAX | Language ProcessingSenior-level Full TimeLondon, UK1d ago
-
Lead Data Engineer - C1 GBP 52K-52KBusiness Intelligence | Data Engineering | Data Integrity | Data Quality | Data StreamingSenior-level Full TimeUnited Kingdom1d ago
-
Aerodynamic AI Engineer GBP 70K-80KAnomaly Detection | CFD | CI/CD | Computational Fluid Dynamics | Computational GeometryMid-level Full TimeGrove, Wantage, United Kingdom1d ago
-
AWS | Computer Science | Distributed Systems | Kubernetes | LangchainCareer growth | MentorshipSenior-level Full TimeLONDON, LONDON, United Kingdom1d ago
-
AI Engineer GBP 61K-80KAgentic AI | Asynchronous programming | LLM Evaluation | Language Models | Large Language ModelsChoice of laptop | Company trips | Flexi working | Opportunity to learn new skills | Private health insuranceMid-level Full TimeUnited Kingdom1d ago
-
Data Engineer - Brook Green Supply GBP 46K-55KApache Airflow | Apache Flink | Apache Kafka | Apache Spark | CI/CDMid-level Full TimeLondon, England, United Kingdom1d ago
-
Senior Data Engineer GBP 70K-89KAWS | Amazon Redshift | DBT | Data Governance | Data LineageComprehensive benefits package | Share optionsSenior-level Full TimeKnutsford, England, United Kingdom1d ago
-
Data & AI Engineer GBP 50K-60KAWS | Azure | BI | Data Modeling | Data QualityCompany pension | Flexible working | Holiday | Professional qualification fundingMid-level Full TimeThe Corn Exchange, 55 Mark Lane, …1d ago
-
Data Science Team Lead GBP 85K-102KCausal Inference | Data Engineering | Data Modeling | Data Pipelines | DatabricksAdditional holiday | Birthday leave | Cycle to work scheme | Electric car scheme | Emergency family careSenior-level Full TimeLondon, United Kingdom1d ago
-
Senior Data Science Engineer, Sports Modelling GBP 70K-107KCI/CD | Deployment | Docker | Experimentation | Integration TestingSenior-level Full TimeRemote - United Kingdom R1d ago