Data Engineer
Manchester, NorthWest M1, United Kingdom
GBP 50K-60K (estimate) Mid-level Full Time
Tasks
- Build and enhance data pipelines with Spark
- Collaborate with analysts developers and stakeholders on data requirements
- Continuously improve pipeline performance reliability and maintainability
- Define and implement data quality rules and validations
- Design Gold layer data models for analytics
- Develop automated data quality checks and exception reports
- Develop transformation enrichment and aggregation logic
- Document pipelines data models and architectural decisions
- Ensure data access controls and security measures
- Implement Medallion Architecture layers
- Implement data governance standards and metadata management
- Maintain Databricks data platform
- Maintain data lineage documentation
- Monitor troubleshoot pipeline failures and data issues
Perks/Benefits
- Annual discretionary bonus scheme
- Cash back healthcare scheme
- Company contributory pension scheme
- Cycle to work scheme
- Employee assistance programme
- Enhanced family-friendly policies
- Hybrid working
- Life assurance
- Staff referral scheme
- Subsidised gym membership
Skills/Tech-stack
API | Apache Spark | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Key Vault | Azure SQL | Azure SQL Database | Data Governance | Data Lake Storage | Data Lineage | Data Modeling | Data Quality | Data orchestration | Databricks | Delta Lake | Key Vault | Power BI | Python | SQL | SQL Database
Education
N/A
Roles
Related jobs
-
Senior AI / LLM Engineer GBP 70K-92KAI Governance | AI Studio | AKS | Agent Orchestration | Azure AIDiscretionary bonus scheme | Employee discount | Employee sample sales | Flexible benefits allowance | Paid annual leaveSenior-level Full TimeLondon, England, United Kingdom13h ago
-
Software Engineer (Trading Systems) GBP 50K-70K.NET | API Development | API Integration | Alpha Data Platform | Charles RiverMid-level ContractLondon, United Kingdom of Great Britain …14h ago
-
Senior Backend AI Engineer GBP 28K-30KAPI Design | AWS | Azure | Backend Development | Cloud platformEmployee assistance programme | Employee wellbeing support | Flexible working options | Free virtual GP service | Holiday allowanceSenior-level Full TimeWelwyn Garden City, United Kingdom of …15h ago
-
C++ | Data Mining | Data Regression | Deep learning | Distributed ComputingSenior-level Full TimeLondon, UK16h ago
-
Senior Software Specialist - AI/ML - Monetisation GBP 100K-140KAds Auction | Behavior Modeling | Bidding | Budgeting | C++Senior-level Full TimeLondon, UK16h ago
-
Data Engineer GBP 42K-45KAPI Integration | Apache Kafka | BigQuery | Cloud Platforms | Data GovernanceCycle to work scheme | Discounts for family and partners | Health cash plan | Life insurance | Paid time offMid-level Full TimeLondon, England, United Kingdom16h ago
-
Lead Software Engineer - Data & AI GBP 84K-126KAWS | Amazon Athena | Amazon SageMaker | Apache Flink | Apache KafkaSenior-level Full TimeLONDON, LONDON, United Kingdom17h ago
-
Senior AI Engineer (AI Platform) GBP 72K-90KAI Foundry | API Gateway | API Management | APIM | APIsDiscretionary bonus scheme | Employee discount | Employee sample sales | Flexible benefits allowance | Paid annual leaveSenior-level Full TimeLondon, England, United Kingdom19h ago
-
Data Engineer – Data Science & Technology GBP 38K-50KData Architecture | Dimensional Modeling | ELT | ETL | NumPyAccess to fitness facilities | Charitable contribution match | Charity volunteer days | Continued learning opportunities | Employee assistance programsMid-level Full TimeLondon, United Kingdom1d ago
-
Senior-level Full TimeCanary Wharf, 1 Churchill Place, United …1d ago
-
Mid-level Full TimeCanary Wharf, 1 Churchill Place, United …1d ago
-
Junior Analytics Engineer GBP 30K-35KAzure Synapse | Data Documentation | Data Governance | Data Modelling | Data QualityMentoring programs | Training programsEntry-level Full TimeLondon, United Kingdom1d ago
-
Data Analytics Engineer- Senior Consultant II GBP 60K-65KAWS | Apache Flink | Apache Kafka | Apache Spark | AzureContinuous Learning Hub | Cycle to work scheme | Discounted gym membership | Employee assistance programme | Flexible working optionsSenior-level Full TimeBelfast 10 Mays Meadow, United Kingdom1d ago
-
Lead Cyber Automation and AI Engineer GBP 84K-109KArchitecture Troubleshooting | Artificial Intelligence | Automation | Cloud | ContainerizationEmployee development opportunities | Flexible working arrangements | Team collaborationSenior-level Full TimeWaterloo Street, United Kingdom1d ago
-
AI Solution Lead GBP 58K-75KAPI Integration | Azure Machine Learning | CI/CD | Data Governance | DevSecOpsContributory pension | Cycle to work | Dental | Enhanced family leave | Enhanced sick paySenior-level Full TimeWarminster, Wiltshire, United Kingdom1d ago
-
Senior-level Full TimeHolborn - London, United Kingdom1d ago
-
Senior-level Full TimeHolborn - London, United Kingdom1d ago
-
Senior Machine Learning Engineer GBP 70K-80KAWS | CI/CD | Data Pipelines | Databricks | Deep learningCross-functional collaboration | Hybrid work | MentorshipSenior-level Full TimeHolborn - London, United Kingdom1d ago
-
Senior Data Engineer GBP 70K-70KAgile | Behavior-Driven Development | CI/CD | Code review | Data EngineeringDiscounted gym membership | Electric vehicle leasing | Experience days | Hybrid work | Life assuranceSenior-level Full TimeLondon, United Kingdom1d ago
-
Data Engineer GBP 57K-57KAgile | Alteryx | Automated testing | Behavior-Driven Development | CI/CDDiscounted gym membership | Electric vehicle leasing | Experience days | Hybrid work | Life assuranceMid-level Full TimeLondon, United Kingdom1d ago
-
AI Research Engineer - 3D Computer Vision GBP 75K-108KComputer Vision | Deep learning | Domain Adaptation | Edge Computing | Feature MatchingEnhanced parental leave | Family emergency leave | Gym membership | Learning allowance | Mental health supportSenior-level Full TimeBerlin; London; Munich R1d ago
-
Principal Data Engineer GBP 50K-60KAWS | AWS CDK | AWS Glue | AWS Lake Formation | AWS LambdaFlexible working | Hybrid work options | Learning and development | Remote work options | Supportive cultureSenior-level Full Timeleeds, Manchester1d ago
-
Lead Data Engineer GBP 50K-60KAWS | AWS CDK | AWS Glue | AWS Lambda | Amazon AthenaFlexible working | Hybrid options | Professional growth | Remote options | Supportive cultureSenior-level Full Timeleeds, Manchester1d ago
-
Senior Machine Learning Engineer GBP 72K-85KBatch Machine Learning | CI/CD | Containerization | Data Pipelines | Data parallelismDisability accommodations during hiring process | Employee discount | Employee sample sales | Flexible benefits allowance | Paid annual leaveSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Data Engineer GBP 51K-55KApache Airflow | Azure Data | Azure Data Factory | Azure Functions | C#Hybrid work options | Knowledge sharing culture | Professional growth opportunities | Work flexibilityMid-level Full TimeBristol, 3 RIVERGATE, United Kingdom1d ago