Senior Data Engineer
Manchester, Greater Manchester, United Kingdom
GBP 55K-80K Senior-level Full Time
Tasks
- Architect real-time streaming solutions
- Bridge Kafka to S3 and Redshift
- Build infrastructure as code for data infrastructure
- Collaborate with data scientists on real time ML inference pipelines
- Design stream processing applications
- Diagnose and resolve distributed system bottlenecks
- Drive Iceberg lakehouse ingestion strategy
- Enforce data quality validation in streams
- Implement CI/CD pipelines for data applications
- Implement monitoring for streaming health
- Manage AWS resources for streaming and lakehouse
- Mentor engineers on streaming best practices
- Optimize Kafka producers consumers and stream joins
Perks/Benefits
- Additional annual leave
- Discretionary bonus
- Employee assistance programme
- Enhanced maternity and paternity pay
- Financial coaching
- Long service recognition
- Online GP 24 7
- Pension contributions
- Wagestream access
- Wellbeing support
Skills/Tech-stack
AWS | AWS Glue | Amazon MSK | Amazon Redshift | Amazon S3 | Apache Flink | Apache Iceberg | Apache Kafka | CI/CD | EMR | Encryption | Event Driven | Event-driven architecture | Exactly once | Exactly-once processing | IAM | Java | PySpark | Python | Schema Registry | Stateful stream processing | Stateless Stream Processing | Stream processing | Terraform | VPC
Education
N/A
Roles
Related jobs
-
Senior Data Engineer, Python, Spark GBP 75K-90KAWS | Airflow | Data Modeling | GCP | HDFSCommuter benefits | Disability insurance | Financial wellness resources | Flexible remote work Fridays | Healthcare benefitsSenior-level Full TimeCambridge, United Kingdom6h ago
-
Senior Data Engineer, Python, Spark GBP 75K-90KAWS | Airflow | Data Modeling | GCP | HDFSDental insurance | Financial wellness support | Healthcare | Hybrid work | Life insuranceSenior-level Full TimeManchester, United Kingdom6h ago
-
Digital and AI Solution Lead GBP 58K-70KAgentic AI | Automation | Cause analysis | Data Engineering | Data platformCycle scheme | Flexible working | Life assurance | Pension scheme | Perk At Work Discount SchemeSenior-level Full TimeCheshire-Warrington, UK11h ago
-
Member of Technical Staff (Robotics Research Lead) GBP 250K-270K3D Perception | 3D Reconstruction | C++ | Computer Vision | Deep learningSenior-level Full TimeUK, Remote R17h ago
-
Benchmarking | C++ | CUDA | OpenCL | Operator fusionHybrid work | Work from homeSenior-level Full TimeLondon19h ago
-
Analytics Engineer GBP 27K-27KAmazon Redshift | BigQuery | DAG | Data Modeling | Data TransformationCommunity programs | Convenience benefits | Family benefits | Finance benefits | Growth opportunitiesMid-level Full TimeLondon - The River Building HQ20h ago
-
AWS | Ansible | As-a-Service | Authentication | AutomationOn-call rotation support | RSUsSenior-level Full TimeLondon, United Kingdom20h ago
-
AWS | Ansible | Authentication | CDC | CI/CDMid-level Full TimeLondon, United Kingdom20h ago
-
Accuracy Checking | Alerting | Automation | Cause analysis | Completeness CheckingSenior-level Full TimeLondon21h ago
-
Senior Data Engineer GBP 60K-70KAPI Integration | CI/CD | Cloud Computing | Data Quality | Data ValidationSenior-level Full TimeLondon21h ago
-
Senior Data Platform Engineer GBP 60K-70KAWS | Argo | CI/CD | DBT | DatadogCycle to work scheme | Electric vehicle salary sacrifice scheme | Employee referral bonus | Flexible working | Hybrid or remote optionsSenior-level Full TimeLondon, United Kingdom R21h ago
-
MLOps Engineer GBP 55K-55KCI/CD | Cloud Native | Data Governance | GitLab | GrafanaFlexible working arrangements | Hybrid workingMid-level Full TimeGB-ENG-LAN-Preston21h ago
-
Machine Learning Engineer GBP 26K-28KAlerting | Algorithms | Apache Spark | Azure | Cloud ComputingAdoption leave | Annual bonus scheme | Employee assistance programme | Employee wellbeing support | Free 247 virtual GP serviceMid-level Full TimeWelwyn Garden City, United Kingdom of …22h ago
-
Mid-level Full TimeLondon, England, GB1d ago
-
Senior AI Engineer (Contract) GBP 65K-80KAPI Integration | AWS Bedrock | Confluence | Containerization | EmbeddingsCycle to work scheme | Electric vehicle salary sacrifice scheme | Employee referral programme | Flexible working | Health and wellbeing supportSenior-level ContractLondon R1d ago
-
Senior Data Engineer - Data Science Platform GBP 60K-70KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageEmployee discount | Employee sample sales | Flexible benefits allowance | Paid annual leave | Personalized learningSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Staff MLOps Engineer (AI/ML Platform) GBP 70K-99KAWS EKS | Apache Spark | Data Drift | Databricks | Experiment trackingSenior-level Full TimeRemote, Remote, United Kingdom R1d ago
-
Power BI Developer GBP 51K-65KALM Toolkit | Active Directory | Azure Active Directory | DAX | DAX StudioFlexible working hours | Hybrid work | Work-life balanceSenior-level Full TimeGBR-Abingdon-Culham (057GB), United Kingdom1d ago
-
DevOps Engineer - ML GBP 58K-58KAI Services | AKS | Application Insights | Azure | Azure AI25 days holiday | Enhanced pension plan | Family-friendly benefits | Flexible working | Health insuranceMid-level Full TimeCentral London, United Kingdom1d ago
-
Mid-level Full TimeNorthampton, Barclays Campus, Pavilion Drive, United …1d ago
-
Analytics Engineer GBP 60K-70KAI tools | CI/CD | Code review | Containerization | DBT CoreAnnual leave | Barista bar | Company pension contributions | Cycle to work scheme | HealthcareMid-level Full TimeLondon, United Kingdom1d ago
-
Mid-level Full TimeCanary Wharf, 1 Churchill Place, United …1d ago
-
AI Search | AWS | AWS Bedrock | AWS S3 | AWS SageMakerMid-level Full TimeLondon, GB1d ago
-
AI Search | AWS | AWS Bedrock | AWS EKS | AWS S3Flexible work culture | Hybrid work | Work-life balanceMid-level Full TimeManchester, GB1d ago
-
Senior ML Engineer GBP 80K-85KBenchmarking | Biomedical Data | Code review | Data loading | Deep learningHybrid workSenior-level Full TimeLondon, UK, United Kingdom, Aberdeen City1d ago