Senior Data Engineer, Python, Spark
Manchester, United Kingdom
GBP 75K-90K (estimate) Senior-level Full Time
Tasks
- Build scalable fault tolerant batch and streaming pipelines
- Debug production data processing systems
- Define data mappings and transformations
- Design data models and self service metrics
- Ensure data quality and resilience
- Maintain and evolve data platforms
- Optimize performance of large data clusters
- Participate in architecture discussions and roadmap
Perks/Benefits
- Dental insurance
- Financial wellness support
- Healthcare
- Hybrid work
- Life insurance
- Mental health support
- Retirement plan
- Vision insurance
Skills/Tech-stack
AWS | Airflow | Data Modeling | GCP | HDFS | Hadoop | Hive | Java | Kafka | Looker | MapReduce | Presto | Python | SQL | Spark | YARN
Education
Roles
Related jobs
-
Data Science & AI Specialist GBP 28K-28KAWS Bedrock | Apache Pulsar | Apache Spark | Artificial Intelligence | CI/CDCarers leave bonus | Discounted mobile and broadband | Equalized maternity paternity and adoption leave | Holiday purchase scheme | Paid carer’s leaveMid-level Full TimeLondon, GB, E1 8EP R5h ago
-
Staff Software Engineer, Inference GBP 325K-390KAWS | Batching | Caching | Distributed Systems | GCPFlexible work environment | Flexible working hours | Generous vacation | Parental leaveSenior-level Full TimeLondon, UK21h ago
-
Sr. Software Engineer, Inference GBP 225K-325KAWS | Batching | Caching | Distributed Systems | GCPFlexible working hours | Generous vacation | Parental leave | Visa sponsorshipSenior-level Full TimeLondon, UK21h ago
-
Forward Deployed Engineer GBP 74K-120KAWS | Apache Spark | Azure | CI/CD | Data PipelinesCustomer travel | Hybrid workSenior-level Full TimeLondon, United Kingdom22h ago
-
API | AWS | Artificial Intelligence | Automation | AzureEntry-level Full TimeBirmingham, England, United Kingdom23h ago
-
Software Engineer (Data Platform) GBP 50K-65KAPI Development | Azure | CI/CD | Data Modeling | Data PipelinesEthical pension scheme | Flexible working | Hybrid working | Office snacks | Paid time offMid-level Full TimeLondon, United Kingdom1d ago
-
Mid-level Full TimeWorsley1d ago
-
Junior Data Engineer GBP 30K-40KApache Airflow | Apache Spark | Data Governance | Data Modeling | Data ProcessingGrowth plan | Gym discounts | Learning resources | Mental health support | MentorshipEntry-level Full TimeLondon R1d ago
-
Senior Security Data Engineer GBP 91K-106KClickHouse | DBT | Data Engineering | Databricks | Google WorkspaceDental insurance | Family leave | Flexible paid time off | Healthcare | Paid holiday time offSenior-level Full TimeManchester, UK1d ago
-
DataOps Engineer GBP 50K-60KAWS | Ansible | Bash | Docker | ELTFlexible working hours | Remote work option | Team collaborationMid-level Full TimeUnited Kingdom R1d ago
-
Senior AI Engineer, AI Lab GBP 90K-131KBLEU | Bark | DVC | ElevenLabs | Fine TuningAnnual leave | Employee assistance program | Free Economist content online subscription | Moving home allowance | Parental leaveSenior-level Full TimeLondon - Commercial R1d ago
-
Lead Software Engineer – Data Platform GBP 72K-109KAWS Glue | AWS Lambda | Amazon Aurora | Amazon S3 | Apache IcebergHealthcare | Paid volunteering days | Retirement planning | Wellbeing initiativesSenior-level Full TimeLondon, United Kingdom1d ago
-
Mid-level Full TimeUK - Duxford, United Kingdom1d ago
-
Gen AI Engineer GBP 84K-109KAI Search | AWS | Azure | Azure AI | Azure AI SearchEthical work culture | Hybrid working | Learning courses and certifications | Mentoring and career progression | Training and development opportunitiesSenior-level Full TimeLondon, GB1d ago
-
Senior Machine Learning & AI Engineer GBP 72K-80KCloud Computing | Data Pipelines | Feature Engineering | GCP | Machine LearningDiscounted shopping | Flexible working | Hybrid working | Paid time off | Parental leaveSenior-level Full TimeBristol Harbourside, United Kingdom1d ago
-
Gen AI Engineer GBP 84K-109KAWS | Azure | Azure OpenAI | BigQuery | CI/CDHybrid working | Learning and development opportunities | Wellbeing programsSenior-level Full TimeLondon, GB1d ago
-
Sr. Software Engineer - Cloud, Data Platform GBP 97K-124KAWS | Apache Flink | Apache Kafka | Apache Spark | AvroCompetitive vacation and holidays | Employee networks | On-site amenities | Paid adoption leave | Paid parental leaveSenior-level Full TimeLondon (Haggerston), United Kingdom1d ago
-
Machine Learning Manager, Borrowing GBP 100K-160KAI Platform | AWS | BigQuery | Cloud platform | Credit RiskEquipment provided | Flexible working hours | Learning budget | Relocation support | Visa sponsorshipMid-level Full TimeCardiff, London or Remote (UK) R2d ago
-
AI Engineer GBP 75K-100KC# | Cloud Computing | MLOps | Machine Learning | NoSQLAnnual leave | Benefit discounts | Healthcare cash plan | Life insurance | Pension matchingSenior-level Full TimeLondon, United Kingdom2d ago
-
Principal Data Engineer | Digital Services GBP 51K-55KAPIs | Azure | Data Lake | Data Modeling | Data PipelinesBenefits | Flexible working | Hybrid working | Local government pension schemeSenior-level Full TimePreston, Lancashire, United Kingdom2d ago
-
Data Engineer (Maternity Cover 12 month FTC) GBP 26K-26KAccess Control | Batch Processing | CI/CD | Cloud Platforms | Cloud platformPersonal development opportunities | Staff discountEntry-level Contract TemporaryBury, GB-BUR, BL9 8RR, GBR2d ago
-
Bayesian Inference | Data Science | Hypothesis Testing | Language Processing | Machine LearningDental coverage | Enhanced parental leave | Family-friendly flexibility | Flexible working | Hybrid workingNone Full TimeUnited Kingdom2d ago
-
MLOps Engineering Specialist GBP 55K-58KAWS | AWS CDK | AWS CloudFormation | Alerting | Amazon SageMakerDiscounted mobile and broadband | Equalized maternity paternity adoption leave | Gym memberships | Holiday purchase scheme | Online private GP 24 7Mid-level Full TimeLondon, GB, E1 8EP R2d ago
-
Data Engineer GBP 31K-34KAPI | AWS | Apache Spark | Data Lake | Data WarehouseAnnual leave | Hybrid working | Mentoring and coaching | Pension scheme | Performance bonusMid-level Full TimeBelfast, United Kingdom2d ago
-
Bash | Cloud infrastructure | Cloud platform | Data Processing | DockerAsynchronous work culture | Entrepreneurial team | Hands-off management | Inclusive workplace | Opportunity to make impactMid-level Full TimeLondon, United Kingdom2d ago