Data Engineer (Mid‑Level ), Global
Tasks
- Build Azure Data Factory pipelines
- Build data pipelines
- Create Azure Synapse SQL views and tables
- Create SQL transformations
- Create Spark transformations
- Design data pipelines
- Develop PySpark notebooks
- Maintain data pipelines
- Maintain operational runbooks
- Maintain pipeline documentation
- Monitor pipeline performance
- Optimize spark job performance
- Orchestrate batch pipelines
- Orchestrate incremental pipelines
- Perform data quality checks
- Produce curated datasets
- Run code reviews
- Troubleshoot pipeline failures
- Validate data outputs
Perks/Benefits
Skills/Tech-stack
Agile | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Functions | Azure Synapse | CI/CD | Data Factory | Data Lake Storage Gen2 | Data Modeling | Dimension tables | ELT | ETL | Fact tables | Git | Logic Apps | PySpark | Python | SQL
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Roles
Related jobs
-
Robotics Engineer GBP 45K-55KA Star | Algorithm Optimization | C++ | Computer Vision | DWAAnnual leave | Employee assistance program | Enhanced company pension | Every other Friday off | Eye care schemeMid-level Full TimeGB-Ipswich11h ago
-
Data Engineer GBP 37K-45KAPI Integration | AWS | AWS Glue | Amazon EMR | Amazon RedshiftAnnual leave | Company pension scheme | Cycle to work | Hybrid working | Performance bonusMid-level Full TimeBristol, England, United Kingdom R15h ago
-
Director of Deployment Analytics, Digital & Technology Transformation, EMEA (12 Month+ FTC) GBP 99K-120KAnalytics | Business Intelligence | Data Governance | Data Interpretation | Data QualityExecutive-level Full TimeLONDON, United Kingdom15h ago
-
Senior Data Engineer GBP 75K-115KAPI Development | AWS CloudWatch | AWS Glue | AWS Lambda | Amazon AthenaEmployee assistance programme | Extensive leave policy | Flexible working options | Health insurance | Life insuranceSenior-level Full TimeLondon, England, United Kingdom1d ago
-
Senior-level Full TimeGlasgow Campus, United Kingdom1d ago
-
Software Engineering Lead / Applied AI Engineering GBP 80K-109KAPI Development | Apache Spark | Automation | CI/CD | Data EngineeringSenior-level Full TimeUK - London (London Wall), United …1d ago
-
Senior-level Full TimeAltrincham, England, United Kingdom1d ago
-
Senior Machine Learning Engineer GBP 75K-90KAWS | Azure | Cloud Architecture | Cloud platform | DockerCoaching | Dental insurance | Enhanced parental leave | Flexible working | Hybrid workingSenior-level Full TimeUK - Remote R1d ago
-
Data Engineer GBP 55K-85KAWS Lambda | AWS Step Functions | Automated testing | Bitemporal Data | Data IntegrityMid-level Full TimeLondon UK1d ago
-
Data Engineer GBP 52K-75KAWS | Apache Flink | Apache Spark | Elasticsearch | Google CloudFlexible hybrid schedule | Opportunity for impact | Remote work option | Work with cutting-edge technologyMid-level Full TimeLondon R1d ago
-
Digital Analytics Lead - CRM GBP 65K-70KAdobe Analytics | Artificial Intelligence | Attribution Modeling | CRM Analytics | Conversion TrackingElectric vehicle charging | Employee assistance programme | Employee events | Fitness programme | Free parkingSenior-level Full TimeLondon, United Kingdom1d ago
-
Senior-level Full TimeLondon, UK2d ago
-
Data Science Specialist GBP 70K-80KArtificial Intelligence | CI/CD | Containers | Data Analysis | Exploratory Data AnalysisDental coverage | Gym membership discount | Healthcare discount | Holiday purchase scheme | Paid carer’s leaveSenior-level Full TimeChelmsford, GB, CM1 1NP2d ago
-
Senior Software Engineer, Embedded UI GBP 75K-90KAI | API Design | BrightScript | JavaScript | Language ModelsCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Health insuranceSenior-level Full TimeManchester, United Kingdom R2d ago
-
AWS | Agent Orchestration | Agent systems | Azure | Cloud platformContributory pension scheme | Employer pension contribution | Enhanced Adoption Pay | Enhanced maternity pay | Private healthcareMid-level Full TimeSheffield, GB, S1 4NB2d ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeMid-level Full TimeLondon, United Kingdom3d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Remote-friendly, distributed teamMid-level Full TimeBrighton, United Kingdom3d ago
-
Senior Software Engineer, Training & Experimentation USD 180K-250KAPI Development | AWS | Asynchronous processing | Automated testing | Azure401k matching | Company holidays | Company winter break | Dental coverage | Flexible scheduleSenior-level Full TimeLondon, England, United Kingdom; New York, … R3d ago
-
Machine Learning Intern GBP 40K-62KComputer Vision | Diffusion Models | Fine Tuning | LLM Fine-tuning | Language ProcessingFlexible start date | Full time or part time | Paid internshipEntry-level InternshipLondon,England,United Kingdom3d ago
-
Energy Intelligence & Analytics Manager GBP 45K-60KAutomation | Business Intelligence | Dashboarding | Data Governance | Data ModelingMid-level Full TimeHarlow, England, United Kingdom3d ago
-
Staff Software Engineer, Inference GBP 116K-155KAutoscaling | BF16 | Benchmarking | C++ | CUDACritical illness cover | Employee assistance programme | Family dental insurance | Family medical insurance | Life assuranceSenior-level Full TimeLondon, England3d ago
-
Embedded Software Engineer GBP 55K-62KAgile | C++ | Confluence | DO-178C | DOORSCycle and tech scheme | Employee assistance programme | Flexible working | Life insurance | Pension schemeMid-level Full TimeGlasgow, Scotland, United Kingdom3d ago
-
Scientific Data Engineer - Product Safety GBP 70K-74KAWS | Apache Spark | CI/CD | Data Lakehouse | Data ModelingAnnual paid leave | Life insurance | Pension scheme | Private medical insurance | Work-life balanceSenior-level Full TimeBracknell, BERKSHIRE, United Kingdom3d ago
-
Quantitative Developer - Python GBP 40K-50KData cleaning | Financial markets | Machine Learning | Python | Software developmentWork-life balance initiativesMid-level Full TimeLondon3d ago
-
Data Engineer III GBP 80K-90KAWS | Automated testing | CI/CD | CockroachDB | DBTCo-working space access | Company retreat | Equipment stipend | Fully remote | Home office stipendSenior-level Full TimeUnited Kingdom R3d ago