Data Engineer
London - Shell Centre, United Kingdom
GBP 40K-45K (estimate) Mid-level Full Time
Tasks
- Build and maintain Power BI datasets and dashboards
- Define and maintain data contracts and governance
- Design build maintain scalable ETL/ELT pipelines
- Develop batch and near real time data pipelines
- Ingest transform validate and publish trading and market data
- Manage integrations between on premises cloud and legacy systems
- Monitor troubleshoot and root cause data pipeline issues
- Process trades prices reference data and operational feeds
- Use version control and implement CI CD pipelines
- Write code associated tests and documentation
Perks/Benefits
Skills/Tech-stack
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Databricks | Azure DevOps | Azure Key Vault | Azure SQL | Azure SQL Database | Azure Service | Azure Service Bus | CI/CD | Cause analysis | Data Factory | Data Governance | Data Lake Storage | Data Modeling | Data Validation | Debugging | ELT | ETL | GitHub | Key Vault | Microsoft Azure | Power BI | Python | Root Cause Analysis | Root cause | SQL | SQL Database | Service Bus | SonarQube | Version control
Education
N/A
Roles
Related jobs
-
Senior AI/ML Engineer GBP 62K-80KAKS | AWS | Airflow | Azure | Azure Machine LearningAnnual holiday allowance | Colleague discount | Cycle to work scheme | Employee assistance programme | Employee discountsSenior-level Full TimeLondon, London, United Kingdom2h ago
-
Software Engineer (Embedded) GBP 45K-59KADC | ARM microcontrollers | Agile | Attestation | Bare MetalMid-level Full TimeCambridge13h ago
-
Senior-level Full TimeLondon, England, United Kingdom13h ago
-
Data Engineer GBP 50K-68KAgile | Apache Spark | Azure Cosmos | Azure Cosmos DB | Azure DataDE&I initiatives involvement | Employee discount | Flex benefits allowance | Industry events participation | Paid annual leaveSenior-level Full TimeLondon, England, United Kingdom20h ago
-
Senior-level Full TimeLondon, United Kingdom21h ago
-
Data Engineer GBP 49K-50KAPI Integration | Azure Data | Azure Data Factory | Azure Data Lake | Azure SQLEmployee assistance program | Enhanced family leave | Identity and background checks | Life insurance | Paid time offMid-level Full TimeCardiff, Wales CF1, United Kingdom22h ago
-
AI Solutions Developer GBP 55K-70KAWS | Autogen | CI/CD | CrewAI | FastAPIAdditional days off | Coffee and snacks | Extra parental leave | Health insurance | Hybrid workMid-level Full TimeLondon, England, United Kingdom1d ago
-
Applied AI & ML Lead – Markets Operations GBP 81K-109KAWS | AutoPrompt | Azure | Bedrock | Cloud platformSenior-level Full TimeLONDON, United Kingdom1d ago
-
Buildpacks | Data Structures | Data Structures and Algorithms | Envoy | GoSenior-level Full TimeLondon1d ago
-
Software Specialist (TL) - AI/ML - Monetisation GBP 35K-40KC++ | Causal Inference | Data Analysis | Data Modeling | Data PipelinesSenior-level Full TimeRemote, UK | London, UK R1d ago
-
Coach - Data Engineer Level 5 GBP 55K-70KAWS | Apache Airflow | Apache Spark | Azure | Batch dataCertification support | Flexible working | Fully remote | Professional development | Travel as requiredSenior-level Full TimeLondon, United Kingdom R1d ago
-
Computer Vision | Diffusion Models | Generative AI | Language Models | Large Language ModelsMid-level Full TimeLondon, UK1d ago
-
Senior AI Engineer GBP 77K-100KAWS | Agent architectures | Anthropic API | Cloud platform | DevOpsAnnual leave | Community involvement | Course and certification support | Cycle to work | Give as you earnSenior-level Full TimeBristol, England, United Kingdom1d ago
-
Senior AI Engineer GBP 78K-100KAWS | Anthropic API | Azure | DevOps | DevSecOpsCourse and training budget | Hybrid working | Learning budget | Mentoring and coaching | Paid time offSenior-level Full TimeManchester, England, United Kingdom1d ago
-
Senior-level Full TimeLondon, United Kingdom1d ago
-
Lead Data Engineer GBP 90K-106KApache Spark | Azure | Data Migration | Data Modeling | Data cleaningSenior-level Full TimeUK - London1d ago
-
Applied AI & ML Lead - Markets Operations GBP 81K-109KAWS | Artificial Intelligence | Azure | Cloud platform | Data AnalysisSenior-level Full TimeLONDON, LONDON, United Kingdom1d ago
-
MLOps Engineering Specialist GBP 50K-66KAWS SageMaker | Amazon Web Services | CI/CD | Distributed tracing | Drift DetectionMid-level Full TimeLondon, GB, E1 8EP1d ago
-
Treasury Data Engineer GBP 40K-50KApache Airflow | Backend Web Services | Cloud Platforms | Data Modeling | Networking protocolsMid-level Full TimeLondon1d ago
-
Senior Applied Scientist GBP 225K-325KBlack | Cloud Computing | Deep learning | Git | GitHubHybrid work | Mentoring and coachingSenior-level Full TimeLondon1d ago
-
Staff Applied Scientist GBP 225K-325KBlack | Cloud Computing | Complex Planning | Data Analysis | Deep learningCoaching | Hybrid work scheme | MentorshipSenior-level Full TimeLondon1d ago
-
Senior Analytics Engineer GBP 70K-81KAzure | BI platforms | BigQuery | DBT | Data ModelingCompany pension | Cycle to work scheme | Enhanced parental leave | Gympass subscription | Health cash planSenior-level Full TimeLondon1d ago
-
Mid-level Full TimeLondon, England, United Kingdom1d ago
-
AI Engineer, Rates VP GBP 81K-130KAPIs | Agent systems | Agentic Frameworks | ETL | EmbeddingsEmployee assistance program | Employee discounts | Learning and development resources | Life insurance | Paid leaveExecutive-level Full Time33 CANADA SQUARE CANARY WHARF LONDON, …1d ago
-
Senior-level Full Time40 BANK STREET, United Kingdom1d ago