Senior Data Engineer
London - Shell Centre, United Kingdom
GBP 60K-65K (estimate) Senior-level Full Time
Tasks
- Automate initial and incremental data loads
- Build data pipelines for structured and unstructured data
- Create data mapping documents
- Design data pipelines
- Generate data pipelines using agentic workflows
- Ingest transform and load data into data lake or database
- Optimize data pipelines and data platforms
- Perform deep SQL query analysis and optimization
- Support operations teams during critical issues
- Triaging pipeline and data layer issues
Perks/Benefits
- Flexible working hours
- Paid parental leave
- Remote work possibility
- Training and development programs
Skills/Tech-stack
Apache Cassandra | Apache Spark | Azure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Databricks | Azure SQL | Azure SQL Database | Blob Storage | Cosmos DB | Data Factory | Data Lake | Data Lake Storage | Data Warehousing | Databricks | Delta Lake | ETL | GitHub Copilot | MongoDB | Polars | PySpark | Python | SQL | SQL Database | Stored Procedures | Triggers | User Defined Functions
Education
N/A
Roles
Related jobs
-
Senior Analytics Engineer - Cyber Data Platform GBP 43K-54KApache Airflow | Azure | CI/CD | DBT | Data GovernanceAnnual bonus scheme | Bank holidays | Employee assistance programme | Flexible working | Free virtual GP serviceSenior-level Full TimeWelwyn Garden City, United Kingdom of …6h ago
-
APIs | C++ | Cloud Computing | Docker | GoEntry-level Full TimeLondon, UK8h ago
-
API Design | AWS | Apache Airflow | Apache Iceberg | Apache KafkaSenior-level Full TimeGLASGOW, LANARKSHIRE, United Kingdom9h ago
-
AI Engineer, Retrieval & Agentic Systems GBP 90K-105KAgentic Systems | BM25 | Information Retrieval | Knowledge graph | PythonHybrid work | Paid time off | Pension plan | Private health insuranceMid-level Full TimeLondon1d ago
-
Senior AI Engineer, Agentic Systems GBP 105K-120KAPI Design | AWS | Agent systems | Agentic Systems | BM25Annual leave | Hybrid work | Pension plan | Private health insuranceSenior-level Full TimeLondon1d ago
-
Fullstack Machine Learning Developer (Junie) GBP 100K-131KAgent workflows | Command Line | Command-line Interface | Cost Optimization | Language ModelsExtra time off | Gym or Sports Stipend | Language classes | Learning and development | Lunch allowanceMid-level Full TimeAmsterdam, Netherlands; Belgrade, Serbia; Berlin, Germany; … R1d ago
-
Associate AI Engineer GBP 57K-63KAmazon Web Services | Artificial Intelligence | CI/CD | Docker | EmbeddingsEmployee assistance program | Employee share purchase plan | Hybrid working | Learning and development | Life assuranceMid-level Full TimeLondon, United Kingdom1d ago
-
AWS | Agile | Angular | Automated testing | CI/CDSenior-level Full TimeBOURNEMOUTH, DORSET, United Kingdom1d ago
-
Agentic AI Engineer, AI Automation GBP 70K-85KAI Search | AWS | Agents SDK | Agile | AutogenBirthday day off | Employee resource groups | Extra holiday allowance | Flexible working | Hybrid workingEntry-level Full TimeLondon1d ago
-
Staff Machine Learning Engineer GBP 74K-100K3D Graphics | Computer Vision | Convolutional Neural Networks | Coordinate transformations | Data PipelinesCycle garage and showers | Dental scheme | Free soft drinks | Hybrid working | On-site barSenior-level Full TimeUnited Kingdom, London1d ago
-
Senior Software Engineer - Data Infrastructure GBP 80K-90KAWS | Amazon SQS | Apache Flyte | BigQuery | Data IngestionSenior-level Full TimeLondon1d ago
-
Staff Forward Deployed Engineer GBP 84K-126KAI Agents | AWS | Amazon Redshift | Azure | BigQueryCompany equity | Life insurance | Mental health support | Paid time off | Paid volunteering leaveSenior-level Full TimeManchester1d ago
-
Staff Engineer - Data platform GBP 85K-110KAWS | Apache Flink | Apache Iceberg | Apache Kafka | Apache SparkSenior-level Full TimeUnited Kingdom1d ago
-
A/B | A/B Testing | AWS | Airflow | B testingCreativity and Wellness benefits | Employee equity plan | Flexible vacation policy | Free German courses | Gym membershipSenior-level Full TimeBerlin, London1d ago
-
Machine Learning Engineer GBP 29K-29KMachine Learning | Model Training | Python | Workflow OptimizationEducational programme | GPU compute access | Mentorship | Real-world projectsEntry-level Full TimeLondon, England, United Kingdom1d ago
-
Machine Learning Engineer GBP 75K-95KCI/CD | Containerization | Distributed Computing | GPU Computing | MLOpsEmployee discount | Employee sample sales | Paid annual leave | Personalized learning | Private medical careMid-level Full TimeLondon, England, United Kingdom1d ago
-
Business Intelligence Data Engineer GBP 50K-70KAirbyte | BigQuery ML | CI/CD | Cloud Functions | Cloud RunMid-level Full TimeUK (Home based) R1d ago
-
Mid-level Full TimeThornton-Cleveleys, United Kingdom1d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Remote distributed setting | Supportive teamMid-level Full TimeEdinburgh, United Kingdom1d ago
-
Bash | Cloud infrastructure | Cloud platform | Data Processing | DockerMid-level Full TimeBirmingham, United Kingdom1d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Flexible management approach | Remote-friendly work environmentMid-level Full TimeNottingham, United Kingdom1d ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeMid-level Full TimeGlasgow, United Kingdom1d ago
-
Senior AI Engineer GBP 78K-109KAI Ops | AIOps | Agile | Azure Cognitive | Azure Cognitive ServicesFlexible working patterns | Hybrid work | Knowledge sharing | Mentoring | TrainingSenior-level Full TimeManchester, United Kingdom1d ago
-
Associate, Data Science GBP 50K-58KCI/CD | Cloud Computing | Containers | Data Engineering | DevOpsCollaborative culture | Continuous learning | Hybrid work arrangement | Inclusion and diversity focusMid-level Full TimeLondon, United Kingdom1d ago
-
Senior-level Full TimeFleet Place Office, United Kingdom1d ago