Data Engineer - AWS
London, England, United Kingdom
GBP 55K-65K (estimate) Mid-level Full Time
Tasks
- Build batch data processing
- Build real-time data processing
- Collaborate with DevOps engineers
- Collaborate with architects
- Create curated data assets
- Design data pipelines
- Develop data platform components
- Develop data transformation processes
- Ensure data quality and integrity
- Ingest data from APIs
- Ingest data from SFTP
- Integrate new data sources
- Optimize pipelines for performance scalability cost
- Process event streams with Kafka
- Share knowledge and drive continuous improvement
- Support data source onboarding
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | Amazon EMR | Amazon S3 | Apache Kafka | Batch Processing | CI/CD | Data Quality | Data Transformation | Data Validation | Data pipeline | Distributed Systems | GitHub | Infrastructure as Code | PySpark | Python | Real Time | Real-time Processing | Schema management | Terraform | Time processing | “as-code”
Education
N/A
Roles
Related jobs
-
Applied AI & ML Lead – Markets Operations GBP 81K-109KAWS | AutoPrompt | Azure | Bedrock | Cloud platformSenior-level Full TimeLONDON, United Kingdom5h ago
-
Senior Software Engineer - Data Snapshot (CASS) GBP 65K-90KAlgorithms | Apache Spark | C plus plus | Cassandra | Continuous DeploymentSenior-level Full TimeLondon5h ago
-
Buildpacks | Data Structures | Data Structures and Algorithms | Envoy | GoSenior-level Full TimeLondon5h ago
-
Software Specialist (TL) - AI/ML - Monetisation GBP 35K-40KC++ | Causal Inference | Data Analysis | Data Modeling | Data PipelinesSenior-level Full TimeRemote, UK | London, UK R6h ago
-
Computer Vision | Diffusion Models | Generative AI | Language Models | Large Language ModelsMid-level Full TimeLondon, UK6h ago
-
Senior AI Engineer GBP 77K-100KAWS | Agent architectures | Anthropic API | Cloud platform | DevOpsAnnual leave | Community involvement | Course and certification support | Cycle to work | Give as you earnSenior-level Full TimeBristol, England, United Kingdom7h ago
-
Applied AI & ML Lead - Markets Operations GBP 81K-109KAWS | Artificial Intelligence | Azure | Cloud platform | Data AnalysisSenior-level Full TimeLONDON, LONDON, United Kingdom8h ago
-
MLOps Engineering Specialist GBP 50K-66KAWS SageMaker | Amazon Web Services | CI/CD | Distributed tracing | Drift DetectionMid-level Full TimeLondon, GB, E1 8EP9h ago
-
AI Engineer, Rates VP GBP 81K-130KAPIs | Agent systems | Agentic Frameworks | ETL | EmbeddingsEmployee assistance program | Employee discounts | Learning and development resources | Life insurance | Paid leaveExecutive-level Full Time33 CANADA SQUARE CANARY WHARF LONDON, …18h ago
-
Senior-level Full Time40 BANK STREET, United Kingdom18h ago
-
Mid-level Full TimeLondon22h ago
-
Lead AI Engineer, AI Lab GBP 78K-100KAI orchestration | Agentic Systems | Audio Processing | Audio to Text | BenchmarkingAnnual leave | Counselling and wellbeing resources | Employee assistance program | Free access to Economist content | Hybrid workSenior-level Full TimeLondon - Commercial R1d ago
-
Associate Principal Engineer (Big Data Engineer) GBP 52K-60KAirflow | Apache Kafka | Azure | CI/CD | DBTMid-level Full TimeLondon, United Kingdom1d ago
-
Senior Data Engineer GBP 60K-80KAWS | Agile | Apache Airflow | Asana | AzureContinuous learning | Health insurance | Pension plan | Remote work flexibility | Work-life balanceSenior-level Full TimeLondon, United Kingdom R1d ago
-
Artificial Intelligence | CI/CD | Continuous Deployment | Continuous integration | Continuous integration Continuous deploymentSenior-level Full TimeLondon, UK1d ago
-
Applied AI Engineer GBP 100K-160KAI Agent | Backend Development | Cloud services | Docker | JavaScriptExperimentation time | International team culture | Learning-focused environment | Remote work flexibilityMid-level Full TimeLondon, United Kingdom1d ago
-
Applied AI & Data Scientist GBP 50K-61KAgent-based | Agent-based systems | Data Engineering | Data Pipelines | Deep learningAnnual leave | Family-friendly initiatives | Flexible work options | Hybrid working | Pension schemeMid-level Full TimeEdinburgh - Central Area, Midlothian, United …1d ago
-
Lifecycle Data Analyst GBP 50K-60KA/B | A/B Testing | B testing | Data Quality | Data VisualizationMid-level Full TimeUnited Kingdom, London1d ago
-
Member of technical staff - Research - Model - London GBP 230K-340KData Pipelines | Deep learning | Distributed Training | Evaluation | GitCareer development | Continuous learning | Hybrid work | Professional growthSenior-level Full TimeLondon1d ago
-
AWS | Ansible | Automation | Backups | CDCCareer growth | Mentorship and coaching | On-call rotations | RSU benefitsSenior-level Full TimeLondon, United Kingdom1d ago
-
AWS | Ansible | As-a-Service | Authentication | AutomationMid-level Full TimeLondon, United Kingdom1d ago
-
Senior Analytics Engineer GBP 60K-70KBigQuery | DBT | Data Mapping | Data Modeling | Data VisualizationAnnual Product Allowance | Annual performance bonus | Bank holidays | Career growth | Employee discountsSenior-level Full TimeBorehamwood, United Kingdom1d ago
-
Data Science & AI Specialist GBP 30K-30KAWS Bedrock | Apache Druid | Apache Pulsar | Artificial Intelligence | CI/CDCarers leave up to 2 weeks | Discounted mobile and broadband | Equalized maternity paternity and adoption leave | Holiday purchase scheme | Hybrid workingMid-level Full TimeLondon, GB, E1 8EP R1d ago
-
Revenue Operations Data Engineering Partner GBP 60K-80KAutomation | B2B SaaS | B2B SaaS Revenue Mechanics | CRM Integrations | CRM hygieneSenior-level Full TimeLondon, United Kingdom1d ago
-
Backend Engineer - Data Platform GBP 80K-105KAgile | Algorithms | Apache Beam | Apollo | Cloud PubSubWork from home flexibilitySenior-level Full TimeLondon1d ago