Staff Data Platform Engineer
Tasks
- Build and operate globally distributed data transfer infrastructure
- Build hub and spoke data transfer model
- Design and build distributed data transfer pipelines
- Design cloud to cloud and cross cloud networking solutions
- Develop pipeline logic with Python
- Ensure reliability observability and operational alerting
- Implement infrastructure as code with Terraform
- Ingest large volumes of sensor data at PB scale
- Lead technical design and delivery for data transfer hub
- Orchestrate parallelized transfer jobs
- Set technical direction for data transfer hub
Perks/Benefits
Skills/Tech-stack
AWS | Alerting | Apache Kafka | Azure | Backfill | Cloud Networking | Cross-cloud | Cross-cloud networking | Data Integrity | Data Pipelines | Flyte | GCP | Infrastructure as Code | Monitoring | Object storage | Python | Retry strategies | Terraform | Workflow Orchestration | “as-code”
Education
N/A
Related jobs
-
Forward Deployed Solution Engineer – Applied AI FDE GBP 80K-110KAWS | Angular | Azure | CI/CD | ContainersFlexible work options | Remote or required in officeSenior-level Full TimeLondon, United Kingdom11h ago
-
CI/CD | Containerization | Data Pipelines | Data parallelism | Deep learningEmployee discount | Employee sample sales | Flexible benefits allowance | Paid annual leave | Personalised learningSenior-level Full TimeLondon, England, United Kingdom13h ago
-
Mid-level Full TimeUnited Kingdom13h ago
-
AI Engineer GBP 70K-108KAPI Development | Agent Frameworks | Application Architecture | Code review | Deployment AutomationDiscounts and vouchers | Hybrid working | Learning and development support | Paid time for charity work | Parental leave policiesSenior-level Full TimeNewbury, England, United Kingdom14h ago
-
API Integration | Active Batch | Artificial Intelligence | Azure Active Batch | Azure DataHybrid working | Mentorship | Pair programming supportSenior-level Full TimeLeicester, Leicestershire, United Kingdom18h ago
-
AWS | Apache Spark | Cloud platform | Data Architecture | Data EngineeringCareer development | Travel opportunitiesSenior-level Full TimeLondon, United Kingdom20h ago
-
AI Engineer GBP 100K-125KAPI Development | AWS | Apache Spark | CI/CD | DatabricksCustom designed offices | Enhanced parental leave | Enhanced pension | Free breakfasts and snacks | Life insuranceMid-level Full TimeLondon, England, United Kingdom1d ago
-
Data and AI Developer GBP 72K-108KAI Governance | AWS | Artificial Intelligence | Cloud Computing | Data EngineeringSenior-level Full TimeNottingham, Eng, United Kingdom1d ago
-
Android | C# | C++ | Caffe | Computer VisionSenior-level Full TimeFarnborough, United Kingdom1d ago
-
Mid-level Full TimeFarnborough, United Kingdom1d ago
-
Data Platform Manager GBP 55K-74KActive Directory | Azure | Azure Active Directory | CI/CD | Data GovernanceHealth and Wellbeing Programmes | Hybrid work flexibilityMid-level Full TimeNewcastle, United Kingdom1d ago
-
Principal Software Engineer (AI/ML) GBP 54K-70KAPI Development | Agent planning | Agentic Workflows | Agile | ComplianceBonus scheme | Dental coverage | Employee networks | Employer pension contribution | Financial adviceSenior-level Full TimeGB - Luton - Cap. Green …1d ago
-
Data and AI Developer GBP 84K-119KAI Governance | AWS | Artificial Intelligence | Cloud infrastructure | Data GovernanceSenior-level Full TimeNottingham, England1d ago
-
Principal Software Engineer / Principal AI Engineer GBP 93K-150KAPI Integration | AWS | Agile | Azure | CI/CDComprehensive pension plan | Employee assistance program | Employee discounts | Flexible working hours | Generous vacationSenior-level Full TimeUK - London (London Wall), United …1d ago
-
Principal Software Engineer / Principal AI Engineer GBP 100K-154KAPI Integration | AWS | Agile | Azure | CI/CDEmployee assistance program | Employee discounts | Family care leave | Flexible work hours | Internal communities and networksSenior-level Full TimeUK - London (London Wall), United …1d ago
-
Senior Software Engineer / Senior AI Engineer GBP 97K-131KAPIs | AWS | Agent workflows | Alerting | AzureComprehensive pension plan | Employee assistance program | Employee discounts | Flexible working hours | Sabbatical leaveSenior-level Full TimeUK - London (London Wall), United …1d ago
-
Senior Software Engineer / Senior AI Engineer GBP 97K-131KAPIs | AWS | Agentic Workflows | Alerting | AzureAdoption leave | Comprehensive pension plan | Employee assistance program | Employee discounts | Family care leaveSenior-level Full TimeUK - London (London Wall), United …1d ago
-
Senior Software Engineer/ Senior AI Engineer GBP 97K-131KAPIs | AWS | Agent workflows | Azure | CI/CDComprehensive pension plan | Employee assistance program | Employee discounts | Employee recognition and rewards | Flexible working hoursSenior-level Full TimeUK - London (London Wall), United …1d ago
-
Senior Software Engineer/ Senior AI Engineer GBP 97K-131KAPIs | AWS | Agent workflows | Alerting | AzureEmployee assistance program | Employee discounts | Flexible working hours | Internal communities and networks | Pension planSenior-level Full TimeUK - London (London Wall), United …1d ago
-
Junior Data Engineer GBP 37K-50KAPIs | Apache NiFi | Big Data | CI/CD | CloudDental coverage | Financial advice | Flexi days | Flexible benefits | Flexible hoursEntry-level Full TimeGB - Bristol - Coldharbour Lane, …1d ago
-
AI Engineer - FDE (Forward Deployed Engineer) GBP 80K-110KAWS | Apache Spark | Azure | Cloud platform | Data EngineeringConference presentations | Remote work | Travel opportunitiesSenior-level Full TimeRemote - United Kingdom R1d ago
-
Systems Research Engineer GBP 74K-121KAI infrastructure | C++ | Data Locality | Distributed Systems | Fault ToleranceNone Full TimeEdinburgh, United Kingdom1d ago
-
Data Engineer GBP 50K-70KAmazon Web Services | Apache Flink | Apache Kafka | Apache Spark | Cloud platformBenefits package | Flexible work schedule | Remote work one day per weekMid-level Full TimeLondon R1d ago
-
Lead Data Engineer GBP 75K-80KAWS | Agile | Apache Spark | Azure | CI/CDAnnual leave | Cycle to work scheme | Discounted gym membership | Employee events and clubs | Employer pension schemeSenior-level Full TimeLeeds1d ago
-
Machine Learning and Quant Engineer - London GBP 110K-200KData Engineering | Financial Modeling | Machine Learning | Python | Scenario AnalysisDirect influence on investment decisions | Equity | In-office work | Technical autonomyMid-level Full TimeLondon1d ago