Staff Data Platform Engineer
Tasks
- Build and operate globally distributed data transfer infrastructure
- Build hub and spoke data transfer model
- Design and build distributed data transfer pipelines
- Design cloud to cloud and cross cloud networking solutions
- Develop pipeline logic with Python
- Ensure reliability observability and operational alerting
- Implement infrastructure as code with Terraform
- Ingest large volumes of sensor data at PB scale
- Lead technical design and delivery for data transfer hub
- Orchestrate parallelized transfer jobs
- Set technical direction for data transfer hub
Perks/Benefits
Skills/Tech-stack
AWS | Alerting | Apache Kafka | Azure | Backfill | Cloud Networking | Cross-cloud | Cross-cloud networking | Data Integrity | Data Pipelines | Flyte | GCP | Infrastructure as Code | Monitoring | Object storage | Python | Retry strategies | Terraform | Workflow Orchestration | “as-code”
Education
N/A
Related jobs
-
Mid-level Full TimeUnited Kingdom12h ago
-
API Integration | Active Batch | Artificial Intelligence | Azure Active Batch | Azure DataHybrid working | Mentorship | Pair programming supportSenior-level Full TimeLeicester, Leicestershire, United Kingdom16h ago
-
AWS | Apache Spark | Cloud platform | Data Architecture | Data EngineeringCareer development | Travel opportunitiesSenior-level Full TimeLondon, United Kingdom18h ago
-
AI Engineer GBP 100K-125KAPI Development | AWS | Apache Spark | CI/CD | DatabricksCustom designed offices | Enhanced parental leave | Enhanced pension | Free breakfasts and snacks | Life insuranceMid-level Full TimeLondon, England, United Kingdom1d ago
-
Data and AI Developer GBP 72K-108KAI Governance | AWS | Artificial Intelligence | Cloud Computing | Data EngineeringSenior-level Full TimeNottingham, Eng, United Kingdom1d ago
-
Android | C# | C++ | Caffe | Computer VisionSenior-level Full TimeFarnborough, United Kingdom1d ago
-
Mid-level Full TimeFarnborough, United Kingdom1d ago
-
Data Platform Manager GBP 55K-74KActive Directory | Azure | Azure Active Directory | CI/CD | Data GovernanceHealth and Wellbeing Programmes | Hybrid work flexibilityMid-level Full TimeNewcastle, United Kingdom1d ago
-
AI Engineer - FDE (Forward Deployed Engineer) GBP 80K-110KAWS | Apache Spark | Azure | Cloud platform | Data EngineeringConference presentations | Remote work | Travel opportunitiesSenior-level Full TimeRemote - United Kingdom R1d ago
-
Systems Research Engineer GBP 74K-121KAI infrastructure | C++ | Data Locality | Distributed Systems | Fault ToleranceNone Full TimeEdinburgh, United Kingdom1d ago
-
Data Engineer GBP 50K-70KAmazon Web Services | Apache Flink | Apache Kafka | Apache Spark | Cloud platformBenefits package | Flexible work schedule | Remote work one day per weekMid-level Full TimeLondon R1d ago
-
Lead Data Engineer GBP 75K-80KAWS | Agile | Apache Spark | Azure | CI/CDAnnual leave | Cycle to work scheme | Discounted gym membership | Employee events and clubs | Employer pension schemeSenior-level Full TimeLeeds1d ago
-
Machine Learning and Quant Engineer - London GBP 110K-200KData Engineering | Financial Modeling | Machine Learning | Python | Scenario AnalysisDirect influence on investment decisions | Equity | In-office work | Technical autonomyMid-level Full TimeLondon1d ago
-
Lead Data Engineer GBP 86K-102KAWS | Apache Airflow | Apache Hive | Apache Spark | AutomationAnnual leave | Flexible benefits | Pension contributions | Private medical insurance | Staff travel benefitsSenior-level Full TimeUnited Kingdom-London-London Luton Airport2d ago
-
Research Engineer, Tool Use, DeepMind GBP 155K-163KAgents | Data Analysis | Data Pipelines | Data Visualization | JAXSenior-level Full TimeLondon, UK2d ago
-
Frontend Engineer, Robotics HMI USD 115K-160KAWS | CI/CD | Datadog | Django | Docker401k match | Cell phone reimbursement | Dental insurance | Employee assistance program | EquityMid-level Full TimeChicago || Bolingbrook, IL R3d ago
-
Senior Specialist Solutions Architect (AI/ML) GBP 74K-110KAWS | Apache Spark | Artificial Intelligence | Azure | Data ScienceSenior-level Full TimeLondon, United Kingdom3d ago
-
AWS Glue | Airflow | Automation | By Design | CI/CDBlended working arrangements | Defined contribution pension | Employee assistance programme | Enhanced maternity leave | Enhanced paternity leaveSenior-level Full TimeLondon, England, United Kingdom3d ago
-
Lead Analytics Engineer GBP 75K-90KAWS | Amazon Web Services | DBT | Omni | SQL3 office days per week in London | Hybrid remote workSenior-level Full TimeLondon3d ago
-
Senior AI Engineer GBP 70K-98KAgentic Workflows | C++ | Evals | Fine Tuning | LLM orchestrationEnhanced parental leave | Enhanced pension scheme | Hybrid working | L and D budget | Private health insuranceSenior-level Full TimeLondon, United Kingdom3d ago
-
Senior Data Engineer GBP 55K-62KAgile | Azure | Azure Data | Azure Data Factory | Azure DatabricksAnnual leave | Flexible benefits | Income protection | Pension | Private medical insuranceSenior-level Full TimeLeeds, United Kingdom R3d ago
-
Amazon Web Services | Apache Flink | Apache Spark | Avro | AzureCareer progression | Hybrid work | Inclusive team culture | Training and developmentMid-level Full TimeLondon; South East (England)3d ago
-
Machine Learning Engineer GBP 80K-110KAWS | Azure | Best practices | Cloud infrastructure | Cloud platformCoaching | Enhanced parental leave | Flexible working | Hybrid working | Private dentalMid-level Full TimeUK - London3d ago
-
Alerting | Anomaly Detection | Automated testing | CI/CD | Change ManagementSenior-level Full TimeLondon3d ago
-
Forward Deployed Engineer, Applied AI, Google Cloud GBP 97K-140KAPIs | Agent systems | Cloud platform | Conversational AI | Evaluation PipelinesTravel up to 50 percentSenior-level Full TimeLondon, UK; Zürich, Switzerland3d ago