DataOps Engineer
Tasks
- Audit data pipelines
- Contribute to CI CD workflows for data pipelines
- Deploy and manage data pipeline versions
- Detect data anomalies
- Document SOPs and troubleshooting guides
- Enhance platform observability with logging metrics alerting
- Ensure timely execution of data pipelines
- Implement data quality checks
- Improve pipeline performance
- Maintain OLAP lake storage and materialized views
- Monitor and troubleshoot orchestration failures
- Operate data ingestion layers
- Support internal users with query performance and data issues
- Validate data schema
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Airflow | Alerting | Anomaly Detection | Apache Doris | Audit Logging | Azure | CI/CD | ClickHouse | Data Lake | Data Quality | Data Warehousing | Delta Lake | Docker | ELK | Flink | GCP | Grafana | Hudi | Iceberg | Kafka | Kubernetes | Logging | Metrics | OLAP | Partitioning | Prometheus | Python | SQL | Schema Validation | Schema evolution | Spark | Workflow Orchestration
Education
N/A
Roles
Related jobs
-
Senior Data Engineer GBP 63K-70KAzure | CI/CD | Data Lineage | Data Quality | DatabricksCareer development opportunities | Flexible working arrangements | Inclusive work environmentSenior-level Full TimeLondon, London, GB6h ago
-
Senior Analytics Engineer GBP 55K-65KAmazon Redshift | BigQuery | DAG | Data Modeling | Data TransformationCharitable time off | Generous annual leave | Healthcare | Parental leave | PensionSenior-level Full TimeLondon - The River Building HQ23h ago
-
Senior AI Engineer, London GBP 85K-110KCloud Computing | Data Processing | Deep learning | Distributed Systems | LLMCompany pension | Cycle to work scheme | Income protection and life insurance | Mental health support | Paid time offSenior-level Full TimeLondon23h ago
-
Data Engineer GBP 70K-100KAPI Integration | AWS | Amazon Redshift | Apache Airflow | Apache ParquetMid-level Full TimeManchester, United Kingdom1d ago
-
Senior-level Full TimeLeicester, Leicestershire, United Kingdom1d ago
-
Machine Learning Engineer GBP 26K-26KCausal Inference | Deep learning | Information Retrieval | Language Models | Language ProcessingCompetitive compensation with equity | Continuous learning | Flexible working | High ownership | Hybrid workMid-level Full TimeLondon - The River Building HQ1d ago
-
Senior Machine Learning Engineer GBP 70K-92KCausal Inference | Deep learning | Explore/Exploit | Information Retrieval | Language ModelsCompetitive salary | Continuous learning | Equity options | Flexible working | Hybrid workSenior-level Full TimeLondon - The River Building HQ1d ago
-
Data Warehouse Developer, GCP GBP 68K-80KAI Agents | Analytics hub | BigLake | BigQuery | Cloud ComposerHybrid work from homeEntry-level Full TimeManchester, England, United Kingdom1d ago
-
Staff Data Engineer GBP 70K-100KAWS | Advanced Analytics | Batch Ingestion | BigQuery | Bronze Silver GoldEmployee wellbeing programme | Enhanced maternity/paternity | Hybrid work | Learning budget | Pension matchSenior-level Full TimeLondon1d ago
-
Machine Learning Engineer GBP 42K-50KAI Pipelines | BigQuery | CI/CD | Cloud Build | Cloud ComposerDental insurance | Enhanced parental leave | Hybrid work | Learning budget | Pension matchMid-level Full TimeLondon1d ago
-
Lead Machine Learning Engineer GBP 70K-87KAI Agents | AI Pipelines | BigQuery | CI/CD | Cloud BuildDental insurance | Employee wellbeing program | Enhanced maternity leave | Enhanced paternity leave | Hybrid workSenior-level Full TimeLondon1d ago
-
Embedded Software Quality Engineer GBP 45K-60KAS9115 | CI/CD | Code Analysis | Code Coverage | CppcheckDisability income protection | Employee assistance program | Life insurance | Pension plan | Private healthcareMid-level Full TimeFarnborough, Hampshire, United Kingdom1d ago
-
Machine Learning Engineer GBP 50K-65KAzure ML | Azure ML pipelines | CatBoost | Data Analysis | Data CleansingMid-level Full TimeUK025 - Erskine,United Kingdom(UK025)1d ago
-
Senior Data Quality Engineer GBP 72K-80KAirflow | Apache Beam | Apache Kafka | BDD | BehaveFlexible working | Hybrid working | Job share | Parental leave | Pension contributionSenior-level Full TimeBristol Harbourside, United Kingdom1d ago
-
AWS | Apache Spark | Autogen | CrewAI | DatabricksCycle to work scheme | Discount vouchers | Gym membership | Private healthcareEntry-level Apprenticeship Full TimeUK113 - Newcastle UK (UK113), United …1d ago
-
Amazon Web Services | Autogen | CrewAI | Databricks | ETLCycle to work scheme | Discount vouchers | Gym membership | Mentoring | Private healthcareEntry-level Apprenticeship Full TimeUK113 - Newcastle UK (UK113), United …1d ago
-
Senior Quantitative Developer GBP 50K-63KAgile | Behavior-Driven Development | CI/CD | Containerization | Derivatives pricingSenior-level Full TimeUK - Park House, United Kingdom1d ago
-
Agile | Airflow | CI/CD | DBT | Data MonitoringAnnual leave | Contributory pension scheme | Enhanced family leave benefits | Flexible holiday scheme | Gym discountSenior-level Full TimeROL Dragon Court, United Kingdom1d ago
-
Senior Software Engineer (Backend) GBP 74K-90KApache Kafka | CI/CD | Docker | Elasticsearch | GitHub ActionsSenior-level Full TimeLondon, London, City of, United Kingdom … R1d ago
-
AWS | Autosys | Azure | CI/CD | Cloud PlatformsAnnual leave | Employee assistance program | Hybrid work option | Learning and development resources | Life insuranceExecutive-level Full Time60 SYDENHAM ROAD, TITANIC QUARTER BLOCKS …1d ago
-
Analytics Engineer GBP 50K-75KAzure | CI/CD | Cloud Data | Cloud Data Warehouse | DBTEnhanced parental leave | Flexible benefits budget | Hybrid working | Income protection | Life insuranceEntry-level Full TimeLondon1d ago
-
Data Engineer GBP 60K-60KAzure Data | Azure Data Factory | Azure Data Lake | Azure SQL | Continuous DeliveryAnnual leave | Buy additional leave | Discounts and cashback | Health care cash plan | Life assuranceMid-level Full TimeUnited Kingdom, United Kingdom, United Kingdom1d ago
-
Machine Learning Engineer (Recommenders) GBP 65K-80KBatch Processing | CI/CD | Containerization | Deep learning | Distributed ComputingEmployee communities | Employee discount | Flexible benefits allowance | Online learning resources | Paid annual leaveMid-level Full TimeLondon, England, United Kingdom1d ago
-
GCP Data Engineer GBP 100K-125KApache Airflow | BigQuery | CI/CD | Cloud Composer | Cloud StorageHybrid workingExecutive-level Full TimeEdinburgh, Scotland, United Kingdom1d ago
-
BI Analyst GBP 65K-85KAnomaly Detection | Azure AI | Data Modeling | Data Warehousing | ELTEmployee stock purchase options | Health insurance | Learning programs | Paid time off | Product DiscountsMid-level Full TimeLeeds, United Kingdom1d ago