Databricks Data Engineer
Tasks
- Apply data quality checks
- Build lakehouse solutions
- Collaborate with stakeholders
- Conduct code reviews
- Curate data products
- Design data pipelines
- Develop data models
- Enforce data governance
- Implement CI/CD
- Implement Delta Lake patterns
- Implement alerting
- Implement logging
- Improve reliability
- Ingest data
- Integrate pipelines with ML workflows
- Manage costs
- Monitor jobs and clusters
- Perform automated testing
- Support production workloads
- Transform data
- Troubleshoot production issues
- Tune Spark performance
- Use version control
Perks/Benefits
Skills/Tech-stack
Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Data Factory | Data Lake Storage Gen2 | Databricks | Delta Lake | MLflow | Python | SQL
Education
N/A
Roles
Related jobs
-
Data Governance | Data Lake | Data Privacy | Data Quality | Data WarehouseCharity initiatives involvement | Flexible working | Health and wellbeing support | Learning hub access | Sustainability initiatives involvementSenior-level Full TimeLuton, GB, LU2 9TN5h ago
-
Senior Cloud Data Security Engineer GBP 70K-80KAWS | Access Control | Access Management | Ansible | Azure25 days annual leave | Healthcare | Pension | Sharesave schemeSenior-level Full TimeNottingham, England, United Kingdom17h ago
-
Senior Data Engineer GBP 46K-51KAzure | Azure DevOps | Azure SQL | CI/CD | Data CatalogFlexible working | Learning and development | PensionSenior-level Full TimePreston, Lancashire, United Kingdom19h ago
-
C++ | JAX | Machine Learning | Performance Tuning | ProfilingMid-level Full TimeLondon, UK20h ago
-
Applied AI Analytics Lead 14 Months FTC GBP 70K-84KAzure Foundry | Copilot Studio | DBT | Data Quality | Microsoft FabricHybrid work modelSenior-level ContractLondon, England, United Kingdom R20h ago
-
Data Engineer GBP 45K-54KAuthentication | Data Governance | Data Ingestion | Data Modeling | Data QualityMid-level Full TimeWaltham Abbey, United Kingdom21h ago
-
AWS SageMaker | Amazon Bedrock | Amazon ECS | Amazon EKS | Amazon S3Senior-level Full TimeGLASGOW, LANARKSHIRE, United Kingdom21h ago
-
API Design | AWS | Amazon Bedrock | Amazon ECS | Amazon EKSMid-level Full TimeGLASGOW, LANARKSHIRE, United Kingdom21h ago
-
Data Engineer - Brook Green Supply GBP 46K-55KApache Airflow | Apache Kafka | Apache Spark | CI/CD | Cloud PlatformsMid-level Full TimeLondon, England, United Kingdom1d ago
-
Machine Learning Engineer GBP 70K-85KAWS | Anomaly Detection | Apache Spark | Azure | Data ProcessingMid-level Full TimeGlasgow Campus, United Kingdom1d ago
-
Lead Data Engineer GBP 86K-106KAWS | Apache Airflow | Azure | CI/CD | DBTFlexible working | Gym membership | Share scheme | Training budget | Volunteering daysSenior-level Full TimeBirmingham, England, United Kingdom1d ago
-
Data Engineer - Claims Analytics GBP 33K-33KAzure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Key VaultAnnual leave increase | Bonus scheme | Cashback | Cycle to work | DiscountsMid-level Full TimeLondon / Hybrid - 37, United … R1d ago
-
AI Engineer GBP 26K-26KAPI | C# | Design Patterns | GitHub | JavaCompany sharesave scheme | Flexible remote or office work | Learning fund | Paid volunteering time | Pension Contribution SchemeMid-level Full TimeLeeds, UK, United Kingdom1d ago
-
Enterprise Data Engineer GBP 42K-42KAgile | Automated testing | Azure | Azure Data | Azure Data FactoryAnnual leave | Flexible benefits | Flexible working | Pension scheme | Personal private health careMid-level Full TimeEnterprise House, United Kingdom1d ago
-
Data Engineer GBP 41K-44KAmazon Redshift | Confluence | Data Warehouse | ETL | JiraBonus | Free bus travel | Hybrid work | PensionEntry-level Full TimeStockport Shared Service Centre, United Kingdom1d ago
-
Data Engineering Technical Lead GBP 70K-80KData Modeling | Data Quality | Data integration | Database Administration | ETLHybrid work model | Mentoring and development | Vendor support coordinationSenior-level Full TimeLondon, GBR, United Kingdom1d ago
-
Applied Scientist III, Intelligent Talent Acquisition - Lead Generation & Detection Services GBP 60K-80KA/B | A/B Testing | Algorithms | B testing | C++Senior-level Full TimeEdinburgh, Scotland, GBR1d ago
-
Investment Data Engineer GBP 75K-96KAI Tooling | Automated reporting | Data Architecture | Data Normalization | Database systemsSenior-level Full TimeLondon, United Kingdom1d ago
-
DataOps Engineer GBP 72K-85KAir-gapped | Automation | Bash | CI/CD | DevSecOpsFlexible working arrangements | Hybrid working | Training and development | Work-life balanceSenior-level Full TimeLondon, GB1d ago
-
Sr. Applied AI Engineer GBP 90K-115KAWS | Azure | GCP | Generative AI | Machine LearningHome-office equipment | Hybrid work model | Self-development budgetSenior-level Full TimeLondon Office1d ago
-
Applied AI Engineer GBP 80K-95KAPI Integration | Generative AI | Machine Learning | Python | REST APIsHome office setup | Hybrid work model | Mentorship | Self-development budgetMid-level Full TimeLondon Office1d ago
-
Staff AI Engineer GBP 70K-90KAWS | Airflow | Batch Processing | Data Pipelines | DockerAnnual wellbeing allowance | Employee stock option program | Paid time off | Parental leave | Retirement contributionsSenior-level Full TimeLondon R1d ago
-
Mid-level Full TimeLeeds/Sheffield/Manchester, UK (Hybrid) R1d ago
-
Senior Data Engineer, Unified Platform GBP 73K-95KAirflow | Amazon S3 | Apache Flink | Apache Kafka | Apache SparkSenior-level Full TimeLondon1d ago
-
Machine Learning Engineer GBP 68K-85KComputer Vision | Convolutional Neural Networks | Deep learning | Diffusion Models | Generative ModelsCycle garage and showers | Dental scheme | Free soft drinks | Hybrid working | On-site barSenior-level Full TimeUnited Kingdom, London1d ago