Team Lead - Databricks Engineer
Dublin, Co. Dublin, Ireland
EUR 70K-90K (estimate) Senior-level Full Time
Tasks
- Apply Unity Catalog and IAM best practices for security
- Build structured streaming pipelines for real time processing
- Collaborate on knowledge sharing
- Conduct code and design reviews
- Configure and optimize Databricks clusters and Spark jobs
- Design and develop Databricks lakehouse solutions
- Develop and optimize ETL/ELT pipelines
- Drive Agile Scrum SDLC process
- Implement Delta Live Tables for ingestion and transformations
- Implement monitoring for pipeline performance and data quality
- Lead and mentor data engineers
- Manage infrastructure and resource provisioning with Terraform
Perks/Benefits
Skills/Tech-stack
AWS | Data Modeling | Databricks | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | Git | Glue | IAM | Lakehouse | Lambda | Python | S3 | SQL | Spark | Streaming | Structured Streaming | Terraform | Unity Catalog
Education
N/A
Roles
Data Engineer | Engineer | Lead | Team Lead
Related jobs
-
AWS | Apache Flink | Apache Kafka | Apache Spark | Cloud ComputingAutonomy | Equity opportunities | Flexible time off | Flexible work schedule | Fully remoteSenior-level Full TimeIreland R12h ago
-
AWS | Agent architecture | Alerting | Azure | BenchmarkingSenior-level Full TimeDublin1d ago
-
AI Engineer EUR 48K-75KAWS | Agent Frameworks | Azure | Crossplane | EvaluationHybrid work | Professional development opportunities | Remote flexibilityMid-level Full TimeDublin - Charlotte, Ireland R1d ago
-
Senior Staff Backend Engineer EUR 75K-85KBackpressure | CAP Theorem | Circuit Breaker | Concurrency | Consensus AlgorithmsCareer development | Employee resource groups | Flexible work from home | Internet reimbursement | Mental wellness programsSenior-level Full TimeIreland-Cork R1d ago
-
Software Engineer EUR 78K-108KAPI Development | AWS | Azure | Deep learning | Distributed SystemsHybrid work schedule | Relocation benefitsMid-level Full TimeDublin LMR - Long Mile Works …1d ago
-
Machine Learning Developer (Freelance) USD 116K-116KLLM | Langchain | MLOps | NumPy | PandasEnglish CV required | Flexible project participation | Part-time hours | Project based workMid-level FreelanceIreland - Remote R1d ago
-
Freelance Machine Learning Engineer USD 116K-116KLLM | Langchain | MLOps | Machine Learning | NumPyPart-time project-basedMid-level FreelanceIreland - Remote R1d ago
-
Apache Airflow | CI/CD | DBT | Data Governance | Data meshSenior-level Full TimeCork, Ireland R1d ago
-
Freelance Data Science Engineer (Python & SQL) USD 116K-116KData Analysis | Data Ingestion | Data cleaning | Exploratory Data Analysis | Feature EngineeringFlexible schedule | Part-time hours | Project based workMid-level FreelanceIreland - Remote R3d ago
-
Data Platform Engineer - Fixed Term Contract GBP 70K-90KBatch Processing | CI/CD | Cloud Native | Cloud-native infrastructure | Data ContractsSenior-level Contract TemporaryGermany (Remote) ; Ireland (Remote); Netherlands … R4d ago
-
Bash | Coverage.py | Dagger | Docker | Functional testingFlexible hours | Fully remote work | Project-based collaboration | Supportive global communitySenior-level Full TimeIreland - Remote R4d ago
-
Analytics Engineer EUR 85K-85KApache Airflow | DBT | Data Contracts | Data Governance | Data ModelingMid-level Full TimeRemote - Ireland R4d ago
-
A/B | A/B Testing | AUC | AWS | AzureAutonomy | Dental insurance | Fitness Wellness Support | Fully remote | Health insuranceMid-level Full TimeIreland R5d ago
-
API Integration | CRM | Claude | HubSpot | JavaScriptCareer growth opportunities | Collaborative international team | Continuous learning and development | Flexible work from anywhere | Fully remoteMid-level Full TimeIreland R5d ago
-
API Design | Agent-based | Agent-based Architectures | CI/CD | Cloud ComputingEmployee stock plan | Flexible scheduling | Fully remote | Health insurance | Home office setup allowanceMid-level Full TimeIreland R5d ago
-
Site Reliability Engineer EUR 60K-85KAWS | Ansible | Argo CD | Argo Workflows | CI/CDFlexible work schedule | On-call rotationSenior-level Full TimeIreland, Dublin5d ago
-
AI Engineer (x2) – Legal Innovation Team EUR 64K-85KAudit Logging | Authentication | Data Modeling | Database Design | Document ManagementSenior-level Full Time10 Earlsfort Terrace, Dublin 2, D02 …6d ago
-
Agentic AI - Senior Software Engineer EUR 51K-69KAI SDK | AI System Design | Agent Orchestration | CI/CD | CLISenior-level Full TimeDublin, Ireland6d ago
-
Electromagnetism | Mechanics | NumPy | Numerical Simulation | PandasFlexible workload | Part-time schedule | Project based workSenior-level FreelanceIreland - Remote R6d ago
-
Software Engineer III - AWS EUR 51K-84KAPI | AWS CloudFormation | AWS Glue | AWS Lambda | Amazon AthenaSenior-level Full TimeDublin, Ireland8d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Flexible management approach | Remote or distributed settingMid-level Full TimeDublin, Ireland8d ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Competitive salary | Remote-friendly | Supportive teamMid-level Full TimeGalway, Ireland8d ago
-
Automation | Benchmarking | Build systems | C# | C++Hybrid work model | Off site work flexibility | On site work in IrelandMid-level Full TimeIRL - Leixlip, Ireland8d ago
-
ASIC | Array signal processing | Beamforming | Bit Exact Modeling | C#Mentoring | Professional growth | Work-life balanceSenior-level Full TimeIreland, Limerick8d ago
-
Senior-level Full TimeDublin, Ireland11d ago