Team Lead - Databricks Engineer
Dublin, Co. Dublin, Ireland
EUR 70K-90K (estimate) Senior-level Full Time
Tasks
- Apply Unity Catalog and IAM best practices for security
- Build structured streaming pipelines for real time processing
- Collaborate on knowledge sharing
- Conduct code and design reviews
- Configure and optimize Databricks clusters and Spark jobs
- Design and develop Databricks lakehouse solutions
- Develop and optimize ETL/ELT pipelines
- Drive Agile Scrum SDLC process
- Implement Delta Live Tables for ingestion and transformations
- Implement monitoring for pipeline performance and data quality
- Lead and mentor data engineers
- Manage infrastructure and resource provisioning with Terraform
Perks/Benefits
Skills/Tech-stack
AWS | Data Modeling | Databricks | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | Git | Glue | IAM | Lakehouse | Lambda | Python | S3 | SQL | Spark | Streaming | Structured Streaming | Terraform | Unity Catalog
Education
N/A
Roles
Data Engineer | Engineer | Lead | Team Lead
Related jobs
-
Data Engineer, Data Architecture and Engineering EUR 49K-72KBig Data | Code review | Data Infrastructure | Data Modeling | Data ProcessingSenior-level Full TimeDublin, Ireland5h ago
-
Cloud Computing | Data Pipelines | ETL | Google Colab | Information RetrievalCareer growth opportunities | Continuous learning culture | Coworking access | Flexible schedule | Fully remote workMid-level Full TimeIreland R11h ago
-
Amazon Redshift | Code review | DBT | Data Modeling | Data ObservabilityCareer development | Charitable contribution | Fitness reimbursement | Health insurance | Private pensionMid-level Full TimeIreland R1d ago
-
Algorithm Design | Data orchestration | Linux | Machine Learning | PandasMid-level Full TimeDublin, Ireland1d ago
-
AI Software Engineer II (R-18544) EUR 52K-68KAPI Development | Agentic Workflows | CI/CD | FastAPI | GCPEducational assistance program | Employee Health Insurance | Family-friendly leave | Flexible working (hybrid model) | Holiday buy & sellMid-level Full TimeDublin - Ireland R1d ago
-
CI/CD | ClearML | Cloud Native | Git | GoContinuous learning | Mentoring and coaching | Open source community participationSenior-level Full TimeRemote Ireland R1d ago
-
APIs | AWS EMR | AWS Glue | AWS Lambda | AirflowAsync-friendly culture | Conference support | Course support | Flexible working hours | Fully remote workSenior-level Full TimeIreland R2d ago
-
Senior Lead AI Software Engineer EUR 132K-198KData Privacy | Deep learning | Language Processing | MLOps | Machine LearningAccommodation support | Health and wellbeing benefits | Travel up to 10 percentSenior-level Full TimeDublin, IE2d ago
-
Senior Databricks Data Engineer EUR 57K-77KApache Spark | Azure Data | Azure Data Factory | Azure Synapse | Azure Synapse AnalyticsIncome protection insurance | Learning initiatives | Life assurance | Pension | Private healthcareSenior-level Full TimeDublin, Leinster, Ireland2d ago
-
Sr SWE, Fraud Science Data Platform EUR 70K-102KAlgorithmic Complexity | BigQuery | Clustering | Community Detection | Connected componentsFinancial security resources | Healthcare coverage | Hybrid work schedule | Mental health resources | Paid time offSenior-level Full TimeIRL - County Dublin - Dublin …2d ago
-
MATLAB | NumPy | Pandas | Python | RFlexible workload | Freelance work | Part-time projects | Project-based assignmentsSenior-level FreelanceIreland - Remote R2d ago
-
AWS | Apache Airflow | Apache Flink | Apache Kafka | Apache SparkFully paid parental leave | Fully remote-first | Home office stipend | Inclusive diverse workplace | Manager coachingSenior-level Full TimeIreland R2d ago
-
Senior AI Software Engineer EUR 56K-79KAWS | AutoGPT | Azure | CI/CD | CrewAIHybrid work | On site three days per weekSenior-level Full TimeCork, Ireland5d ago
-
Senior Data Engineer EUR 70K-101KAWS | Apache Airflow | Azure | CI/CD | Cloud NativeComprehensive benefits | Flexible hybrid work environment | Pension contribution | Post offer health screenings | Vaccination requirementsSenior-level Full TimeGalway, Ireland5d ago
-
Senior-level Full TimeDublin, Co. Dublin, Ireland5d ago
-
Freelance Machine Learning Engineer USD 116K-116KLLMs | Langchain | MLOps | NumPy | PandasFreelance assignments | Part-time project-based workMid-level FreelanceIreland - Remote R5d ago
-
Machine Learning Developer (Freelance) USD 116K-116KData Analysis | Deep learning | Langchain | MLOps | Machine LearningFlexible workload | Part-time availability | Project based workMid-level FreelanceIreland - Remote R5d ago
-
Principal Data Engineer EUR 70K-90KAWS | Apache Airflow | Apache Kafka | Apache Spark | Batch ProcessingHybrid work model | Remote work flexibilitySenior-level Full TimeIreland - Dublin - Hybrid R5d ago
-
Senior-level Full TimeIE-Dublin-WW6d ago
-
Senior-level Full TimeDublin, Ireland6d ago
-
Freelance Data Science Engineer (Python & SQL) USD 116K-116KLLM | Langchain | MLOps | Matplotlib | NumPyFlexible schedule | Paid work | Part-time work | Project based workMid-level FreelanceIreland - Remote R6d ago
-
Technical Services Engineer EUR 74K-95KAWS | Active Directory | Automation | Azure | Backup and RestoreEmployee wellbeing support | Hybrid work | Mentoring | Parental leaveMid-level Full TimeDublin6d ago
-
Media Insights & Analytics Lead GBP 70K-90KAttribution Modeling | Experimentation | Identity resolution | Incrementality testing | Marketing MixConference Courses Books Development | Health benefits | Mental health care program | Paid time off | Parental leave programSenior-level Full TimeIreland; United Kingdom R7d ago
-
API Integration | Automation | Data Modeling | Indexing | JSONCollaborative fast-paced culture | End-to-end responsibility | Flexible work location | Fully remote | High-ownership environmentSenior-level Full TimeIreland R7d ago
-
Senior-level Full TimeDublin, Ireland7d ago