Databricks Test Automation Engineer
Dublin, Co. Dublin, Ireland
EUR 45K-66K (estimate) Mid-level Full Time
Tasks
- Automate testing for data pipelines
- Create automated checks for data transformations
- Create automated test dashboards
- Design and execute data quality test strategies
- Develop automated testing frameworks for Databricks notebooks
- Generate quality metrics and testing reports
- Implement automated data reconciliation
- Implement automated validation for structured streaming pipelines
- Implement pipeline monitoring test frameworks
- Maintain test documentation
- Perform Spark job testing and optimization validation
- Perform performance testing for Spark jobs
- Test Delta Lake tables and ACID transactions
- Test ETL/ELT processes
- Test Unity Catalog features and access controls
- Validate Delta Live Tables
- Validate data quality with automated tests
Perks/Benefits
- N/A
Skills/Tech-stack
ACID transactions | AWS Glue | AWS Lambda | AWS S3 | Agile | Apache Spark | Data Governance | Data Quality | Databricks | Delta Lake | Delta Lake ACID transactions | Delta Live Tables | Delta Live) | ELT | ETL | Git | Lakehouse | PySpark | SQL | Structured Streaming | Test automation | Unity Catalog
Education
Related jobs
-
Analytics Engineer EUR 85K-85KApache Airflow | DBT | Data Contracts | Data Documentation | Data ModelingMid-level Full TimeRemote - Ireland R1d ago
-
Staff Robotics AI/ ML Engineer EUR 95K-135KCI/CD | Computer Vision | Control loops | Decision Making | Decision making under uncertaintySenior-level Full TimeIreland, Limerick2d ago
-
Executive-level Full TimeAthlone, Ireland4d ago
-
Principal AI Engineer EUR 68K-90KCI/CD | Cloud Computing | Containerization | Data Governance | Data QualitySenior-level Full TimeDublin, Ireland5d ago
-
Bash | Benchmarking | C# | C++ | ConcurrencyHybrid work model | On site and off site flexibilityMid-level Full TimeIRL - Leixlip, Ireland5d ago
-
Senior Developer, Agentic AI Engineering EUR 60K-80KA2A | AWS | Agent to Agent | Asyncio | BlackFlexible work schedule | Professional development support | Remote and in-office flexibilitySenior-level Full TimeIreland, Dublin5d ago
-
Senior Machine Learning Engineer EUR 60K-78KCUDA | Distributed Training | GPU Acceleration | Generative AI | Language ModelsSenior-level Full TimeDublin, County Dublin, Ireland5d ago
-
Software Developer, Scaled Ops AI Acceleration Team GBP 110K-130KAI infrastructure | Artificial Intelligence | Data Analysis | Data Mining | DebuggingSenior-level Full TimeDublin, Ireland | London, UK5d ago
-
Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure SQLCompany trips | Equipment provided | Flexible working hours | Health insurance | Pension schemeSenior-level Full TimeIreland - Remote R6d ago
-
Access Management | Auditability | Azure | Azure Data | Azure Data LakeEmployee assistance programme | Family leave options | Healthcare Scheme | Hybrid working | Pension schemeSenior-level Full TimeDublin, IE, IE6d ago
-
Senior-level Full TimeDublin, Ireland7d ago
-
Senior Data Engineer EUR 55K-65KAgile | Apache NiFi | Apache Spark | Cloud Native | Cloud-native developmentSenior-level Full TimeDublin, Ireland7d ago
-
Data Analysis | Data Ingestion | Data cleaning | Exploratory Data Analysis | Feature EngineeringFlexible schedule | Part-time freelance | Remote workMid-level FreelanceIreland - Remote R7d ago
-
Embedded Software Engineer EUR 66K-72KC# | C++ | Debugging | Embedded Linux | EthernetCareer growth opportunities | Collaborative environment | Comprehensive benefits | Flexible work schedules | Innovation in space technologySenior-level Full TimeGalway, Ireland11d ago
-
Senior Machine Learning Engineer EUR 60K-78KAI | AWS | Deep learning | Docker | Git25 days holidays | Flexible work location | Health insurance | Inclusive environment | Life insuranceSenior-level Full TimeIreland12d ago
-
Quantitative Developer - Asset Based Finance EUR 71K-106KCloud Architecture | DBT | Dagster | Data Governance | Data ModelingSenior-level Full TimeDublin12d ago
-
Machine Learning Engineer EUR 35K-90KAirflow | Azure Cloud | Big Data | Big Data Technologies | Data LakeMid-level Full TimeGalway-Bonham Quay Dock Road, Ireland13d ago
-
Machine Learning Engineer II EUR 49K-66KCloud Platforms | Data Visualization | Docker | Hadoop | KubernetesFlexible work arrangementsMid-level Full TimeDublin, IE13d ago
-
Senior Databricks Test Automation Engineer EUR 54K-77KAPI Development | AWS | Data Modeling | Databricks | Delta LakeFlexible working arrangements | Inclusive culture | Training and education supportSenior-level Full TimeDublin, Co. Dublin, Ireland14d ago
-
Sr Machine Learning Engineer EUR 64K-96KAWS | Azure | BigQuery | Cloud platform | Computer VisionFlexible work model | Healthcare coverage | Mental health resources | Paid time offSenior-level Full TimeIRL - County Dublin - Dublin …15d ago
-
Data Engineer (Data & AI) EUR 43K-70KAI | Analytics | CI/CD | Data Governance | Data ManipulationBonus | Flexible locations | Learning and development | Pension scheme | ReferralsSenior-level Full TimeDublin/Hybrid R15d ago
-
C# | Java | Message Queues | Metadata-Driven Development | Object-OrientedFlexible work arrangements | Inclusive cultureSenior-level Full TimeIreland, Dublin16d ago
-
Data Modelling | Databricks | ETL | Java | Microsoft FabricCareer development support | Global opportunities | Hybrid remote work | Relocation assistance | Visa sponsorshipSenior-level Full TimeRemote , Ireland R16d ago
-
Data Engineer - Ireland EUR 39K-53KAlgorithms | Apache Hive | Apache Kafka | Apache Spark | BigQueryMid-level Full TimeIreland17d ago
-
Senior Data Engineer - Ireland EUR 54K-70KAirflow | Cloud Platforms | Composer | Data Governance | Data LakehouseSenior-level Full TimeIreland18d ago