Staff Engineer, Datalake Platform
Tasks
- Architect unified Iceberg platform
- Build developer facing client libraries
- Coordinate cross team migration planning
- Define API contracts for Iceberg
- Define access control policy design
- Design authorization model
- Design automated self service tooling for cost efficiency
- Design metastore service for Iceberg table management
- Design object storage abstraction layer
- Enforce data segregation and lifecycle controls
- Ensure backward compatibility and rollback
- Identify storage inefficiencies at petabyte scale
- Implement audit logging and access review infrastructure
- Implement per table credential vending
- Lead compliance architecture
- Mentor senior engineers through architecture decisions
- Migrate Hive Metastore workloads to new platform
- Own technical design reviews and reliability standards
Perks/Benefits
Skills/Tech-stack
Access Control | Amazon S3 | Apache Flink | Apache Iceberg | Apache Spark | Audit Logging | Authorization | Azure Blob | Azure Blob Storage | Blob Storage | Bucket Provisioning | Compaction | Compliance | Compute Engine | Compute Engine Integration | Data Integrity | Data Migration | Data Segregation | Distributed Storage | Distributed Systems | Engine integration | Hive Metastore | IAM | Lifecycle Management | Object storage | PyIceberg | REST API | Rollback Strategy | Snapshot management | Storage Abstraction | Trino
Education
N/A
Roles
Related jobs
-
AWS | Apache Kafka | Automated testing | CI/CD | Caching100 percent remote | Company offsite weeks | Engineering Ownership | Flexible work location | High-growth environmentSenior-level Full TimeIreland R1d ago
-
Alerting | Data Pipelines | Distributed Systems | GPU Performance | InferenceMid-level Full TimeIreland R1d ago
-
LLM Application Engineer EUR 49K-67KAPIs | Agentic Workflows | Backend Development | Context engineering | DatabasesMid-level Full TimeIreland R1d ago
-
Staff Engineer - Data Platform Experience EUR 86K-126KApache Kafka | Cloud Platforms | Cloud cost allocation | Cost Allocation | Developer ProductivityBest in class onboarding | Continuous career development | Cross departmental buddy program | Employee stock purchase plan | Hybrid workplaceSenior-level Full TimeDublin, Ireland; Madrid, Spain; Paris, France R2d ago
-
Senior AI Engineer (Agents) EUR 39K-54KAI Agents | AWS | Azure | Best practices | CI/CDRemote/hybrid workSenior-level Full TimeCork, Ireland R5d ago
-
AWS | AWS CloudFormation | AWS Glue | AWS Glue Catalog | AWS LakeCollaborative Agile environment | Flexible work location | Fully remote | Technical autonomySenior-level Contract Full TimeIreland R6d ago
-
AI Agents | APIs | Artificial Intelligence | CI/CD | Data schemasCareer growth opportunities | Collaborative culture | Continuous learning support | Flexible working environment | Inclusive workplace focused on innovation trust and meaningful impactSenior-level Full TimeIreland6d ago
-
Software Development Engineer - Data Platform EUR 72K-108KAWS | Agile | ArgoCD | Bare Metal | CI/CDCompany benefits | Flexible work schedule | On-call support | Remote work optionSenior-level Full TimeIreland, Dublin7d ago
-
(Senior) Analytics Engineer (m/w/d) EUR 50K-50KAzure | DAX | Data Modeling | Databricks | Low CodeBlood donation during work hours | Buddy program | Business bike | Corporate benefits | Flexible work hoursSenior-level Full TimeOffice Münster, Germany7d ago
-
Senior-level Full TimeIreland, Limerick7d ago
-
Senior Data Engineer EUR 71K-87KApache Spark | Caching | Cluster optimization | Databricks | Execution PlansBike to work | Career mentorship | Educational assistance | Employee assistance program | Employee discountSenior-level Full Time InternshipDublin, County Dublin, Ireland8d ago
-
API Integration | Access Control | Audit Logging | CI/CD | Cloud DataAnnual offsite gatherings | Dependent coverage support | Employee equity | Flexible, asynchronous work | Fully remoteMid-level Full TimeIreland R8d ago
-
Senior AI Engineer-1 USD 140K-254KAPIs | Agentic Systems | Artificial Intelligence | Data Pipelines | Data PreparationSenior-level Full TimeDublin, Ireland (Mountain View)8d ago
-
Sr. Software Engineer (Distributed Systems) EUR 96K-144KAgile | ArgoCD | CI/CD | Cassandra | ChefFlexible work schedule | In office at least half of time quarterly | Remote work optionSenior-level Full TimeIreland, Dublin8d ago
-
Senior Software Engineer, Atlas Clusters EUR 80K-115KAWS | Azure | Backend Development | C# | Cloud platformSenior-level Full TimeDublin9d ago
-
Staff Software Engineer, AI Reliability Engineering EUR 235K-295KAPIs | Chaos Engineering | Cloud infrastructure | Distributed Systems | GPUFlexible working hours | Generous vacation | Optional equity donation matching | Parental leave | Visa sponsorshipSenior-level Full TimeDublin, IE9d ago
-
API Design | ARINC | Bare Metal | C++ | DO-178Health benefits | Recovery support | Travel opportunitiesSenior-level Full TimeDublin, Dublin, Ireland9d ago
-
Principal Software Engineer, Data Platform EUR 70K-90KAPI Design | AWS | AWS DynamoDB | Apache Iceberg | Apache KafkaHybrid work modelSenior-level Full TimeDublin, Ireland12d ago
-
AWS | Agentic AI | Apache Spark | Automated testing | CI/CDSenior-level Full TimeDublin, Ireland12d ago
-
Principal Cybersecurity Data Engineer EUR 104K-156KApache Airflow | Apache Kafka | Apache Spark | CI/CD | Cloud NativeFlexible work schedule | Hybrid work modelSenior-level Full TimeIreland, Dublin12d ago
-
Lead Data Engineer – AI & Foundation Models EUR 54K-72KAWS | Apache Spark | Azure | CI/CD | Cloud platformSenior-level Full TimeDublin, Ireland13d ago
-
Associate TSE II EUR 61K-96KAWS | Application Architecture | Automation | Backup | BenchmarkingHybrid workingMid-level Full TimeDublin, Ireland14d ago
-
Senior Developer - DevOps/MLOps EUR 71K-90K.NET | AI Act | AI Foundry | Access Control | Audit LoggingBike to work | Educational assistance | Employee assistance programme | Employee discount | Financial adviceSenior-level Full TimeDublin, County Dublin, Ireland14d ago
-
Software Engineer (Data Governance) EUR 65K-65KAPI Development | Amazon QuickSight | Amazon Redshift | Apache Airflow | Apache DruidMid-level Full TimeDublin15d ago
-
Lead Platform Engineer EUR 55K-72KAWS | AWS Backup | Access Control | Amazon Aurora | Amazon Aurora PostgreSQLSenior-level Full TimeDublin, Ireland15d ago