Senior Data Engineer - Databricks
Tasks
- Apply CI CD best practices for pipeline development
- Apply role based access control and secrets management
- Build automation to reduce manual intervention
- Build data quality monitoring and observability
- Design and build Databricks pipelines to ingest transform and serve ERP and CRM data
- Enforce data quality and incremental processing patterns
- Enforce multi tenant data isolation patterns
- Implement Delta Lake architecture including schema design and partitioning
- Implement data security and compliance best practices
- Integrate data pipelines with AI and analytics ecosystem
- Maintain and report SLA performance metrics
- Maintain technical documentation runbooks and architecture decision records
- Migrate legacy ETL and ELT pipelines to Databricks
- Monitor, alert, and respond to incidents
- Optimize data pipelines to reduce compute costs and processing windows
- Own Databricks production support
- Perform on call incident response across time zones
- Provision validate and harden tenant data pipelines
Perks/Benefits
Skills/Tech-stack
ACID transactions | AWS | Access Control | Apache Spark | Automated testing | Azure | Azure Databricks | CI/CD | Data Governance | Data Observability | Data Partitioning | Data Quality | Data Security | Databricks | Delta Lake | Deployment tooling | ELT | ETL | Incremental processing | Multi-tenant | Multi-tenant architecture | PostgreSQL | PySpark | Python | Role-Based Access | Role-Based Access Control | SQL | Schema evolution | Secrets management | Streaming | Tenant architecture | Version control
Education
N/A
Roles
Related jobs
-
Analytical Engineer PLN 249K-284KApache Airflow | DBT | Data Contracts | Data Modeling | Data ObservabilityBreakfast and lunch catering | Career growth budget | Dental coverage | Family forming support | Fertility healthcare supportSenior-level Full TimeWarsaw R1d ago
-
Power BI Engineer PLN 153K-218KAgile | Analytics engineering | DAX | DBT | Data ModelingAgile/Scrum environment | Flexible schedule | Flexible working hours | Hybrid or remote workplace | Technical equipmentMid-level Full TimeKraków, Lesser Poland Voivodeship, Poland - … R1d ago
-
Senior Machine Learning Engineer (E3) PLN 283K-358KAB Testing | AWS | BigQuery | Cloud Dataflow | Data DriftSenior-level Full TimeKrakow, Poland R1d ago
-
Generative AI Pipeline Engineer (Tech Lead) PLN 180K-360KAPI Integration | Agentic Workflows | Blender | CAD | ComfyUIFitness allowance | Home office support | Medical insurance coverage | Ongoing learning | Paid vacationSenior-level Full TimeWarsaw, Remote, PL R2d ago
-
AWS | Azure | Data Analysis | GCP | GitAutonomy | Continuous learning | Flexible working environment | Fully remote work | Opportunity to contribute to impactful projectsMid-level Full TimePoland R2d ago
-
Analyst II, Full Stack (Revenue Analytics) PLN 226K-346KAutomation | Cloud Computing | DBT | Data Governance | Data ModelingEmployee Resource and Community Groups | Employee stock purchase plan | Flexible spending wallets | Learning and development programs | Parental benefitsMid-level Full TimeRemote Poland R2d ago
-
Airflow | ArgoCD | BigQuery ML | CI/CD | Cloud RunCafeteria fringe benefits | English classes | Flexible working hours | Hackathons | MacBook or Dell equipmentEntry-level Full TimeWarszawa, PL, 00-841 R3d ago
-
LLM Engineer - Freelancer PLN 190K-276KAnthropic | Azure | Cost Optimization | Embeddings | LLM APIsFreelance work | Full-time work | Part-time workMid-level FreelanceWrocław, Lower Silesian Voivodeship, Poland - … R3d ago
-
Senior Data Engineer (Poland, Contract) PLN 152K-171KAWS ECS | Agile | Amazon Aurora | Apache Spark | AutomationAnnual company retreat | Confidential advice line | Flexible working hours | Remote work | Wellness supportSenior-level ContractPoland R7d ago
-
Senior AI Engineer PLN 242K-242KAPI Gateway | API Management | AWS Bedrock | AWS ECS | AWS ECS FargateEmployer supported pension plan | Extra paid holidays | Flexible benefits | Home-office allowance | Hybrid work modelSenior-level Full TimeWarszawa, Mazowieckie, PL R7d ago
-
Senior Cloud Data Engineer (Azure Fabric Dynamics) PLN 266K-392KApache Spark | Azure | CI/CD | Cost estimation | Data LakehouseAI tools | Remote work | Training and certificationsSenior-level Full TimeGliwice R9d ago
-
AI Developer Enablement Lead PLN 276K-408KArtificial Intelligence | CI/CD | Change Management | Code generation | Code reviewCalm app subscription | Days of disconnect | EAP support | Fitness reimbursement | Medical, dental & vision coverageSenior-level Full TimePoland R9d ago
-
Middle Data Engineer PLN 206K-328KApache Airflow | Apache Spark | Data Modeling | Data Observability | Data Pipeline MonitoringBonuses | Corporate gifts | Educational courses | English classes | Hybrid workSenior-level Full TimePoland - Remote R9d ago
-
Database Administrator Engineer GBP 50K-55KDMV | Database performance | Database performance tuning | Deadlock troubleshooting | Execution PlansFlexible work environment | Referral bonusMid-level ContractHybrid (United Kingdom); Remote (Poland) R9d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R10d ago
-
Senior Java Engineer PLN 231K-328KBig Data | Concurrency | Data Governance | Data Lakes | Data ManagementSenior-level ContractRemote (Poland) R10d ago
-
Mid/Senior SASE Automation Engineer PLN 237K-353KAnsible | CI/CD | Continuous integration | Docker | NGFWFlexible hours | Hands-on onboarding | Hybrid work | Internal training sessions | Remote workSenior-level Full TimeWarszawa, Poland R11d ago
-
Cyber Security AI Engineer (Cybersecurity Platforms) PLN 237K-353KAgent Orchestration | Azure | CI/CD | Django | DockerCompany events | Discounts and perks | Employee capital plans | Fitness and wellness programs | Free parkingSenior-level Full Time(HE)Remote KRK Pawia, Poland R11d ago
-
Staff/Lead Python Engineer (FastAPI, Orchestration) EUR 82K-112KAsynchronous programming | Cloud platform | Concurrent programming | FastAPI | GenAIAnnual learning and development budget | Co-working budget | Company laptop | Enhanced parental leave | Flexible scheduleSenior-level Full TimeRemote Anywhere R12d ago
-
Lead AI Engineer (Python) PLN 254K-380KAWS | CI/CD | Data Warehousing | Langchain | Language ModelsEmployee workshops | Flexible working hours | Life insurance | Paid annual service break | Paid public days off PolandSenior-level Full TimeKrakow/Remote within Poland R14d ago
-
Engineering - Internal AI Transformation USD 74K-110KAPIs | Agentic Workflows | Auditability | Authentication | DockerAnnual company offsite | Co-working stipend | Learning and development stipend | Social travel stipendEntry-level Full TimeUnited States R14d ago
-
Principal Data Engineer PLN 270K-414KAnalytics | Apache Spark | Cloud platform | Data Architecture | Data GovernanceE-learning access | Hybrid work | Life insurance | Lunch subsidy | Medical insuranceSenior-level Full TimeŁódź, PL, 90-118 R14d ago
-
Senior Data Engineer PLN 258K-360KAWS | AWS Glue | Amazon Kinesis | Apache Airflow | Apache FlinkAutonomy | High-impact projects | Influence | Mentorship | Room to growSenior-level Full TimePoland - Remote R14d ago
-
Principal Data Engineer with GCP PLN 22K-34KApache Spark | Cloud platform | Databricks | Google Cloud | Google Cloud PlatformE-learning platform | Hybrid work | Life insurance | Lunch subsidy | Medical insuranceSenior-level Full TimeŁódź, PL, 90-118 R14d ago
-
Staff AI Engineer - AI Infrastructure & Agentic Platform PLN 355K-384KAWS | Agentic Frameworks | Amazon Bedrock | Asynchronous programming | ConcurrencyAccident insurance | Additional vacation days | Life insurance | Private medical insurance | Remote work optionsSenior-level Full TimePoland R15d ago