Sr Databricks Engineer
Tasks
- Build CI CD pipelines for Databricks jobs notebooks and workflows
- Configure Unity Catalog for data governance and security
- Configure cluster sizing and auto scaling for cost and performance
- Design batch and streaming data pipelines on Databricks using Spark
- Implement ETL and ELT workflows on Delta Lake
- Optimize pipeline and query performance using Spark internals
- Select table and file formats for workload and governance
- Tune Spark performance using caching broadcast joins and shuffle management
Perks/Benefits
- N/A
Skills/Tech-stack
ACID transactions | AWS | Apache Spark | Asset bundles | Avro | Azure | Broadcast joins | CDC | CI/CD | Catalyst Optimizer | Cloud platform | Compaction | Data Governance | Data Security | Databricks | Databricks (asset) | Databricks Asset Bundles | Delta Lake | ELT | ETL | File handling | Google Cloud | Google Cloud Platform | Iceberg | JSON | Liquid clustering | ORC | Optimize | Parquet | Partitioning | PySpark | SQL | Scala | Schema evolution | Shuffles | Small file handling | Spark Catalyst optimizer | Spark SQL | Time Travel | Unity Catalog | VACUUM | Z-ordering
Education
N/A
Roles
Data Engineer | Databricks Engineer | Engineer | Senior Data Engineer
Regions
Countries
States
Cities
Related jobs
-
AWS | Azure | Cloud Platforms | Data Analysis | Data PreprocessingAutonomy | Flexible work environment | Fully remote work | Professional growth and continuous learning | Work-life balanceMid-level Full TimeUnited Arab Emirates R13h ago
-
GenAI Software Engineer - Node.js & LLMs AED 228K-240KAWS Bedrock | Agile | Amazon Web Services | Azure OpenAI | CI/CDSenior-level Full TimeDubai, Dubai, United Arab Emirates19h ago
-
Forward Deployed AI Engineer AED 174K-204KAWS | Agentic Workflows | Apache Iceberg | Apache NiFi | AzureContinued Career Development | Employee resource groups | Flexible WFH | Generous PTO | Paid volunteer timeMid-level Full TimeUAE-Dubai, United Arab Emirates R1d ago
-
Data Engineer, Information System, Dubai AED 84K-96KBig Data | Data Engineering | Data Modeling | Data Warehousing | Distributed SystemsMid-level Full TimeDubai, Dubai, United Arab Emirates2d ago
-
Dell AI Infrastructure & MLOps Engineer - (6 Month Only) AED 156K-192KAI Stack | Apache Spark | Best practices | Dell PowerScale | Dell R570Mid-level Full TimeDubai, Dubai, United Arab Emirates2d ago
-
Generative AI / Azure AI Engineering Lead AED 192K-234KAI Foundry | AI Search | Active Directory | Agent systems | Agentic AISenior-level Full TimeDubai, Arenco Tower, United Arab Emirates2d ago
-
Sr. Data Engineer - PySpark (Banking) AED 150K-228KAgile | Apache Spark | CI/CD | Data Mart | Data TransformationSenior-level Full TimeDubai, Dubai, United Arab Emirates5d ago
-
Senior Full Stack Engineer - GenAI AED 216K-240KAI Agents | AWS | Agent Frameworks | Anthropic | App integrationsSenior-level Full TimeDubai, Dubai, United Arab Emirates5d ago
-
Data Intelligence Machine Learning Engineer AED 144K-162KAWS | Active Learning | Azure ML | DVC | Data GenerationMid-level Full TimeUnited Arab Emirates - Dubai Office5d ago
-
Senior Data Intelligence Machine Learning Engineer AED 192K-240KAWS | Active Learning | Azure | Azure ML | DVCSenior-level Full TimeUnited Arab Emirates - Dubai Office5d ago
-
Lead Data Intelligence Machine Learning Engineer AED 174K-240KAWS | Active Learning | Azure ML | Computer Vision | DVCSenior-level Full TimeUnited Arab Emirates - Dubai Office5d ago
-
Data Intelligence MLOps Engineer AED 240K-312KAWS SageMaker | Airflow | Azure Machine Learning | Bash | CI/CDSenior-level Full TimeUnited Arab Emirates - Dubai Office5d ago
-
Senior-level Full TimeUnited Arab Emirates - Dubai Office5d ago
-
Senior-level Full TimeDubai, Arenco Tower, United Arab Emirates5d ago
-
Engineer, Data & Agentic AI AED 120K-144KData Engineering | Data Modeling | ETL | Language Processing | MLOpsCareer development | Performance-based culture | Training opportunitiesMid-level Full TimeAbu Dhabi, United Arab Emirates6d ago
-
Data Insights & Analytics Lead USD 120K-180KAmplitude | Blockchain Analytics | Contract analytics | Data Visualization | Dune AnalyticsCollaborative work culture | Dental insurance | Global team | Health insurance | Professional developmentSenior-level Full TimeCayman R6d ago
-
Access Control | Data Governance | Data Lineage | Data Pipelines | Data QualitySenior-level Full TimeDubai, Dubai, United Arab Emirates6d ago
-
APIs | AWS Lambda | AWS RDS | AWS S3 | AWS SNSContinuous learning | Flexible work environment | High autonomy | Professional growthSenior-level Full TimeUnited Arab Emirates7d ago
-
Senior Software Engineer - AI Agents & GenAI AED 180K-228KAPI Keys | CI/CD | Function Calling | Git | GuardrailsSenior-level Full TimeDubai, Dubai, United Arab Emirates7d ago
-
Engineer, Data & Agentic AI AED 120K-144KCloud Computing | Data Pipelines | Deep learning | ETL | Language ProcessingTraining and developmentMid-level Full TimeAbu Dhabi, United Arab Emirates8d ago
-
Senior-level Full TimeDubai, Dubai, United Arab Emirates8d ago
-
Data Architect - Treasury AED 240K-323KAccess Control | Amazon Web Services | Azure Synapse | Azure Synapse Analytics | Batch Data IntegrationCareer advancement | Certification support | Employee incentive program | Health insurance | Professional development supportSenior-level Contract Full TimeDubai, Dubai, United Arab Emirates8d ago
-
Agentic DevOps Lead – Data & AI AED 264K-323KAWS | Autogen | CI/CD | Cloud Architecture | Cloud platformCareer progression | Competitive rewards | Continuous coaching | Flexible work arrangements | Travel as neededSenior-level Full TimeDubai, Arenco Tower, United Arab Emirates8d ago
-
Mid-level Full TimeDubai, United Arab Emirates9d ago
-
Senior-level Full TimeDubai, United Arab Emirates12d ago