Senior Data Engineer (Databricks & Cloud Data Platforms)
Johannesburg, Gauteng, South Africa
ZAR 875K-1100K (estimate) Senior-level Contract Full Time
Tasks
- Build batch and streaming ETL/ELT pipelines
- Build orchestration solutions for data workflows
- Collaborate with stakeholders on solution requirements
- Design scalable Databricks based data platforms
- Develop high-performance data processing solutions
- Implement Delta Lake and Unity Catalog
- Implement governance frameworks
- Lead data migration to cloud lakehouse architectures
- Mentor junior engineers
- Optimize Spark workloads for performance and cost
- Produce technical documentation and architecture artifacts
Perks/Benefits
Skills/Tech-stack
AWS Glue | Airflow | Amazon Redshift | Amazon S3 | Azure | Azure Data | Azure Data Factory | Azure Purview | Azure Synapse | Azure Synapse Analytics | Data Factory | Databricks | Databricks Workflows | Delta Lake | ELT | ETL | Lakehouse Architecture | Medallion Architecture | OneLake | PySpark | Python | SQL | Spark | Synapse Analytics | Unity Catalog
Education
Roles
Related jobs
-
Mid-level ContractJohannesburg, South Africa8h ago
-
Machine Learning Engineer ZAR 420K-480KAutomation | Azure Machine Learning | Data analytics | Databricks | Machine LearningMid-level Full TimeSandton - 1 Discovery Place, GP, …23h ago
-
AI/ML | Access Control | Access Management | Amazon Web Services | AsanaRemote workSenior-level Full TimeArgentina - Buenos Aires1d ago
-
Amazon Redshift | Amazon Web Services | Apache Airflow | Apache Iceberg | Apache KafkaGrowth opportunities | Real time collaboration tools | Remote workMid-level Full TimeSouth Africa - Cape Town1d ago
-
Junior Software Analytics Engineer ZAR 216K-264KAPI Integration | Alerting | Android | Authentication | AuthorizationGreat work environment | Total remuneration flexibilityEntry-level Full TimeCape Town1d ago
-
Junior Software Analytics Engineer ZAR 216K-264KAPI Debugging | API Integration | Android Development | Authentication | AuthorizationEntry-level Full TimeCape Town1d ago
-
Azure | CI/CD | Data Governance | Data Lineage | Data PipelinesSenior-level ContractSandton, South Africa1d ago
-
Data Engineer ZAR 348K-420KAWS | AWS CDK | Azure | Azure Data | Azure Data FactoryGrowth opportunities | Hybrid work environment | Learning opportunities | Medical aid | Provident fundMid-level Full TimeJohannesburg, ZA1d ago
-
AWS | AWS S3 | Agile | Apache Airflow | Apache DataprocCareer growth | Collaborative Agile environment | Flexible working arrangements | International large scale projects | Ongoing training and mentoringSenior-level Full TimeSouth Africa2d ago
-
Computer Vision | Deep learning | Diffusers | Diffusion Models | IP AdaptersAnnual vacation | Corporate discounts for English language learning programs | Fully remote | Health allowance | Professional development reimbursementSenior-level Full TimeSouth Africa R2d ago
-
Agentic pipelines | DPO | Deep learning | Direct Preference Optimization | Distributed TrainingContinuous learning opportunities | English language learning discounts | Fully remote | Health support budget | Home-office allowanceSenior-level Full TimeSouth Africa R2d ago
-
Senior-level Full TimeStellenbosch, ZA3d ago
-
AI readiness | AI workflows | AWS | Access Control | Access ManagementCustomer facing technical role | Real growth opportunities | Remote work opportunities | Vetted employersSenior-level Full TimeArgentina - Buenos Aires6d ago
-
AI Data | AI data pipelines | AWS | Access Management | AzureHybrid work with customer travel | Remote workSenior-level Full TimeHonduras - Tegucigalpa6d ago
-
AWS Glue | AWS Lake Formation | AWS Lambda | AWS S3 | AWS Step FunctionsCareer growth | Flexible work culture | Fully remote workSenior-level Full TimeSouth Africa R6d ago
-
Agentic Workflows | Conversational AI | Dialogue Management | Error Handling | Function CallingCareer growth opportunities | Flexible work location | Fully remote | High technical autonomy | Startup-like environmentSenior-level Full TimeSouth Africa R6d ago
-
Senior-level Full TimeStellenbosch, ZA6d ago
-
Machine Learning Engineer ZAR 700K-1200KAgent systems | Language Processing | MLOps | Machine Learning | Model TrainingSenior-level Full TimeDurban, South Africa7d ago
-
Machine Learning Engineer ZAR 700K-1200KComputer Vision | Docker | Git | Kubernetes | Language ProcessingDirect founder collaboration | End-to-end ownership | Fast-paced environment | High autonomySenior-level Full TimeCape Town, South Africa7d ago
-
Machine Learning Engineer ZAR 700K-1200KComputer Vision | Continuous integration | Deep learning | Docker | KubernetesSenior-level Full TimeJohannesburg, South Africa7d ago
-
Mid-level Full TimeCape Town, South Africa7d ago
-
Mid-level Full TimeCape Town, South Africa7d ago
-
Data Engineer (Senior) ZAR 745K-840KAWS Glue | AWS Lambda | Azure | CI/CD | Data ModelingHybrid workSenior-level Full TimeCenturion, South Africa7d ago
-
AI Applications | AI Automation | AI workflows | APIs | Artificial IntelligenceClient facing travel | Technical mentorship | Training workshops facilitationEntry-level Full TimeRosebank, Johannesburg, Gauteng7d ago
-
Database Engineer / Database Administrator ZAR 464K-504KAWS Aurora | AWS RDS | Agile | Automation | Database AdministrationAdvancement opportunities | Annual tutoring stipend for children | Employee development program stipend | Fertility/adoption assistance | Mentorship programMid-level Full TimeRemote, South Africa R8d ago