Data Engineer - Senior
South Africa
R
A ZAR 840K-960K (estimate) Senior-level Contract Full Time
Tasks
- Build and optimize ETL/ELT pipelines
- Build and optimize data processing and storage with Azure Databricks and Delta Lake
- Collaborate with stakeholders to translate requirements into datasets
- Design data models for scalability reliability performance data quality
- Design scalable reliable data solutions
- Develop Azure cloud data solutions
- Develop data transformations with Python and SQL
- Troubleshoot pipeline issues and optimize performance
- Use Git for version control and maintainable collaborative development
Perks/Benefits
Skills/Tech-stack
AI code generation | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure Networking | Azure SQL | Azure Synapse | Azure Synapse Analytics | Code generation | Data Factory | Data Lake Storage Gen2 | Data Modeling | Data Quality | Delta Lake | ELT | ETL | Git | Python | SQL | Synapse Analytics
Education
N/A
Roles
Related jobs
-
APIs | AWS Lambda | AWS S3 | Amazon RDS | Amazon SNSAutonomy | Collaborative team environment | Fully remote | Ownership | Professional development focusSenior-level Full TimeSouth Africa R1d ago
-
Data Engineer ZAR 420K-464KAPI | Artificial Intelligence | DBT | Data Compliance | Data GovernanceCollaborative culture | Employee wellness programs | Hybrid work | Modern tooling | Professional developmentMid-level Full TimeCape Town, South Africa R1d ago
-
AWS | Autogen | Azure | Claude | Cloud platformBackup power required | Fixed internet required | Public holiday compensation | Remote workSenior-level Full TimeSouth Africa - Remote R2d ago
-
AI Evaluation | API Integration | AWS | Access Control | Agent OrchestrationFlexible work hours | Fully remoteSenior-level Full TimeEgypt R3d ago
-
Backpressure handling | Chunking | Cloud platform | Cloudflare Queues | Cloudflare WorkersCompetitive pay | Growth opportunities | Remote workSenior-level Full TimeSouth Africa - Cape Town R6d ago
-
Lead Data and AI Integration Specialist ZAR 540K-720KAWS | Anomaly Detection | Authentication | Automation | AzureEqual opportunity workplace | Hybrid working model | Travel to primary office as requiredSenior-level Full Time15 Alice Lane, South Africa R7d ago
-
AWS | AWS CloudFormation | AWS Glue | AWS Glue Catalog | AWS LakeAgile collaboration | Flexible work location | Fully remote | Technical autonomySenior-level Contract Full TimeSouth Africa R7d ago
-
Full Stack AI Developer ZAR 540K-540KAI workflow | AI workflow automation | Anthropic API | Cloud deployment | DockerFull-time remote work | US business hoursMid-level Full TimeSouth Africa - Remote R11d ago
-
Data Engineer ZAR 396K-456KAPI Development | AWS | Apache Spark | Automation | CI/CDCareer development opportunities | Employee discounts | Hybrid working | Paid sabbaticals | Paid time offSenior-level Full TimeCape Town, Western Cape, South Africa R1mo ago
-
Principal Clinical Data Engineer - based in South Africa ZAR 663K-663K21 CFR | 21 CFR Part 11 | CFR Part 11 | Cause analysis | Clinical dataSenior-level Full TimeSouth Africa-Bloemfontein-Remote R1mo ago
-
Airflow | Amazon Redshift | BigQuery | CI/CD | ChromaDBFlexible working hours | Remote workSenior-level Full TimeSouth Africa - Remote R1mo ago
-
Senior Clinical Data Engineer ZAR 663K-663K21 CFR | 21 CFR Part 11 | CFR Part 11 | Cause analysis | Clinical ResearchSenior-level Full TimeSouth Africa-Bloemfontein-Remote R1mo ago
-
AWS | Amazon Redshift | Apache Airflow | Azure | BigQueryCareer growth opportunities | Collaborative culture | Continuous learning | Flexible working arrangements | Fully remoteSenior-level Full TimeSouth Africa - Remote R1mo ago