Senior Data Engineer
Tasks
- Architect lakehouse data solutions
- Build CI/CD pipelines
- Build and maintain data pipelines
- Collaborate with data scientists and analysts
- Create technical documentation
- Deploy containerized solutions
- Design scalable ETL and ELT workflows
- Develop data models and schemas
- Implement data governance and security controls
- Implement real-time streaming with Kafka
- Mentor junior data engineers
- Monitor infrastructure performance and optimize cost
- Optimize distributed data processing with Spark
- Process large volumes of structured and unstructured data
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Amazon S3 | Apache Airflow | Apache Spark | Azure | Azure Blob | Azure Blob Storage | Blob Storage | CI/CD | Data Governance | Data Modeling | Databricks | Dimensional Modeling | Docker | Flink | Git | Google BigQuery | Google Cloud | Informatica | Java | Kafka | Kubernetes | Python | SQL | Scala | Spark Streaming | Talend
Education
N/A
Roles
Related jobs
-
Senior-level Full TimeCape Town2d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudMid-level Full TimeJohannesburg, South Africa2d ago
-
Entry-level Full TimeStellenbosch, ZA2d ago
-
Data Analytics Technical Specialist ZAR 780K-900KAccess Management | Analysis Services | Azure DevOps | Azure SQL | Azure SQL DatabaseFlexible working | Hybrid workingSenior-level Full TimeCape Town, South Africa3d ago
-
AI Solutions Engineer ZAR 400K-464KAI Agents | API Integration | Azure AI | Chatbots | Data AnalysisMid-level Full TimeCape Town, South Africa Office3d ago
-
Senior Specialist Machine Learning Engineer ZAR 700K-840KAWS EC2 | AWS IAM | AWS S3 | Access Control | Amazon CloudWatchMedical aid | Retirement benefits | Risk benefitsSenior-level Full TimeMidrand, ZA3d ago
-
Entry-level Full TimeAbsa 270 Republic Road, South Africa3d ago
-
Senior Machine Learning Engineer ZAR 745K-860KAWS SageMaker | AWS Sagemaker Endpoints | Agile | Amazon Web Services | Big DataBirthday leave | Confidential counselling | Financial guidance | Free parking | Legal supportSenior-level Full TimeSouth Africa4d ago
-
Senior Machine Learning Engineer ZAR 745K-860KAWS | Agile | Big Data | BigQuery | Cloud platformBirthday leave | Confidential counselling | Financial guidance | Free parking | Free subscription takealotMORESenior-level Full TimeSouth Africa4d ago
-
Apache Spark | Azure Data | Azure Data Factory | Azure Databricks | Azure EventMid-level ContractSandton, South Africa4d ago
-
API | AWS | AWS Lambda | Amazon RDS | Amazon S3Autonomy | Collaborative team | Continuous learning | High ownership | Professional growthSenior-level Full TimeSouth Africa4d ago
-
Senior Data Scientist (Azure Data Engineering & MLOps) ZAR 900K-1200KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Contract Full TimeCape Town, Western Cape, South Africa4d ago
-
Senior Data Scientist (Azure Data Engineering & MLOps) ZAR 900K-1200KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Contract Full TimeCape Town, Western Cape, South Africa4d ago
-
Specialist Support Engineer: DataOps ZAR 420K-464KApache Spark | Batch Processing | Cloud Migration | Data Modeling | Data StreamingMid-level Full Time15 Alice Lane, South Africa4d ago
-
AI readiness | AWS | Access Management | Architecture | AsanaCareer growth opportunities | Customer facing exposure | Remote workSenior-level Full TimeHonduras - Tegucigalpa5d ago
-
AWS | Access Control | Access Management | Azure | Cloud hybridRemote workSenior-level Full TimeArgentina - Buenos Aires5d ago
-
Agile | Amazon Redshift | Amazon Web Services | Apache Airflow | Apache IcebergRemote workMid-level Full TimeSouth Africa - Cape Town5d ago
-
Technology/Domain Specialist II - Kafka (OVA5718) ZAR 360K-420KACLs | Ansible | Apache Kafka | Azure | Azure MonitorMid-level Full TimeJohannesburg, South Africa5d ago
-
Azure Bot | Azure Bot Service | Azure Cognitive | Azure Cognitive Services | Azure Machine LearningMid-level Full TimeMidrand, South Africa6d ago
-
Alerting | Amazon Aurora | Amazon Aurora PostgreSQL | Amazon Redshift | Anomaly DetectionContinuous improvement opportunities | Cross-functional collaboration | Fully remote work | Ownership over data reliability strategySenior-level Full TimeSouth Africa R6d ago
-
AI Engineering - Master Specialist ZAR 780K-920KApache Kafka | Apache Spark | C++ | CUDA | Cloud ComputingSenior-level Full TimeJohannesburg, ZA6d ago
-
Senior Data Engineer ZAR 1100K-1200KAPI | Apache Flink | Apache Spark | Apache Spark Streaming | Azure DataSenior-level Full TimeSouth Africa9d ago
-
Data Engineer ZAR 300K-420KAWS | Apache Airflow | Azure | Azure Data | Azure Data LakeFlexible team culture | Inclusive and diverse workplace | Support for employment equityMid-level Full TimeCenturion, GP, South Africa9d ago
-
Specialist - Data Engineer.MTN Software Solutions ZAR 464K-480KApache Hadoop | Apache Kafka | Apache Spark | Data Modeling | Data PipelinesMid-level Full TimeRoodepoort, Gauteng, South Africa10d ago
-
Cloud Data Engineers (OVA5720) ZAR 348K-420KAPI Integration | Apache Spark | Azure Data | Azure Data Factory | Azure EventHybrid work modelMid-level Full TimeJohannesburg, South Africa10d ago