Data Engineer
Tasks
- Architect data pipelines
- Build and orchestrate real time data access APIs
- Coordinate cross functional delivery and support
- Create infrastructure as code for data platforms
- Develop ETL and ELT workflows for batch and streaming
- Drive CI CD data pipeline integration
- Engineer scalable fault tolerant data pipelines
- Implement data quality validation reconciliation and auditing
- Implement data security masking and anonymization
- Maintain automated metadata and data lineage
- Perform data profiling and root cause resolution
- Provide level 2 and level 3 support for data incidents
- Provision hybrid cloud data platform
Perks/Benefits
- N/A
Skills/Tech-stack
ADLS Gen 2 | API Development | Anonymization | Apache Airflow | Apache Spark | Azure | CI/CD | Data Lakehouse | Data Lineage | Data Observability | Data Quality | Data Security | Data Vault | Data masking | Data profiling | Databricks | Delta Lake | Dimensional Modeling | Docker | ELT | ETL | Infrastructure as Code | Kubernetes | Message Queues | Microsoft Fabric | PySpark | Python | SQL | Streaming Data | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Roles
Related jobs
-
Data pipeline | Deep learning | GPU | JAX | Latency optimizationCareer growth | Flexible working hours | Fully remote | High autonomy | Professional developmentSenior-level Full TimeSouth Africa R6h ago
-
Senior-level Full TimePretoria, South Africa20h ago
-
Agile | Apache Kafka | Avro | CI/CD | Confluent PlatformCollaborate with global engineering teams | Continuous learning and professional development | Exposure to large-scale distributed systems | Flexible hybrid working model | Modern Agile development environmentSenior-level Full TimePretoria, South Africa20h ago
-
Machine Learning Engineer UAH 1377K-1549KData Preparation | GPU Computing | JAX | Machine Learning | Machine Learning EngineeringSenior-level Full TimeEurope21h ago
-
Intermediate Analytics Engineer ZAR 300K-420KData Architecture | Data Contracts | Data Governance | Data Lineage | Data ModelingMid-level Full TimeJohannesburg, ZA1d ago
-
Junior Analytics Engineer ZAR 216K-240KAPIs | Compliance | Data Architecture | Data Contracts | Data GovernanceEntry-level Full TimeJohannesburg, ZA1d ago
-
Senior-level Full TimeCape Town3d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudMid-level Full TimeJohannesburg, South Africa4d ago
-
Entry-level Full TimeStellenbosch, ZA4d ago
-
Data Analytics Technical Specialist ZAR 780K-900KAccess Management | Analysis Services | Azure DevOps | Azure SQL | Azure SQL DatabaseFlexible working | Hybrid workingSenior-level Full TimeCape Town, South Africa5d ago
-
AI Solutions Engineer ZAR 400K-464KAI Agents | API Integration | Azure AI | Chatbots | Data AnalysisMid-level Full TimeCape Town, South Africa Office5d ago
-
Senior Specialist Machine Learning Engineer ZAR 700K-840KAWS EC2 | AWS IAM | AWS S3 | Access Control | Amazon CloudWatchMedical aid | Retirement benefits | Risk benefitsSenior-level Full TimeMidrand, ZA5d ago
-
Entry-level Full TimeAbsa 270 Republic Road, South Africa5d ago
-
Senior Machine Learning Engineer ZAR 745K-860KAWS SageMaker | AWS Sagemaker Endpoints | Agile | Amazon Web Services | Big DataBirthday leave | Confidential counselling | Financial guidance | Free parking | Legal supportSenior-level Full TimeSouth Africa5d ago
-
Senior Machine Learning Engineer ZAR 745K-860KAWS | Agile | Big Data | BigQuery | Cloud platformBirthday leave | Confidential counselling | Financial guidance | Free parking | Free subscription takealotMORESenior-level Full TimeSouth Africa5d ago
-
Apache Spark | Azure Data | Azure Data Factory | Azure Databricks | Azure EventMid-level ContractSandton, South Africa5d ago
-
API | AWS | AWS Lambda | Amazon RDS | Amazon S3Autonomy | Collaborative team | Continuous learning | High ownership | Professional growthSenior-level Full TimeSouth Africa6d ago
-
Senior Data Scientist (Azure Data Engineering & MLOps) ZAR 900K-1200KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Contract Full TimeCape Town, Western Cape, South Africa6d ago
-
Senior Data Scientist (Azure Data Engineering & MLOps) ZAR 900K-1200KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Contract Full TimeCape Town, Western Cape, South Africa6d ago
-
Specialist Support Engineer: DataOps ZAR 420K-464KApache Spark | Batch Processing | Cloud Migration | Data Modeling | Data StreamingMid-level Full Time15 Alice Lane, South Africa6d ago
-
AI readiness | AWS | Access Management | Architecture | AsanaCareer growth opportunities | Customer facing exposure | Remote workSenior-level Full TimeHonduras - Tegucigalpa6d ago
-
AWS | Access Control | Access Management | Azure | Cloud hybridRemote workSenior-level Full TimeArgentina - Buenos Aires6d ago
-
Agile | Amazon Redshift | Amazon Web Services | Apache Airflow | Apache IcebergRemote workMid-level Full TimeSouth Africa - Cape Town6d ago
-
Technology/Domain Specialist II - Kafka (OVA5718) ZAR 360K-420KACLs | Ansible | Apache Kafka | Azure | Azure MonitorMid-level Full TimeJohannesburg, South Africa6d ago
-
Azure Bot | Azure Bot Service | Azure Cognitive | Azure Cognitive Services | Azure Machine LearningMid-level Full TimeMidrand, South Africa8d ago