Data Engineer
Tasks
- Architect data pipelines
- Automate data validation reconciliation and auditing
- Build real time data access APIs
- Coordinate incident resolution for data issues
- Define master data management standards
- Drive continuous improvement of data engineering practices
- Embed data quality as code
- Enforce data security and privacy compliance
- Engineer and maintain data pipelines
- Establish data observability
- Implement Data Ops automation testing orchestration
- Implement batch and streaming data ingestion
- Implement data masking and anonymization
- Implement infrastructure-as-code
- Load data into data warehouse and data lake
- Maintain data lineage in metadata hub
- Manage Azure data platform
- Mentor junior engineers
- Transform and deliver structured semi structured and unstructured data
Perks/Benefits
- N/A
Skills/Tech-stack
Ab Initio | Anonymization | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Event | Azure Event Hubs | Azure Synapse | CI/CD | Data Factory | Data Governance | Data Lake | Data Lineage | Data Management | Data Quality | Data Security | Data Validation | Data Warehouse | Data masking | Databricks | Event Hubs | Infrastructure as Code | Kafka | Master Data | Master Data Management | Message Queues | Metadata Management | Microsoft Fabric | Observability | PySpark | Python | SQL | Scala | “as-code”
Education
N/A
Roles
Related jobs
-
Senior Data Engineer - Agents Systems GBP 97K-136KApache Flink | Data Contracts | Data Processing | Data Quality | Feature StoreSenior-level Full TimeUnited Kingdom1d ago
-
Senior Data Engineer - AI Infrastructure GBP 101K-136KApache Flink | Data Contracts | Data Pipelines | Data Quality | Feature StoreSenior-level Full TimeUnited Kingdom1d ago
-
Senior Data Platform Consultant ZAR 600K-730KAWS | Azure | CI/CD | Classification | Data EngineeringHybrid workSenior-level Full TimeCape Town, South Africa R1d ago
-
Mid-level Full Time4 Merchant Place, 1 Fredman Drive, …1d ago
-
Senior-level Full TimeCape Town, South Africa2d ago
-
Agile | Artificial Intelligence | Data Architecture | Data Engineering | Data GovernanceHybrid work environment | Mentorship | Training and development | Work from home daysEntry-level Full TimeMidrand, GP, South Africa R2d ago
-
Bash | Cloud platform | Cost Optimization | Data Pipelines | Distributed SystemsAccess to cutting-edge technology | Autonomy | Collaborative culture | Distributed first | Fully remoteSenior-level Full TimeSouth Africa R2d ago
-
AI Development Specialist (Azure) OVA5342 ZAR 840K-1080K.NET | AI Search | AI Studio | AKS | API ManagementSenior-level Full TimeJohannesburg, South Africa2d ago
-
AI Data | AI data pipelines | AWS | Access Management | AzureCompetitive pay | Growth opportunities | Remote work flexibilitySenior-level Full TimeHonduras - Tegucigalpa4d ago
-
AI readiness | AI workflows | AWS | Access Control | Access ManagementRemote work | US time zone collaboration | Video interview requirementSenior-level Full TimeArgentina - Buenos Aires4d ago
-
Senior Applied AI Engineer – Prompting & Evaluation ZAR 450K-650KAgentic Workflows | Azure OpenAI | Context design | Embeddings | GitHub CopilotCoaching and mentoring culture | Community of Practice | Hybrid work policy | Travel between UK and South Africa officesSenior-level Full TimeJohannesburg, SA, ZA | Stellenbosch, SA, …5d ago
-
Combinatorics | Graph theory | MATLAB | NumPy | Number theoryProject-based assignmentsSenior-level FreelanceSouth Africa - Remote R5d ago
-
Data Development Manager ZAR 464K-480KAutomation | Data Warehouse | Data pipeline | Database Design | ETLMid-level Full TimeJohannesburg, ZA5d ago
-
Mid-level ContractJohannesburg, South Africa5d ago
-
Machine Learning Engineer ZAR 420K-480KAutomation | Azure Machine Learning | Data analytics | Databricks | Machine LearningMid-level Full TimeSandton - 1 Discovery Place, GP, …6d ago
-
Senior Machine Learning Engineer ZAR 745K-900KAI Platform | AWS SageMaker | Apache Airflow | Azure ML | Computer VisionSenior-level Full TimeJohannesburg, South Africa6d ago
-
Head: Data Engineering ZAR 660K-750KABAC | Apache Airflow | Apache Hudi | Apache Iceberg | Apache SparkExecutive-level Full TimeAbsa 270 Republic Road, South Africa6d ago
-
Azure | CI/CD | Data Governance | Data Lineage | Data PipelinesSenior-level ContractSandton, South Africa7d ago
-
Data Engineer ZAR 348K-420KAWS | AWS CDK | Azure | Azure Data | Azure Data FactoryGrowth opportunities | Hybrid work environment | Learning opportunities | Medical aid | Provident fundMid-level Full TimeJohannesburg, ZA7d ago
-
Senior-level Full TimeJohannesburg, South Africa7d ago
-
Senior-level Full TimeCape Town, South Africa7d ago
-
Senior-level Full TimeDurban, South Africa7d ago
-
AWS | AWS S3 | Agile | Apache Airflow | Apache DataprocCareer growth | Collaborative Agile environment | Flexible working arrangements | International large scale projects | Ongoing training and mentoringSenior-level Full TimeSouth Africa8d ago
-
Computer Vision | Deep learning | Diffusers | Diffusion Models | IP AdaptersAnnual vacation | Corporate discounts for English language learning programs | Fully remote | Health allowance | Professional development reimbursementSenior-level Full TimeSouth Africa R8d ago
-
Agentic pipelines | DPO | Deep learning | Direct Preference Optimization | Distributed TrainingContinuous learning opportunities | English language learning discounts | Fully remote | Health support budget | Home-office allowanceSenior-level Full TimeSouth Africa R8d ago