Data Engineer
Tasks
- Architect data pipelines
- Automate data validation reconciliation and auditing
- Build consumption APIs and services
- Coordinate incident resolution
- Define master data management standards
- Develop batch and streaming ingestion
- Embed data quality as code
- Enforce data security policies
- Implement Data Ops with automation testing orchestration
- Implement data masking and anonymisation
- Load data into data warehouse and data lake
- Maintain data lineage and metadata
- Maintain scalable fault tolerant data pipelines
- Manage hybrid cloud data platform
- Perform data profiling and root cause analysis
- Provide level-2/level-3 support
- Set up data infrastructure observability
- Support agile sprint planning and delivery
- Transform data into structured formats
- Use infrastructure as code
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Automation | Azure Data | Azure Data Factory | Azure Event | Azure Event Hubs | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Governance | Data Lake | Data Lineage | Data Management | Data Quality | Data Security | Data Warehousing | Data anonymization | Data masking | Data profiling | Databricks | Event Hubs | Infrastructure as Code | Kafka | Master Data | Master Data Management | Message Queues | Metadata Management | Microsoft Fabric | Observability | Orchestration | PySpark | Python | Scala | Synapse Analytics | Testing | “as-code”
Education
N/A
Roles
Related jobs
-
Big Data Support Engineer ZAR 360K-480KApache Kafka | Apache Spark | Data Modeling | Database Design | HDFSMid-level Full Time15 Alice Lane, South Africa1d ago
-
Specialist Support Engineer: DataOps ZAR 360K-480KApache Spark | Data Modeling | Data Streaming | Database Design | HDFSMid-level Full Time15 Alice Lane, South Africa1d ago
-
Intermediate Analytics Engineer ZAR 384K-480KApache Airflow | Cloud Computing | DBT | Data Governance | Data LineageMid-level Full TimeJohannesburg, ZA1d ago
-
Senior Analytics Engineer ZAR 660K-840KAPI Development | Cloud Computing | Data Architecture | Data Governance | Data LineageMentorship | Performance feedback | Professional development | Training and upskillingSenior-level Full TimeJohannesburg, ZA1d ago
-
A/B | A/B Testing | API Design | AWS ECS | AWS EKSSenior-level Full TimeCape Town1d ago
-
SQL DBA (Senior) ZAR 840K-960KAPIs | Azure Data | Azure Data Factory | Azure DevOps | Azure SQLCareer development opportunities | Employee wellness benefits | Hybrid workSenior-level Full TimeCape Town, South Africa1d ago
-
Apache Spark | Automated testing | CI/CD | Data Lineage | DatabricksSenior-level Full TimeCape Town, Western Cape, South Africa1d ago
-
Associate Staff Engineer (Data Engineer) ZAR 420K-480KAPI Development | AWS | Airflow | Apache Kafka | Apache SparkMid-level Full TimeJohannesburg, SA (Hybrid), South Africa R2d ago
-
Data Design and Engineering ZAR 840K-1080KBusiness Intelligence | Data Architecture | Data Cleansing | Data Governance | Data MiningSenior-level Full TimeSouth Africa2d ago
-
Data Engineer ZAR 840K-1080KBusiness Intelligence | Data Architecture | Data Cleansing | Data Governance | Data IntegritySenior-level Full TimeSouth Africa2d ago
-
API Integration | Automation | Claude | Data Modeling | IndexingCollaborative engineering culture | Flexible/remote location | Fully remote work | High-ownership environmentSenior-level Full TimeSouth Africa R2d ago
-
Machine Learning Architect ZAR 840K-1200KAI Agents | AWS | Apache Airflow | Apache Spark | Automated retrainingSenior-level Full TimeSouth Africa - Remote R2d ago
-
Software Data Engineer ZAR 360K-420KAPIs | Apache Beam | Apache Flink | Big Data | CI/CDBirthday leave | Discounted subscription | Financial guidance | Free counseling | Free parkingMid-level Full TimeCape Town2d ago
-
Software Data Engineer ZAR 360K-420KAPIs | Apache Beam | Apache Flink | Apache Storm | Continuous DeliveryBirthday leave | Confidential counseling | Financial guidance | Free parking | Free subscriptionMid-level Full TimeCape Town2d ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScriptCareer growth opportunities | Flexible work environment | Remote workMid-level Full TimeSouth Africa R3d ago
-
Accelerate | Deep learning | Diffusers | Generative AI | Hugging FaceAutonomy and ownership | Career growth opportunities | Flexible globally distributed work environment | Fully remote work | Global collaborationMid-level Full TimeSouth Africa R4d ago
-
Data Curation | Deep learning | Distributed Training | GPU Training | JAXAccess to high performance GPU clusters | Collaborative global culture | Flexible work schedule | Fully remote | Professional growth opportunitiesSenior-level Full TimeSouth Africa R4d ago
-
AWS | Azure | Embeddings | GCP | Hugging FaceFlexible work environment | Fully remote | Growth opportunities | Remote distributed team experienceSenior-level Full TimeSouth Africa - Remote R4d ago
-
Data Engineer ZAR 420K-500KApache Airflow | Apache Spark | Data Governance | Data Modeling | DockerCredit and Criminal Record Check | Curro School Fees Benefit | Employee assistance programme | Financial Services Assistance | Free Sleep CoachingMid-level Full TimeCape Town, South Africa4d ago
-
AWS Lambda | Amazon CloudWatch | Amazon EKS | Amazon Redshift | Amazon SageMakerOn-call rotationSenior-level Full TimeCape Town, Western Cape, South Africa6d ago
-
API Security | Access Control | Airflow | Amazon Redshift | AuthenticationFlexible schedule | Remote workSenior-level Full TimeSouth Africa - Remote R7d ago
-
Application Support Engineer ZAR 300K-420KAgile | Ataccama | Cause analysis | Change Management | CollibraMid-level Full Time15 Alice Lane, South Africa7d ago
-
Principal Specialist VodaPay Insights and Analytics ZAR 540K-720KA/B | A/B Testing | B testing | CVM Analytics | Customer SegmentationCell phone benefits | Data benefits | Fibre connection discounts | Medical aid | Retirement benefitsSenior-level Full TimeMidrand, ZA7d ago
-
Machine Learning Engineer ZAR 420K-600KAirflow | Amazon SageMaker | CI/CD | Docker | Feature EngineeringAgile environment | Collaborative culture | Feedback and empathy | Freedom and responsibility | Self-organizationMid-level Full TimeSouth Africa - Remote R8d ago
-
Data Engineer ZAR 420K-480KAPI | Amazon Redshift | Amazon Web Services | Amazon Web Services (AWS) | Apache SparkMid-level Full TimePinelands, South Africa8d ago