Data Engineering Lead
ZAR 840K-1170K (estimate) Senior-level Contract
Tasks
- Architect data ingestion frameworks
- Automate data validation and monitoring
- Build data pipelines
- Collaborate with stakeholders
- Design enterprise data platforms
- Develop ETL and ELT frameworks
- Develop reusable data integration components
- Ensure data governance compliance
- Implement data quality controls
- Implement data warehousing and lakehouse storage
- Lead incident resolution
- Maintain technical documentation
- Manage data lineage and metadata
- Monitor production data platforms
- Optimize SQL queries and pipeline performance
- Optimize processing times and infrastructure costs
- Perform data transformation and cleansing
- Perform root cause analysis
- Provide technical recommendations and risk assessments
- Secure sensitive data
- Support production deployments and upgrades
- Translate business requirements to technical solutions
- Troubleshoot pipeline failures
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | AWS | AWS Glue | Airflow | Amazon Redshift | Apache Spark | Azure Data | Azure Data Factory | Azure DevOps | CI/CD | Cloud Computing | Data Engineering | Data Factory | Data Governance | Data Lineage | Data Management | Data Modeling | Data Privacy | Data Privacy Regulations | Data Quality | Data Warehousing | Data encryption | Data integration | Databricks | Delta Lake | Development lifecycle SDLC | ELT | ETL | Git | GitHub N/A | Information security | Infrastructure as Code | Lakehouse | Master Data | Master Data Management | Metadata Management | Microsoft Azure | N A | Oracle | Performance optimization | PostgreSQL | Privacy regulations | PySpark | Python | REST APIs | SQL | SQL Performance | SQL Performance Optimization | SQL Server | Snowflake | Software Development Lifecycle | Software Development Lifecycle (SDLC) | Software development | Terraform | “as-code”
Education
N/A
Related jobs
-
BigQuery | Cloud Composer | Cloud Functions | Cloud Storage | Cloud platformContract extension possibility | Hybrid work model | Opportunity for immediate startSenior-level Full TimeJohannesburg, South Africa R1d ago
-
Head of Marketing Data Science and Analytics ZAR 420K-516KAttribution Modeling | Audience Segmentation | Business Suite | Campaign Manager | Cloud DataHybrid work | In office days 4Executive-level Full TimeCape Town, South Africa1d ago
-
Sales Analytics Manager ZAR 384K-384KAlteryx | Data Governance | Data Preparation | Data Visualization | KNIMEMid-level Full TimeJohannesburg, ZA1d ago
-
Analytics Engineer (Data Platform) ZAR 464K-500KAgile | Algorithms | BI reporting | BigQuery | CI/CDBirthday leave | Confidential counselling | Financial guidance | Flexible total remuneration package | Free counsellingMid-level Full TimeCape Town2d ago
-
Analytics Engineer (Data Platform) ZAR 464K-500KAgile | Algorithms | BigQuery | CI/CD | Data CleansingBirthday leave | Confidential counselling | Financial guidance | Free parking | Hybrid working modelMid-level Full TimeCape Town2d ago
-
AWS | Agile | Azure | Canonical Data | Canonical Data ModelsSenior-level Full TimeJohannesburg, GP, South Africa2d ago
-
Senior Manager: Data Platforms ZAR 745K-900KAb Initio | Access Control | Business Continuity | Capacity Planning | Cloud DataSenior-level Full TimeJohannesburg, ZA2d ago
-
AWS | Apache Spark | Azure | Business Intelligence | CI/CDSenior-level Full TimeCape Town, South Africa3d ago
-
Senior-level Full TimePinelands, South Africa3d ago
-
Lead Data Engineer (Data Platform and Cloud) ZAR 663K-663KAzure | Azure Data | Azure Data Factory | Azure Databricks | Azure SynapseHybrid work modelSenior-level Full TimeMidrand, Gauteng, ZA6d ago
-
Lead AI Solutions Architect, Customer Success ZAR 660K-780KAI Solution | AI Solution Architecture | AWS | Azure | By DesignAugust Free Fridays | Education support | Flexible work arrangements | Leadership development programs | Mentorship programsSenior-level Full TimeJohannesburg (Flexible), South Africa7d ago
-
Senior-level Full TimePinelands, South Africa8d ago
-
Data Scientist ZAR 663K-780KAI Agents | AI guardrails | Causal Inference | Data Governance | Experiment designSenior-level Full TimeSandton - 1 Discovery Place, GP, …8d ago
-
Data Scientist ZAR 700K-900KArtificial Intelligence | Causal Inference | Data Governance | Experiment design | ExperimentationSenior-level Full TimeSandton - 1 Discovery Place, GP, …8d ago
-
AI machine learning | AWS | Access Controls | Access Management | AsanaRemote workSenior-level Full TimeArgentina - Buenos Aires9d ago
-
Senior-level Full TimeCape Town13d ago
-
Data/AI Architect (Databricks) ZAR 875K-1170KAI gateway | Access Controls | Apache Spark | Automated testing | CI/CDSenior-level ContractJohannesburg, South Africa16d ago
-
Project Manager Support & Data Analyst ZAR 90K-120KAsana | Business Intelligence | Client Onboarding | Data Reconciliation | Data ValidationFlexible work culture | Fully remote | Paid time off | Professional development and training | Supportive teamMid-level Full TimeJohannesburg, South Africa R21d ago
-
Data Architect ZAR 745K-900KAPI Integration | Analytics | Artificial Intelligence | Azure | Data ArchitectureSenior-level Full TimeJohannesburg, GP, ZA, 214622d ago
-
Senior-level Full TimeCape Town23d ago
-
Senior-level Full TimeStellenbosch, South Africa24d ago
-
Analytics Implementation Specialist ZAR 360K-480KAPI Integration | Adobe Analytics | Adobe Launch | CSS | DashboardsEntry-level Full TimeCape Town, Western Cape, South Africa27d ago
-
Senior-level ContractJohannesburg, South Africa28d ago
-
Senior-level Full TimeJohannesburg, South Africa28d ago
-
SAP Data and Analytics Product Management Specialist ZAR 498K-650KAgile | Analytics Cloud | Analytics architecture | BW/4HANA | Business Data CloudDiversity and inclusion | Hybrid workingSenior-level Full TimeJohannesburg, South Africa30d ago