Data Engineering Lead
ZAR 840K-1170K (estimate) Senior-level Contract
Tasks
- Architect data ingestion frameworks
- Automate data validation and monitoring
- Build data pipelines
- Collaborate with stakeholders
- Design enterprise data platforms
- Develop ETL and ELT frameworks
- Develop reusable data integration components
- Ensure data governance compliance
- Implement data quality controls
- Implement data warehousing and lakehouse storage
- Lead incident resolution
- Maintain technical documentation
- Manage data lineage and metadata
- Monitor production data platforms
- Optimize SQL queries and pipeline performance
- Optimize processing times and infrastructure costs
- Perform data transformation and cleansing
- Perform root cause analysis
- Provide technical recommendations and risk assessments
- Secure sensitive data
- Support production deployments and upgrades
- Translate business requirements to technical solutions
- Troubleshoot pipeline failures
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | AWS | AWS Glue | Airflow | Amazon Redshift | Apache Spark | Azure Data | Azure Data Factory | Azure DevOps | CI/CD | Cloud Computing | Data Engineering | Data Factory | Data Governance | Data Lineage | Data Management | Data Modeling | Data Privacy | Data Privacy Regulations | Data Quality | Data Warehousing | Data encryption | Data integration | Databricks | Delta Lake | Development lifecycle SDLC | ELT | ETL | Git | GitHub N/A | Information security | Infrastructure as Code | Lakehouse | Master Data | Master Data Management | Metadata Management | Microsoft Azure | N A | Oracle | Performance optimization | PostgreSQL | Privacy regulations | PySpark | Python | REST APIs | SQL | SQL Performance | SQL Performance Optimization | SQL Server | Snowflake | Software Development Lifecycle | Software Development Lifecycle (SDLC) | Software development | Terraform | “as-code”
Education
N/A
Related jobs
-
Mid-level Full TimeJohannesburg, South Africa1d ago
-
Senior-level Full TimeStellenbosch, South Africa1d ago
-
AI | Adjudication | Annotation Guidelines | Audio evaluation | CalibrationFully remote | International team collaboration | Professional growth opportunitiesSenior-level Full TimeSouth Africa R1d ago
-
API Design | AWS | AWS CDK | Agent systems | AzureDiversity & inclusion culture | Flexible remote work | Professional growth | Wellbeing supportSenior-level Full TimeSouth Africa1d ago
-
Head, Data & Artificial Intelligence, Business Transformation (Corporate & Investment Banking, Technology) ZAR 750K-900KArtificial Intelligence | Data Architecture | Data Governance | Data Mining | Data QualityExecutive-level Full TimeJohannesburg, GP, South Africa4d ago
-
(1733) Advance Analytics Specialist - ExMan ZAR 603K-650KArtificial Intelligence | Data Governance | Data Visualization | Forecasting | Machine LearningSenior-level Full TimePretoria, South Africa4d ago
-
Analytics Implementation Specialist ZAR 360K-480KAPI Integration | Adobe Analytics | Adobe Launch | CSS | DashboardsEntry-level Full TimeCape Town, Western Cape, South Africa4d ago
-
Senior Analytics Consultant ZAR 480K-650KDashboard Design | Data Analysis | Data Visualization | Microsoft Excel | Microsoft PowerPointMedical aid | PensionSenior-level Full TimeCape Town, South Africa, ZA, Cape …5d ago
-
AWS S3 | Azure | Cloud Integration | Data Governance | Data LineageSenior-level Full TimeCape Town, South Africa5d ago
-
Senior-level ContractJohannesburg, South Africa5d ago
-
API Design | Change Control | Cloud infrastructure | Compliance Management | CybersecuritySenior-level Full TimeJohannesburg, South Africa5d ago
-
Senior-level Full TimeJohannesburg, South Africa5d ago
-
Senior Specialist Analytics Products ZAR 660K-720KDashboarding | Data Analysis | Data Governance | Data Modeling | Data PrivacySenior-level Full TimeAbsa Towers West, South Africa6d ago
-
Director, Data Asset Management ZAR 420K-456KAnalytics | Automation | Compliance | Data Governance | Data IngestionHybrid work environment | Work-life flexibility | Workplace Collaboration OpportunitiesExecutive-level Full TimeJohannesburg, South Africa6d ago
-
Data Engineering Head ZAR 540K-660KData APIs | Data Architecture | Data Governance | Data Pipelines | Data QualityChallenging work | Opportunities to innovate | Opportunities to network and collaborateExecutive-level Full Time2 First Place, Cnr Simmonds & …7d ago
-
Lead AI Engineer - Financial Inclusion GBP 60K-100KIdentity automation | Langchain | MCP | Python | ScriptingCareer growth opportunities | Coaching partnerships | Family-friendly policies | Flexible working arrangements | Fully remoteSenior-level Full TimeNairobi R7d ago
-
SAP Data and Analytics Product Management Specialist ZAR 498K-650KAgile | Analytics Cloud | Analytics architecture | BW/4HANA | Business Data CloudDiversity and inclusion | Hybrid workingSenior-level Full TimeJohannesburg, South Africa8d ago
-
Senior-level Full TimeAbsa Towers North, South Africa8d ago
-
AWS | Airflow | Data Architecture | Data Governance | Data PipelinesSenior-level Full TimeSandton, South Africa9d ago
-
Analytics Engineer (Data Platform) ZAR 400K-420KAgile | Algorithms | BigQuery | CI/CD | Cube designBirthday leave | Confidential counseling | Financial guidance | Flexible equipment setup | Free parkingMid-level Full TimeCape Town11d ago
-
Analytics Engineer (Data Platform) ZAR 400K-420KAgile | Algorithms | BigQuery | CI/CD | Data ModelingBirthday leave | Confidential counselling | Financial guidance | Free parking | Hybrid work modelMid-level Full TimeCape Town11d ago
-
Director, Network Services, Analytics, EEMEA ZAR 480K-660KCompetitive benchmarking | Data Modeling | Elasticity analysis | Excel | Financial ModelingExecutive-level Full TimeJohannesburg, South Africa13d ago
-
Information Security Manager (D) ZAR 480K-480KAI RMF | Adversarial ML | Azure | CI/CD | Data Loss PreventionMid-level Full TimeJohannesburg, South Africa13d ago
-
Data Architect (D) ZAR 840K-960KAI Agents | API Integration | Automated retraining | Big Data | CI/CDSenior-level Full TimeJohannesburg, South Africa13d ago
-
Data Governance and Data Management (D) ZAR 560K-720KBy Design | Cloud Data | Cloud Data Pipelines | DAMA-DMBOK | DCAMSenior-level Full TimeJohannesburg, South Africa13d ago