Data Engineering Lead
Johannesburg, South Africa
ZAR 840K-1170K (estimate) Senior-level Contract
Tasks
- Architect data ingestion frameworks
- Automate data validation and monitoring
- Build data pipelines
- Collaborate with stakeholders
- Design enterprise data platforms
- Develop ETL and ELT frameworks
- Develop reusable data integration components
- Ensure data governance compliance
- Implement data quality controls
- Implement data warehousing and lakehouse storage
- Lead incident resolution
- Maintain technical documentation
- Manage data lineage and metadata
- Monitor production data platforms
- Optimize SQL queries and pipeline performance
- Optimize processing times and infrastructure costs
- Perform data transformation and cleansing
- Perform root cause analysis
- Provide technical recommendations and risk assessments
- Secure sensitive data
- Support production deployments and upgrades
- Translate business requirements to technical solutions
- Troubleshoot pipeline failures
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | AWS | AWS Glue | Airflow | Amazon Redshift | Apache Spark | Azure Data | Azure Data Factory | Azure DevOps | CI/CD | Cloud Computing | Data Engineering | Data Factory | Data Governance | Data Lineage | Data Management | Data Modeling | Data Privacy | Data Privacy Regulations | Data Quality | Data Warehousing | Data encryption | Data integration | Databricks | Delta Lake | Development lifecycle SDLC | ELT | ETL | Git | GitHub N/A | Information security | Infrastructure as Code | Lakehouse | Master Data | Master Data Management | Metadata Management | Microsoft Azure | N A | Oracle | Performance optimization | PostgreSQL | Privacy regulations | PySpark | Python | REST APIs | SQL | SQL Performance | SQL Performance Optimization | SQL Server | Snowflake | Software Development Lifecycle | Software Development Lifecycle (SDLC) | Software development | Terraform | “as-code”
Education
N/A
Related jobs
-
Manager Data & Analytics Fund Solutions ZAR 360K-420KAutomation | Business Intelligence | Dashboarding | Data Compliance | Data ModelingMid-level Full TimeJohannesburg, GP, South Africa1d ago
-
(1833) Manager : Data Risk and Analytics -RSD ZAR 360K-426KAnalytics | Basel Core Principles for Banking Supervision | Business Intelligence | DAMA | Data AuditingMid-level Full TimePretoria, South Africa1d ago
-
Head: Decision Science ZAR 420K-600KArtificial Intelligence | Cloud Computing | Credit Risk | Credit Scoring | Credit risk analyticsSenior-level Full TimeSandton, ZA1d ago
-
AI Agents | API | CI/CD | Cloud Computing | Distributed SystemsCareer advancement and professional development | Collaborative globally distributed team | Flexible working arrangements | Fully remote | Inclusive workplace cultureSenior-level Full TimeSouth Africa R2d ago
-
Senior-level Full TimeAbsa Towers West, South Africa2d ago
-
Mid-level Full TimeSandton - 1 Discovery Place, GP, …2d ago
-
AI Engineering Lead EUR 54K-72KAPI Integration | Analytics | Artificial Intelligence | Automation engineering | Data platformsCareer growth | Health insurance | Housing | Mentorship | Regular feedbackSenior-level Full TimeEurope5d ago
-
AI Foundry | AWS Bedrock | AWS GovCloud | AWS SageMaker | Agentic AI401k matching | Career development resources | Dental insurance options | Flexible work arrangements | Health insurance optionsSenior-level Full TimeSouth Africa5d ago
-
Senior Data Architect ZAR 745K-900KData Architecture | Data Governance | Data Lifecycle Management | Data Modeling | Data QualitySenior-level Full TimePinelands, South Africa5d ago
-
AI Evaluation | Agentic Workflows | Automation | CI/CD | Cloud ArchitectureCollaborative environment | Contract engagement | Cutting-edge technology exposure | High-impact projects | Remote workSenior-level Full TimeSouth Africa R6d ago
-
AI Solutions Lead - Marketing & Content ZAR 600K-720KAI platforms | API Development | AWS | Data Privacy | Enterprise AIRemote work opportunity | Work-life balanceSenior-level Full TimeJohannesburg, Gauteng R7d ago
-
AI architecture | AWS | Cloud platform | Data Governance | Data PrivacyRemote work | Work-life balanceSenior-level Full TimeJohannesburg, Gauteng R7d ago
-
Senior Analytics Engineer ZAR 660K-720KAPI Development | Analytics engineering | Azure | Cloud Computing | Data ArchitectureSenior-level Full TimeJohannesburg, ZA7d ago
-
Executive Head: Pricing & Analytics ZAR 660K-840KARPU | Analytics | Big Data | Business planning | ChurnExecutive-level Full TimeMidrand, ZA7d ago
-
AI Agents | Access Control | Artificial Intelligence | Cybersecurity | Dashboard DevelopmentRegular national travel | Relocation supportSenior-level Full TimeDurban, South Africa8d ago
-
Software Engineer, Analytics ZAR 505K-505KAPI Testing | APIs | Code review | SDLC | Software developmentGlobal career development | Hybrid work | Referral bonus | Social events | Time offMid-level Full TimeCape Town, SA R9d ago
-
Databricks Project Manager ZAR 384K-420KAgile | Change Management | Databricks | Microsoft Project | Project GovernanceMid-level Full TimeCape Town, South Africa9d ago
-
AI analytics | Attribution | Business Intelligence | CRM | Cohort AnalysisHybrid workMid-level Full TimeCape Town, Western Cape, South Africa R9d ago
-
Senior Specialist Analytics & Enablement ZAR 540K-650KAnomaly Detection | Azure Data | Azure Data Factory | Azure Synapse | DAXSenior-level Full TimeSandton, South Africa9d ago
-
Senior Analytics Engineer ZAR 600K-600KAI analytics | DBT | Dagster | Data Governance | Data ModelingFlexible work schedule | Hybrid work | Life insurance | Matched pension scheme | Paid parental leaveSenior-level Full TimeCape Town, South Africa R12d ago
-
APIs | Cloud Computing | Data Architecture | Data Governance | Data LineageSenior-level ContractSandton, South Africa13d ago
-
Analytics Engineer (OVA5693) ZAR 384K-420KAPI Development | Cloud Platforms | Cost Optimization | Data Architecture | Data ContractsHybrid work | Independent contractMid-level Full TimeJohannesburg, South Africa13d ago
-
Analytics Engineer II ZAR 576K-720KApache Airflow | CI/CD | Cloud Data | Cloud Data Platforms | Data GovernanceSenior-level Full TimeStellenbosch, ZA14d ago
-
AWS | Amazon Redshift | Apache Airflow | Apache Spark | BigQueryFlexible hours US time zones | Growth opportunities | Remote workSenior-level Full TimeSouth Africa - Cape Town R19d ago
-
Senior Data Architect ZAR 745K-920KAnalytics Integration | Data Compliance | Data Governance | Data Migration | Data ModelingSenior-level ContractCape Town, South Africa20d ago