Lead Data Engineer
Absa Towers West, South Africa
ZAR 600K-730K (estimate) Senior-level Full Time
Tasks
- Align solutions to group architecture data standards
- Align solutions to group infrastructure standards
- Analyze technical landscape and bank architecture
- Apply design patterns for data solutions
- Assemble large datasets and manage data pipelines
- Automate tasks with infrastructure scripting
- Build analytics tools
- Coach and mentor engineers
- Collaborate with data science teams on customer data requirements
- Conduct peer reviews and testing
- Create and support CI/CD pipelines
- Create data tools for analytics and data science teams
- Create infrastructure for high volume data delivery
- Debug and optimize data engineering code
- Design data architecture
- Develop data pipeline architecture
- Ensure data security separation and quality
- Ensure data solution self service repeatability testability scalability and resilience
- Implement data quality assurance routines
- Lead data retrieval storage processing and distribution
- Lead end to end data solution design and implementation
- Maintain application documentation and governance records
- Manage technical risks and mitigations
- Monitor performance of data products and services
- Optimize automate and improve data processes
- Participate in incident management and disaster recovery
- Provide subject matter expertise for recruiting product engineers
- Solve data volume and scaling challenges
- Support ETL from multiple data sources
Perks/Benefits
- N/A
Skills/Tech-stack
Ansible | CI/CD | Chef | Cloud Computing | Containerization | Data Architecture | Data Engineering | Data Modeling | Data Pipelines | Data Quality | Data Security | Disaster Recovery | Domain-Driven Design | ETL | Incident Management | Infrastructure automation | Jenkins | NoSQL | Relational databases | Resilience | Scalability | Scripting | Streaming Platforms | Test-Driven | Test-Driven Development
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
AMDP | AWS | AWS Glue | AWS Lambda | AWS Step FunctionsAgile working model methodology | Dynamic global team collaboration | Flexible working hours | High work life balance | Modern officesSenior-level Full TimePretoria, South Africa R12h ago
-
APIs | Access Control | Artificial Intelligence | Automation | ChatGPTSenior-level Full TimeSandton, South Africa3d ago
-
Software Engineer (Machine Learning) ZAR 576K-650KAPIs | AWS | Alerting | Auto Scaling | Cloud ComputingSenior-level Full TimeCape Town, Western Cape, ZA3d ago
-
Automation | Azure Data | Azure Data Explorer | CSV | Data ExplorerCross-functional collaboration | Exposure to large scale security telemetry | Professional growth | Remote workMid-level Full TimeSouth Africa R3d ago
-
AI-assisted coding | Assisted coding | Bash | Build Automation | C#Driver LicenseSenior-level Full TimePretoria, South Africa4d ago
-
AI Agents | AI Observability | APIs | AWS Bedrock | Access ControlAsynchronous collaborative model | Distributed international working environment | Flexible parental leave | Fully Remote Working Environment | Mental health and wellbeing supportSenior-level Full TimeSouth Africa R4d ago
-
Mid-level Full Time15 Alice Lane, South Africa5d ago
-
Data Engineer ZAR 600K-720KAPI Development | AWS Data | AWS Data Services | Amazon Web Services | Automated testingSenior-level Full TimeStellenbosch, ZA5d ago
-
Data Engineer ZAR 700K-840KAmazon Web Services | Batch Processing | CI/CD | Data Governance | Data LakeSenior-level Full TimeStellenbosch, ZA5d ago
-
Senior-level Full TimeJohannesburg, South Africa5d ago
-
Automated Workflows | CI_CD | Cloud Computing | Containerization | Data LineageProfessional growthSenior-level Full TimeSouth Africa5d ago
-
.NET MAUI | AI Agents | Agent systems | Amazon Web Services | Application ProgrammingSenior-level Full TimeRivonia, South Africa5d ago
-
.NET MAUI | APIs | Agent systems | Amazon Web Services | AzureFlexible innovation time | Hands-on technical leadership | Mentorship opportunitiesSenior-level ContractJohannesburg, South Africa5d ago
-
Mid-level Full TimeSouth Africa5d ago
-
Software Engineer, SAP Analytics Cloud (SAC) ZAR 360K-540KAPI Engineering | Agile | Analytics Cloud | BPC Planning | BudgetingMid-level Full TimeJohannesburg, GP, South Africa5d ago
-
Senior Analytics Engineer ZAR 660K-720KAPI Development | Analytics engineering | Azure | Cloud Computing | Data ArchitectureSenior-level Full TimeJohannesburg, ZA6d ago
-
API | Authentication | Automated testing | Data Pipelines | GitMid-level Full TimeSouth Africa - Remote R6d ago
-
ARM | Agile | Apache Spark | Azure Data | Azure Data FactorySenior-level ContractJohannesburg, South Africa6d ago
-
Senior Cloud Data Engineer ZAR 745K-900KAirflow | Apache Beam | Apache Spark | BigQuery | Cloud platformContinuous learning opportunities | Medical aid | Paid volunteering leave | Retirement funding options | Tech events and exposSenior-level Full TimeJohannesburg, South Africa6d ago
-
Senior Ai Engineer ZAR 600K-720KAWS | AWS SageMaker | Automated testing | Azure | Azure MLCareer growth opportunities | Collaborative work environment | Conference support | Courses support | Medical Aid OptionalSenior-level Full TimeJohannesburg, South Africa6d ago
-
AI Agents | Access Control | Artificial Intelligence | Cybersecurity | Dashboard DevelopmentRegular national travel | Relocation supportSenior-level Full TimeDurban, South Africa6d ago
-
Senior-level Full TimeHQ Sandton, South Africa7d ago
-
Staff Embedded Software Engineer ZAR 504K-576KC plus plus | C# | Communication Protocols | Containerization | CryptographyAnnual leave | Career development opportunities | Inclusive culture | Work-life balanceSenior-level Full TimeDurban, ZA, 3630 R7d ago
-
Mid-level Full TimeJohannesburg, ZA R7d ago
-
Senior Specialist Analytics & Enablement ZAR 540K-650KAnomaly Detection | Azure Data | Azure Data Factory | Azure Synapse | DAXSenior-level Full TimeSandton, South Africa8d ago