Data Engineer Lead

Johannesburg

Apply now Apply later

Let's Write Africa's Story Together!

Old Mutual is a firm believer in the African opportunity and our diverse talent reflects this.

Job Description

Collaborate in cross-functional teams to architect, design, build and maintain scalable data capabilities via data platform and data products. This entails taking a leading role in the end-to-end data pipeline, from data acquisition and storage to data transformation and analysis. It requires specialist knowledge of modelling, prompt engineering and data specialisation techniques.

The Data Engineer Lead is responsible for understanding the data requirements of the product being designed and translate these into efficient and reliable data engineering solutions. They work closely with data architecture teams to ensure data governance and compliance standards are met, while also brainstorming and implementing innovative solutions to optimize data processing and storage.

The Data Engineer Lead also plays a crucial role in identifying and implementing best practices for data engineering, ensuring data quality and integrity, and troubleshooting any issues or bottlenecks that arise. They are entrusted with the responsibility of building and maintaining a solid foundation for data-driven decision-making, enabling the organization to extract valuable insights from vast amounts of complex data while embedding software quality and engineering practices.

  • Develop and implement portfolio Data modelling, assurance and utilisation strategies and frameworks that align with enterprise approved governance, data and technology strategy and the Data COE. Lead the implementation of these strategies within the portfolio.

  • Serve as though leader and guide in the data domain by sharing knowledge identifying problems, patterns, trends, and support the development of relevant BI and MI solutions.

  • Design and implement scalable and robust processes for ingesting and transforming complex datasets.

  • Contribute to the development of architectural frameworks, apply architecture principles, and drive the development of data architecture models within the organisation.

  • Design and develop data models using dimensional modelling and data vault techniques and ensure stated business requirements are met by these models.

  • Architect, train, validate and test advanced analytics / machine learning models, using enterprise-grade software engineering practices.

  • Design, develops and maintain automated scalable data pipelines that improve estate performance, stability and auditability. These include data pipelines for ETL processing. Monitor and troubleshoot data pipeline issues.

Stakeholder Communication

  • Excellent communication and presentation skills for effectively conveying data status, data-driven insights, and recommendations to stakeholders at all levels.

Ethical and Compliance Awareness

  • Understanding of ethical considerations in data engineering, including data privacy, security, and confidentiality.

Continuous Learning and Adaptability

  • Commitment to staying updated with emerging data engineering trends, technologies, and industry developments.

Minimum Qualifications/Experience

  • Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

  • 10+ years of experience in data engineering with a focus on leadership and project management.

  • Data warehouse technical experience – definition /implementation/ integration.

  • Strong programming skills in Python and DBA skills (SQL/PSQL/DynamoDB or other).

  • Experience with data pipeline and ETL tools and reporting/analytics tools including, but not limited to, any of the following combinations (1) SSIS and SSRS, (2) ETL Frameworks, (3) Data conformance, (4) Caching, (5) Spark (6) AWS data builds.

  • Experience with data modelling, data governance, and data quality.

  • Strong problem-solving skills and ability to work in a fast-paced environment.

  • Strong communication skills and ability to work in a team.

  • Expertise in Machine Learning (ML) and deep learning frameworks.

  • Explaining the thinking behind simple ML algorithms.

  • Proficiency in all aspects of model architecture, data pipeline interaction, and metrics interpretation.

Additional

  • Experience with Big Data technologies such as Hadoop and Spark.

  • Experience with containerization technologies such as Docker and Kubernetes.

Competencies Required ​

  • Multi-functional team Collaboration (Relating)

  • Customer First

  • Execution

  • Innovation (Perspective)

  • Leading with Influence

  • Learning

  • Strategic thinking

  • Personal Mastery

Skills

Education

Closing Date

23 October 2024

The appointment will be made from the designated group in line with the Employment Equity Plan of Old Mutual South Africa and the specific business unit in question.

Old Mutual Limited is pro-vaccination and encourages its workforce to be fully vaccinated against Covid-19.

All prospective employees are required to disclose their vaccination status as part of the recruitment process.

Please refer to the Old Mutual’s Covid-19 vaccination policy for further detail. Kindly note that Old Mutual reserves the right to reinstate the requirement to vaccinate at any point if it is of the view that it is imperative to do so.

The Old Mutual Story!

Apply now Apply later

* Salary range is an estimate based on our AI, ML, Data Science Salary Index πŸ’°

Job stats:  0  0  0

Tags: Architecture AWS Big Data Computer Science Data governance Data pipelines Data quality Data warehouse Deep Learning Docker DynamoDB Engineering ETL Hadoop Kubernetes Machine Learning ML models Pipelines Privacy Prompt engineering Python Security Spark SQL SSIS

Perks/benefits: Career development

Region: Africa
Country: South Africa

More jobs like this