Lead Data Ops Engineer

About Emirates Global Aluminium

Emirates Global Aluminium is the world’s biggest ‘premium aluminium’ producer and the largest industrial company in the United Arab Emirates outside the oil and gas industry.  EGA is an integrated aluminium producer, with operations on four continents from bauxite mining to the production of cast primary aluminium and recycling. EGA employs over 7,000 of these people including more than 1,200 UAE Nationals. EGA operates aluminium smelters in Jebel Ali and Al Taweelah in the United Arab Emirates, an alumina refinery in Al Taweelah, a bauxite mine and associated export facilities in the Republic of Guinea, a speciality foundry in high strength recycled aluminium in Germany, and a recycling plant in the United States.

JOB PURPOSE:
The Lead Data Ops Engineer will play a critical role in building, scaling, automating, and maintaining the company's Big Data infrastructure and Machine Learning operations (MLOps). This individual will work in close collaboration with our Data Architect, Data Science, Data Engineering, and IT teams to ensure the development, deployment and scale of robust, high-performance data processing systems and ML models.

KEY ACCOUNTABILITIES:

• Big Data Infrastructure: Design, build, and maintain high-performance, cloud-based, fault-tolerant, scalable distributed Data infrastructure that supports the company’s data-intensive applications. (Real time/Batch/LLM’s). This includes developing strategies for data storage (TB’s), processing, and analysis, and implementing high-performance, scalable data pipelines for ML models and data products, supporting up to 50-60 use cases a year and thousands of IoT devices.

• Create infrastructure as Code, perform configuration and set up managed data services. Build and deploy a data science playground for research and prototyping for the professional and citizen data science program being rolled out and supporting 15-20 citizen data scientists/ambassadors.

• Machine Learning Operations (MLOps): Develop and manage the ML operational process, working closely with the data science team to implement ML models into production, including edge. This includes streamlining the ML lifecycle, from model development and testing to deployment and implementing the monitoring and alerting strategy.

• Automation and Scalability: Implement automation tools and frameworks to manage system updates/changes. Ensure that all systems and infrastructures can scale effectively with the increase in IoT sensors and devices.

• Continuous Integration and Deployment (CI/CD): Oversee continuous integration and continuous deployment practices for the data and ML pipeline, ensuring that software can be reliably released at any time.

• System Monitoring and Reliability: Monitor system performance and reliability to ensure high levels of performance, availability, and security. This includes identifying and fixing potential and existing system issues. Collaboration and Communication: Strong collaboration with Data Architect/Engineer, data scientists for the implementation and testing of new data services to provide an elastic data infrastructure.

• Security: Oversee and ensure that all Big Data and ML Ops platforms comply with the company's security standards and policies.

• Mentorship and Leadership: Act as a mentor to junior data members, providing guidance and support in their professional development. Promote a culture of performance, collaboration, and continuous learning within the data team.

• Innovation and Continuous Improvement: Stay up-to-date with industry trends and new technologies. Continuously explore innovative solutions and enhancements to the existing data architecture to improve its scalability, reliability, and efficiency.

• Problem Solving: Anticipate and resolve technical issues before they become roadblocks, maintaining the continuity of data flow and ensuring the highest levels of data quality and integrity.

AUTHORITY/DECISION MAKING:

• Infrastructure Design: Decide on the most effective design and implementation of the company's Big Data infrastructure.

• ML Ops Process: Make key decisions on the ML operational process, ensuring that ML models can be effectively integrated into production.

• Automation Tools: Choose the most appropriate automation tools and frameworks for the company's needs.

• CI/CD Practices: Determine the best practices for continuous integration and deployment in the context of the company's operations.

• System Monitoring: Make decisions on system monitoring strategies, including the selection of tools and responses to system performance metrics.

• Security Policies: Have a say in the implementation of security policies as they pertain to the Big Data and ML Ops platforms.

• Budget and Costing: Taking ownership of managing data platform costs and relevant data services.

QUALIFICATIONS & SKILLS:

Domain Expertise:

• Bachelor’s degree required, MS or PhD preferred.

• Bachelor’s in Data Science, Computer Science, Engineering, Statistics and 10+ years of relevant experience.

• Experience: A minimum of 5-7 years of experience in a DevOps role, with a focus on managing Big Data infrastructures and MLOps.

Technical Skills:

• Strong experience with Big Data technologies such as Hadoop, Spark, Kafka, etc.

• Proven expertise in managing and deploying ML models into production.

• Proficient in using CI/CD tools like Jenkins, Travis CI, CircleCI, etc.

• Proficient in using infrastructure automation tools like Terraform, Chef, Puppet, Ansible, etc.

• Strong knowledge of cloud platforms such as Azure (AWS, GCP).

• Experience with containerization technologies like Docker, Kubernetes, etc.

• Familiarity with various database technologies, both SQL and NoSQL.

• Proficiency in programming languages such as Python, Java, or Scala.

• Experience of leveraging MS/Azure ecosystem to manage the development and maintenance of cloud platform operations.

• A broad set of technical skills and knowledge across hardware, software, systems and solutions development.

• A proven track record of using quantitative analysis to impact key business or product decisions.

• Solid grasp of and experience with implementing and operating software development methodologies.

• A solid grasp of common statistical applications and methods (A/B tests and multivariate experiments, probabilities, regression).

• Understanding of Agile Software Development Lifecycle and project planning/execution skills.

• Outstanding communication skills with stakeholders at all levels, managing stakeholders’ expectations and facilitating discussions across high-risk or complexity under constrained timescales.

• Outstanding capability to establish enterprise-scale data integration procedures across the data development life cycle and ensure that teams adhere to these. Able to manage resources to ensure that data services work effectively at an enterprise level.

• Up to date with data innovation and expert in investigating emerging trends in data-related approaches, performing horizon-scanning for the organization and introducing innovative ways of working.

• Expert in Data integration design and can establish standards and well informed on best practices across different industries. The candidate can distinguish how to keep those standards up to date and ensures adherence to them.

• Good understanding of concepts and principles of data modeling and can produce, maintain and update relevant data models for specific business needs. Also, shows good knowledge on how to reverse-engineer data models from a live system.

• Expert in Metadata management and understands how metadata repositories can support different areas of the business. Capable of promoting and communicating the value of metadata repositories and knows how to set up robust governance processes to keep repositories up to date.

• Expert in identifying and anticipating problems and knowing how to prevent them by linking how problems fit into the larger picture. Has the ability to identify and describe problems, help others to describe them, and build problem-solving capabilities in others.

• Expert in setting up team-based data engineering standards for programming tools and techniques and can select appropriate development methods. Acts as an advisor on the application of standards and methods and ensures compliance while taking technical responsibility for all stages and/or iterations in a software development project, providing method-specific technical advice and guidance to project stakeholders.

• Technical Expert in predicting and advising on data engineering future technology changes that present opportunities for a product or program.

• Experienced with reviewing requirements, specifications and defining test conditions with good understanding on how to identify issues and risks associated with work while being able to analyze and report test activities and results.

Agile/Digital Experience:

• Experience in Agile Development, with specific Data Engineer/Data Architect (or similar) experience preferred.

• Understands relationships with Product Owner, Agile Coach, Data Scientist, Designer, and the rest of the technical team.

Individual Skills:

• Exceptional problem-solving skills: demonstrated ability to understand business challenges, structure complex problems, develop solutions.

• Ability to partner and influence key business stakeholders at all levels of the organization.

• Strong skills in team leadership and networking, ability to work across multiple organizations to accomplish diverse goals.

• Exceptional presentation, written, and verbal communication skills.

• Strong communication skills with the ability to align the organization on complex technical decisions.

• Active coach and mentor whose goal is to grow and maximize the team’s potential.

• Strong ability and enthusiasm around data strategy and ability to inspire team and organization around the usage of data.

Mindset & Behaviors:

• High energy and passionate individual who inspires teammates to reach their maximum potential.

• Empathetic coach who can help develop a new group of highly motivated data engineers.

• Excited about trying new solutions outside standard approval.

• Invested in developing a culture of trust, free thought, and complete transparency.