Principal Data Engineer

Canada Life UK looks after the retirement, investment and protection needs of individuals, families and companies. We help to build better futures for our customers, our intermediaries and our employees by operating as a modern, agile and welcoming organisation.

Part of our parent company Great-West Lifeco, Canada Life UK has operated in the United Kingdom since 1903. We have hundreds of respected and supported employees committed to doing the right thing for our customers and colleagues.

Canada Life UK is transforming to create a more customer-focused business by providing our customers with expertise on financial and tax planning, offering home finance and annuities propositions, and providing collective fund solutions to third party customers.

Job Purpose

The Principal Engineer will provide expert-level technical leadership in Azure cloud and Databricks-based solutions. The role requires deep expertise in Azure cloud services, Databricks, data engineering, and DevOps, leading a cross-functional team to build, deploy, and support high-performance data-driven solutions. 

The role involves: 
 

• Designing and implementing scalable data solutions on Azure and Databricks. 
• Developing and optimizing data pipelines and ETL processes using Azure Data Factory, Databricks, and Synapse Analytics. 
• Building and managing CI/CD pipelines for data applications with Azure DevOps. 
• Ensuring compliance with security best practices and governance frameworks in cloud and data solutions. 
• Driving cloud modernization initiatives, including migrating on-prem data workloads to Azure. 
• Leading engineering best practices to ensure data quality, performance, and reliability. 

Duties/Responsibilities 
 

• Develop and optimize data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). 
• Implement data lake architectures using Azure Data Lake, Delta Lake, and Synapse Analytics. 
• Build machine learning and real-time analytics solutions leveraging Databricks MLflow and Azure AI services 
• Develop CI/CD pipelines for data applications using Azure DevOps, Terraform, and Bicep. 
• Automate infrastructure provisioning and infrastructure-as-code (IaC) using Terraform or Bicep. 
• Implement monitoring & logging solutions with Azure Monitor, Datadog, or Prometheus. 
• Enforce security, access control, and compliance in Azure and Databricks. 
• Apply cost optimization strategies for efficient cloud resource usage. 
• Ensure adherence to data governance and lineage using Purview and Databricks Unity Catalog. 
• Lead code reviews, knowledge sharing, and mentoring within the engineering team. 
• Ensure their assigned team, work within organisational guardrails and meet engineering standards and best practice. They are collectively responsible for the quality of applications in their eco-system. 
• Experiment with new Azure and Databricks features to improve solution architecture. 
• Principal engineers help the team develop and implement DevOps practices, balancing build and run responsibilities, removing technical debt and making their applications more automated and reliable over time through removal of toil.  

Skills, Knowledge and Experience 
 

Principal Engineers are expected to have expertise in at least three of the following areas of Engineering practices 

Core skills 

• Automation including Testing 
• Possess strong teamwork, communication, and problem-solving skills to ensure they can collaborate effectively with cross-functional teams and bring out better results. 
• Security Best Practices: Awareness of security principles and best practices to ensure secure software delivery and infrastructure management. 
• Continuous Learning: Keeping up-to-date with the rapidly evolving technologies and trends in the industry. 
• Monitoring and Troubleshooting: Ability to set up monitoring and logging solutions and troubleshoot issues effectively. 
• Strong experience working within an agile development methodology  
• Excellent time and self-management through effective planning & prioritisation of tasks. 
• Proven and demonstrable engineering capability 
• Ability to influence others and strong communication skills 

Data Engineer (New Technology / Microsoft) 

• Expertise in Databricks (Spark, PySpark, Delta Lake, MLflow, Unity Catalog). 
• Strong proficiency in Azure Cloud (Azure Data Factory, Synapse, Data Lake, Functions). 
• Experience with CI/CD pipelines (Azure DevOps, GitHub Actions, Terraform, Bicep). 
• Hands-on experience with SQL and NoSQL databases (Azure SQL, CosmosDB). 
• Scripting & Programming skills (Python, Scala, PowerShell). 
• Deep understanding of DevOps & automation (Terraform, ARM, YAML pipelines). 
• Strong understanding of cloud security, compliance, and governance. 

Qualifications

• Degree level IT or technical/scientific subject.  
• Microsoft Azure Data Engineer or Solutions Architect certification. 
• Databricks Certified Data Engineer or Machine Learning Associate. 
• Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming). 
• Knowledge of machine learning & AI on Databricks. 

Benefits of working at Canada Life

We believe in recognising and rewarding our people, so we offer a competitive salary and benefits package that’s regularly reviewed. As a Canada Life UK colleague, you’ll receive a competitive salary and comprehensive reward package including a generous pension and bonus scheme, along with, income protection, private medical insurance and life assurance.  We have a fantastic number of other benefits and support services as well as regular personal and professional development.

How we work at Canada Life

Our culture is unique and incredibly important to us.  We care about doing the right thing for our people, customers and community and helping others to build better futures.  Our blueprint behaviours shape and influence how we work, and are central to the relationships we have with others.  Every day we are encouraged to be more curious, own the outcome, face into things together and find a way forward. 

We want colleagues to have rewarding careers with us so we invest in the development of our people, technology and workplaces. That’s why we offer a range of training, flexible working and opportunities to grow and develop.

Diversity and inclusion

Building an inclusive workplace with a diverse workforce where everyone can feel they belong and achieve their potential regardless of gender, ethnicity or any other characteristic is a key commitment for us.  We are proud of the progress we’re making in DEI, and we continue for it to be a significant focus. 

“At Canada Life we believe in the power of great people from different backgrounds, experiences and perspectives coming together to build better futures. Emerging talent is crucial to our growth and creating an environment that continues to inspire us all.”  Nick Harding, Chief People Officer, Canada Life UK

We appreciate that everyone has different work and life responsibilities. We’re happy to discuss flexible working arrangements, including part time, for any of our roles should this be a requirement for you.