Cloud Platform Engineer

Cloud Platform Engineer, Hybrid, London

12 month Fixed Term Contract

The Cloud Platform Engineer will be responsible for managing Pay.UK’s Microsoft Azure and AWS cloud platforms including all domains, subdomains, account subscriptions, services and the Azure and AWS platform tenants.

Manage the entire Microsoft Azure Fabric technology stack and associated data processes and procedures, including the development and maintenance of data pipelines and Extract, Transform and Load processes.

Manage the entire AWS technology stack and associated simulation modelling tools, Software Development Kits, code repositories, data processes and procedures, including the development and maintenance of data pipelines development lifecycle processes.

Provide technical knowledge for the design, implementation and maintenance for data ingestion, integration, transformation and optimised schema designs in a multi-cloud environment and maintain the hybrid cloud environment with legacy on-prem engineering systems, technologies and processes.

Conduct research, present architecture and design options, refine architecture design, where necessary and take it through appropriate design authority and governance forums.

Manage the planning for the migration of all on-prem BI infrastructure to the cloud native data platforms. Design, develop and embed the processes to enable effective use of multi-cloud approach.

Manage the platform engineering responsibility with application deployment that enables Big Data technologies and moves the Data & Analytics team in the right direction that improves data maturity. 

Support the migration of legacy databases to a new cloud environment that allows the processing of structured, unstructured and semi-structured data and supports commercial opportunities.

Maintain data integrity, reduce data redundancy and promote good data lineage in addition to managing general system and user management.

Develop and publish applications with the use of Power Apps and leverage the Dataverse and the Common Data Model (CDM) for robust integration. Develop Robotic Process Automation (RPA) with Power Automate and other similar RPA tooling. 

Apply data strategy including maintaining associated Data Policies that enable Pay.UK strategy, regulatory and ecosystem needs, and drive commercial opportunities that may arise from data exploitation.

Key Accountabilities

Develops new data pipelines and ETL processes with the use of a variety of integration and automation tools including SPARK notebooks, DevOps, Azure Data Factory, PBI Dataflows or Power Automate.

• Maintain the data platform backend datalake and all ETL/ELT processes.
• Manage the AWS EC2 instance and all related services, accounts and subscriptions.
• Manage all DevOps pipelines for the Microsoft Fabric platform and all Git repositories affiliated in the AWS environment.
• Manage all SPARK notebooks and oversee the deployment of all machine learning models and pipelines in SPARK clusters.
• Build new applications using Power BI and Power Apps, and manage the dataverse, datamarts and Common Data Model.
• Manages user administration, along with responsibility for the administration of the systems and databases; ensures timely updates to reference data, as relevant across the various data repositories.
• Develops architecture designs and carryout related source to target mapping.
• Oversees the development of the framework, processes and systems required to provide a central view of data.
• Manages the development of a federated data access model across Pay.UK and the wider ecosystem including federated Reporting and Analytics framework.
• Ensures sourcing and integration of external data for enriching Pay.UK datasests and for ensuring data integrity.
• Carries out diagnostic analysis to assess root cause and remedy system errors and data quality issues. 
• Initiates efficiency and effectiveness improvements through automated outlier detection and data validation. Explores, validates and manages information management sources, using independent resources, where approved.
• Promotes self-service reporting and analytical tools adopted across the business.
• Data preparation, cleaning, validating, transforming data to ensure accuracy, completeness, uniformity and suitability for prescriptive and predictive modeling.
• Implements the data engineering and visualisation standards and ensures data policies, processes, governance and tools are updated to agreed standards, to deliver data requirements.
• Comply with Pay.UK’s Data Protection Policy and all associated procedures and processes and communicate any data protection risks to the Data Protection team without delay.

Skills, Experience & Qualifications

• Education – BSc/MSc in STEM subject, or equivalent experience
• Advanced knowledge in administering the Microsoft and AWS BI stack
• Experience Managing Microsoft cloud tenant including configuration and optimisation of tenant settings
• Significant exposure in a wide business intelligence technology stack (Microsoft Fabric, Databricks, ADLS, OneLake, Azure ML Studio, AWS EC2 instance, Git Repositories, R studio, Jupyter Notebooks, SPSS, Visual Studio, SSMS, SSRS, SSAS and SSIS)
• Experience designing cloud technology solutions and developing architecture designs
• Knowledge of a computer programming language including SQL, Python, R, Java and JavaScript

At Pay.UK, we value diversity and inclusivity. Research has shown that candidates from underrepresented groups may hesitate to apply unless they meet all the requirements listed. We encourage all qualified candidates to apply, regardless of how closely their skills and experience match the requirements. We are committed to supporting accessibility needs and creating a welcoming environment for all employees. Become part of our team and contribute to the creation of an inclusive work environment that values everyone's unique input.

About Us

Pay.UK maintains and develops the UK retail payment systems and standards that are core to the economy being able to function on a day-to-day basis.

From Bacs to Faster Payments and cheques – we act as the single operator for all UK retail payments. We put the needs of consumers and businesses at the heart of everything we do, working in the public interest to ensure that the systems the country relies on for its banking transactions are safe, open, innovative and resilient.

Our payment systems underpin the services that enable funds to be transferred between people and institutions. In 2023, the UK's retail payment systems processed 11.4 billion transactions worth £9.3 trillion through Bacs Direct Credit, Direct Debit, Faster Payments, and cheques, and our Current Account Switch Service has facilitated over 10.2 million switches since its launch in 2013.

Every day, people and businesses use the services we provide to get their salaries, pay their bills and make online and mobile banking payments. Our vision for the future is to enable a vibrant economy, with Pay.UK delivering the best-in-class payment infrastructure and standards for the benefit of consumers and businesses nationwide.

Benefits & Additional Information

• 12% Non-contributory pension
• Discretionary annual bonus 
• 30 days annual leave (excluding bank holidays)
• Private medical insurance, life assurance, income protection, health cash plan, dental insurance, Bupa medicals etc
• Employee assistance programme
• Cycle to Work Scheme
• Season ticket loan
• Annual fitness subsidy of up to £500 per annum
• Working from home policy - minimum 40% in the office (eg. 2 days in the office over a 5 day working week)

Please note:

• Some of our benefits are only available to colleagues upon successful completion of the probationary period.