Senior Data Engineer

The Senior Data Engineer is responsible for the architecture, operation and development of Cash Access UK’s data platform, built on Microsoft Fabric and Azure. The role works closely with the Insights & Analysis team, business data owners, the Data Governance Consultant, Technology Operations, suppliers and the Head of Data and Technology to keep the platform reliable, bring new data into it safely and ensure the data it holds can be trusted for reporting and decision making.
The platform brings together data from the banking hubs, the Post Office and community banking channels, incident and survey systems, reference data, member banks and public data sources. It was built with a delivery partner and is now moving into business-as-usual operation, and the Senior Data Engineer takes ownership of it as that transition completes, including the runbook, documentation, backlog, alerting and recovery arrangements.
This is a hands-on engineering role where design, build and operation sit with the post holder rather than with a supplier. The Senior Data Engineer is accountable for the platform’s data pipelines, data models, databases, APIs and Microsoft Fabric integrations, for the engineering standards that govern how they are built and released, and for reducing the organisation’s reliance on supplier-led development.
The role supports data quality and governance by implementing the data quality rules, critical data elements, definitions, lineage, classification, access controls and retention set through the data governance framework, so that governance is built into how data is ingested, transformed and shared. The Data Governance Consultant establishes the framework and Microsoft Purview; this role implements them in the platform.
The role will also help set the direction of the platform over the longer term, including new data sources and integrations, the data foundations for Finance reporting and dashboards, and the governed datasets that allow approved AI tools to be used, working with the wider Technology team, the Insights & Analysis team, Risk and relevant suppliers to prioritise the backlog.
The role also acts as a business partner within the IT function, working with Finance, Risk, Compliance, Projects, Change and wider business teams to make data practical, trusted and usable. It helps unblock data-related delivery issues, clarify requirements and definitions, coordinate supplier action where specialist support is used, and ensure the platform serves business outcomes without becoming a bottleneck.

Responsibilities

• Own the architecture, operation and reliability of the data platform, including the Azure landing zone, the Lakehouse and its Bronze, Silver and Gold layers and the development, test and production environments.
• Lead the transition of the platform from the delivery partner into business-as-usual operation, completing knowledge transfer and taking ownership of the runbook, documentation, backlog, alerting, backup and recovery arrangements.
• Operate the platform day to day, including scheduling and orchestration, monitoring, incident and problem resolution and performance, so that data is available and accurate when the business needs it.
• Design, build and maintain data pipelines for new and existing sources, including member bank and Post Office files, incident and survey systems, reference data and public APIs, replacing manual handling of Excel and CSV files.
• Design and maintain the data models, including star schemas, source-of-truth datasets and semantic models, and build new databases, data marts and APIs that make platform data available to other systems in a controlled way.
• Implement data quality, validation and cleansing controls, applying the rules, critical data elements and definitions set through the data governance framework, so that inconsistent data is identified before it reaches reporting.
• Implement the data governance framework technically, including the integration between Microsoft Fabric and Microsoft Purview, lineage, classification and data retention, working with the Data Governance Consultant and business data owners.
• Own the platform security model, including workspace and row-level access, the secure upload area for external partners and service principals, working with the Technology Operations Manager, the Cyber Security Manager and the MSP.
• Own the engineering practices for the platform, including Azure DevOps source control, CI/CD and infrastructure-as-code deployments, testing, release management and the solution documentation.
• Support the Insights & Analysis team and business data owners by prioritising the data backlog, translating business questions into trusted data, and supporting the Insights & Analysis team with their Power BI reporting built on the platform, including the foundations for Finance reporting.
• Maintain clear reporting on platform health, data quality, risks and issues for the Head of Data and Technology, and reduce key-person dependency by documenting the platform so that others can support it.

As the data warehouse grows there will likely be scope for a Junior Data Engineer who will report into the Senior Data Engineer.

Requirements

Role specific skills

• Microsoft Fabric data engineering: Substantial hands-on experience delivering and operating production workloads on Microsoft Fabric, including OneLake, Lakehouse and Warehouse, Data Factory pipelines.
• Azure platform: Working knowledge of the Azure services around Fabric, including landing zones, resource groups, Entra ID security groups, service principals and deployment service accounts.
• Data architecture and modelling: Strong dimensional modelling skills, including star schemas, fact and dimension tables and slowly changing dimensions, experience of the Bronze, Silver and Gold pattern, and the ability to design source-of-truth datasets for reporting and analytics.
• Languages and tooling: Expert SQL; strong Python and PySpark; Power BI and DAX to the level needed to build and maintain semantic models and support report developers.
• Data integration: Experience ingesting Excel and CSV files, REST APIs, databases and cloud data stores, handling changing and inconsistent source formats, and building APIs or other interfaces that make platform data available to other systems.
• Data quality: Experience implementing validation, reconciliation and data quality monitoring, and working with business data owners to resolve data issues at source.
• Data governance implementation: Experience working with Microsoft Purview, including scanning, classification and lineage, and applying a data governance framework, business glossary and data ownership model in practice.
• Security and access control: Ability to design least-privilege access in Fabric, including workspace roles and row-level security, to secure data exchange with external partners, and to handle financial and personal data in line with data protection requirements.
• DevOps for data: Experience with Git and Azure DevOps, CI/CD and infrastructure as code for Fabric, environment management, automated testing and disciplined release management.
• Platform operations: Experience running a business-critical data platform, including monitoring, alerting, incident and problem management, backup and recovery and performance tuning, and producing and maintaining runbooks.
• Supplier management: Experience taking over a platform from a delivery partner and working with specialist data suppliers and consultants while retaining internal ownership of design decisions and standards.
• Business partnering: Ability to work with Finance, Risk, Compliance, Projects, Change and business teams to understand requirements, agree definitions and explain data clearly in business terms.
• Sector experience: Experience in financial services or another regulated environment, or of working with data from banks, is desirable.

Leadership at Cash Access UK

• Build effective relationships across the Insights & Analysis team, business data owners, technology colleagues and suppliers, creating a constructive environment where data issues are raised and resolved early.
• Lead by influence, bringing clarity, pace and practical problem-solving to data-related activity across the organisation.
• Champion trusted data as an enabler of good decisions, helping teams move from spreadsheets and manual processes to governed, reusable data without creating unnecessary blockers.
• Support colleagues with clear communication, pragmatic guidance and proportionate advice on data requirements.
• Create confidence by following through on data actions, escalating when needed and maintaining transparent documentation and evidence of progress.

Establish psychological safety so colleagues feel confident to raise data quality concerns, risks and blockers early.

Originally posted on Himalayas