Senior Data Engineer

We're looking for a Senior Data Engineer with deep expertise in healthcare data to join a pioneering team building data products that enable world-leading research. Whether your experience comes from academia, healthcare, biotech, or a fast-growing health-tech startup, you'll bring a strong understanding of healthcare data standards such as OMOP or FHIR, alongside the technical leadership required to be a Senior Engineer.

In this role, you'll design and build scalable data pipelines that transform complex health records into trusted, research-ready datasets used by scientists and researchers tackling challenges with global significance. You'll work at the intersection of engineering, data science and epidemiology, helping to establish best practices, shape technical direction and create reusable, high-quality data products that can operate at industrial scale.

If you're a senior engineer who enjoys solving complex data challenges, influencing how healthcare data is curated and delivered, and making a meaningful impact on research that improves lives, we'd love to hear from you.

Our Future Health will be the UK’s largest ever health research programme, bringing people together to develop new ways to detect, prevent, and treat diseases. Diseases like cancer, dementia, diabetes, and heart disease affect the lives of many people in our communities. Our goal is to create a world-leading resource for health research, to improve our understanding and spot the patterns of how and why common diseases start, so treatments can begin sooner and be more effective.

Our plan is to bring together 5 million volunteers from right across the UK who will be asked to contribute information to help build one of the most detailed pictures we have ever had of people’s health. Researchers will be able to use this information to make new discoveries about human health and diseases. So future generations can live in good health for longer.

What you'll be doing:

• Designing and building scalable data pipelines that ingest healthcare data from multiple providers into trusted research environments and data platforms.
• Developing and maintaining data transformation logic in Python to create high-quality, curated and research-ready datasets.
• Contributing to complex, shared codebases while following engineering best practices including code reviews, testing and clean coding standards.
• Collaborating with data scientists, epidemiologists, software engineers and product teams to understand requirements and deliver impactful data solutions.
• Processing large-scale healthcare datasets and implementing robust data quality, validation and governance controls.
• Transforming and harmonising healthcare data into common data models such as OMOP and FHIR to support research and analytical use cases.
• Creating reusable, scalable and maintainable data products that enable efficient consumption by multiple stakeholders.
• Communicating effectively with technical and non-technical stakeholders to translate complex requirements into practical engineering solutions.
• Producing clear and comprehensive technical documentation covering data pipelines, transformation logic, architecture and operational processes.
• Supporting the design, scoping and implementation of data flows across the wider healthcare data ecosystem.
• Leveraging cloud and big data technologies, including Azure, Databricks, Spark and Data Lakes, to deliver high-performing data solutions.
• Adopting and promoting data engineering best practices, healthcare data standards and continuous improvement across the team.
• Keeping abreast of emerging trends, technologies and standards within healthcare, research and data engineering to drive innovation and excellence.

Requirements

To succeed in tis role you will need to demonstrate some of the following:

• Proven Data Engineering experience within complex, large-scale data environments.
• Experience designing, building and maintaining scalable, reliable data pipelines that process large volumes of data from multiple sources.
• Experience working with healthcare data across academia, healthcare, life sciences, health technology or related sectors.
• Knowledge of healthcare data standards and common data models, including OMOP and/or FHIR.
• Experience transforming and remodelling data between different Common Data Models (CDMs).
• Understanding of healthcare terminologies and ontologies, including SNOMED CT, ICD-10 and Read Codes.
• Experience contributing to shared codebases and developing reusable, maintainable data solutions.
• Proficiency in Python and software engineering best practices.
• Good command-line and Unix/Linux skills.
• Experience working with cloud platforms, ideally Microsoft Azure.
• Experience using technologies such as Spark, Databricks and data lake architectures.
• Knowledge of data storage and transformation formats such as Apache Parquet and Delta Lake.
• Experience with containerisation and orchestration technologies, including Docker and Kubernetes.
• Experience using Git/GitHub and working within collaborative development environments.
• Experience working within Agile delivery teams.

Benefits

• Competitive base salary from £74,000 per annum
• Generous Pension Scheme – We invest in your future with employer contributions of up to 12%
• 30 Days Holiday pro rata + Bank Holidays – Enjoy a generous holiday allowance with the flexibility to take bank holidays when it suits you
• Enhanced Parental Leave – Supporting you during life’s biggest moments
• Cycle to Work Scheme – Save 25-39% on a new bike and accessories through salary sacrifice
• Home & Tech Savings – Get up to 8% off on IKEA and Currys products, spreading the cost over 12 months through salary sacrifice
• EV Scheme – Save up to 40% on a brand new electric vehicle all-inclusive package through salary sacrifice
• £1,000 Employee Referral Bonus – Know someone amazing? Get rewarded for bringing them on board!
• Wellbeing Support – Access to Mental Health First Aiders, plus 24/7 online GP services and an Employee Assistance Programme for you and your family
• A Great Place to Work – We have a lovely Central London office in Holborn, and offer flexible and remote working arrangements

You may work remotely from anywhere within the UK, with occasional travel to London.

We recommend you apply as soon as possible as occasionally due to high volumes of applications, we need to close our postings early.

At Our Future Health, we recognise the importance of having a diverse workforce and ensuring that all candidates, regardless of their background, have equitable access to our application process. We proactively encourage applicants who identify as having a disability, neurodiversity, or long-term health conditions to let us know if they require any reasonable adjustments as part of their application process.

If you do require any reasonable adjustments, please email us at

Originally posted on Himalayas