Evaluating digital first primary care – the challenges ahead

The way we see our GP is changing. While face to face interaction has been the traditional way we are now seeing a growth in digital first primary care, which is where the first contact a patient has with their GP is via a digital channel. 

The NHS Long Term Plan sets out a vision to offer digital first primary care to all patients in England within the next 10 years. The ambition is to provide straightforward access to primary care services, eliminate variation in the pathway of care, and help patients manage their own health better, with the idea being that clinicians will be able to focus on those with the greatest need. 

The industry is responding with a wide range of different digital first models, from digital triaging systems, through video-consultation offers, to models where a general practice is almost entirely replaced by a virtual online practice.

What difference is this making?

That’s the sort of question the Improvement Analytics Unit (IAU) loves to try and answer. As part of our partnership with NHS England and Improvement, we are carrying out an evaluation of digital first primary care, working alongside a qualitative evaluator (to be announced soon). 

We will be looking at a range of digital first models across a number of settings. In doing this, we hope to provide some valuable insight into how the different aspects of digital first primary care can be adopted with, or alongside, traditional primary care services to result in better, more accessible and more efficient primary care services for all.

New technologies offer endless possibilities to improve how we work across the whole health sector. But for all the potential these innovations carry, we should also be aware of their inherent challenges, such as the risk of increasing demand for care or worsening health inequalities. 

Robust approaches to evaluation are vital if everyone’s health and care are going to benefit from new technology. Monitoring what is really going on as we introduce these new technologies will help the health service focus resources on what is most effective and avoid any unintentional consequences.  

I’ve made this sound very straightforward. But in reality, the increased use of digital solutions within health care opens up all sorts of new questions and challenges for the people responsible for evaluating their impacts. 

What sort of challenges will evaluators face? 

A number of specialised centres, including the IAU, have been established over the last few years to tackle the challenge of robustly evaluating changes in health service delivery. We are using advanced statistical techniques to compare new models of care with a counterfactual. The latter is, in plain English, our best estimate of what would have happened in the absence of these changes in health service delivery. 

Applying these methods to innovations in digital technology presents a new set of challenges that evaluators will face. These include:

1. If digital technology is available alongside traditional forms of health care, as seems to be more and more the case, how does an evaluator account for the differences in patients that prefer digital services over traditional care? Are some patients more likely to be excluded from using these services because of their complex needs, or experience using digital services?
    
2. When evaluating technology that can evolve over time through self-learning, for instance by updating the algorithm based on the data collected, how often and when does the technology need to be evaluated, and how reliable can we expect the findings to be over time? And if an algorithm performs better for some populations than others, can this further widen pre-existing health inequalities across different patient groups?  
    
3. Changes in technology happen fast and innovations are often designed to have an agile approach. Can the evaluation keep pace with this way of working? 
    
4. The evaluation of data driven technologies will require a solid understanding of the data used. How can we ensure that this essential understanding of how data are collected and used is common across all evaluators, and build this capability? 
    
5. Any evaluation needs to consider the context in which technologies are introduced, so that we can understand what might be contributing towards their impact.

As digital technologies become more common place in the NHS, the regulation that applies to these technologies will change over time, and so will the challenges faced by evaluators. 

Building a new evidence base

As the first evaluations of new technologies emerge, it will be essential to analyse what learning we can draw from these. And we will need to know how to translate this learning into policy and best practice.

The Improvement Analytics Unit will lead the way in contributing to this evidence base, starting with our work to evaluate digital first primary care in the NHS. We are currently talking to a number of providers of digital first primary care services, to identify key questions of interest to the providers and the NHS. We aim to share early findings next year. 

Through our work we hope to develop better ways to evaluate digital technologies, recognising that there are some very specific challenges that make it different to evaluating other forms of innovation. We’ll share what we learn along the way, as well as any tools we will be developing including the syntax used by the IAU.

Arne Wolters (@4RN3W0L73R5) is Acting Head of the Improvement Analytics Unit (IAU)

This content originally featured in our email newsletter, which explores perspectives and expert opinion on a different health or health care topic each month.