UK’s NICE Outlines Standards That AI and Data-Driven Medtech Should Meet For NHS Uptake

On 9 August, the UK National Institute for Health and Care Excellence (NICE) brought in changes to its evidence standards framework (ESF) for digital technologies to include guidance on AI-based products and bring HTA guidelines more in line with regulatory requirements following consultation with experts at Imperial College London, The University of Birmingham and the Alan Turing Institute, the UK’s national institute for data science and artificial intelligence.

The organization describes the framework updates outline as a “subset of early deployment standards” that companies can use within evidence generation programmes for digital health technologies (DHTs) and that will make the ESF guidance document more user-friendly. .

NICE defines a data-driven DHT as one that:

• Contains algorithms that were trained using patient data or datasets, which could be adaptive, meaning they change over time, or fixed; and

• Uses decision thresholds or cut-off values (such as for diagnosing a condition or triaging patients for different treatments) that were created using patient data or datasets.

Framework “Too Prescriptive”

While the latest updates are intended to provide clarity for product developers incorporating AI into their products, Medtech Insight was told by trade body the Association of British HealthTech Industries (ABHI) that the framework could be a barrier to innovation.

Andrew Davies, digital health lead at ABHI, told Medtech Insight that NICE’s ESF requires companies to have generated “an extremely high bar of evidence” prior to the product having been deployed on the NHS, which requires a heavy degree of investment.

“While in principle we support the high standards of evidence requirements proposed, we are concerned that they are too prescriptive for both innovators and commissioners and not sensitive to the challenges faced during evaluations of DHTs,” Davies explained.

In particular, ABHI cited concerns around the capacity and potential expertise of evaluators at both a local and NHS commissioning level to evaluate products using the framework.

Davies pointed out the potential for AI-enabled products to address workforce issues in the NHS and said it is “vital that maximum use is made of the early deployment framework”, as outlined in section D of the NICE ESF.

The early deployment subset of the ESF applies to DHTs that are at an early development stage and are therefore unlikely to meet the standards set out in the framework. To combat this challenge, products that fall into the early deployment category can instead be evaluated through NHS-led evidence-generation programs.

These programs support the piloting or early roll-out of a technology with the aim of gathering data that demonstrate the DHT’s effectiveness, place in the care pathway and economic impact.

Other Issues Identified

NICE consulted with medtech development companies as well as digital health organizations while developing the ESF updates.

In a separate document, the HTA body outlined some of the main feedback it received from consultees and the changes that it made prior to releasing the final guidance.

Of the 50 responses to the public consultation received, 55% agreed that the ESF would not create barriers to innovation, NICE said.

NICE said it received “several comments” in which respondents said more clarity is needed within the ESF and its supporting documents, particularly regarding the placement of the ESF in the innovation landscape and how it relates to regulatory requirements, NICE evaluation and reimbursement decisions.

Other respondents expressed concerns around how the framework would be implemented in practice and whether this would mean further training for evaluators using the framework to assess a product.

NICE said that in response to this feedback it has improved the document’s wording, merged some similar evidence standards considerations for simplicity, and amended itsdescription of the ESF’s position in the user guide.

Overlap with MHRA

A key difference between NICE’s ESF for DHTs and the requirements of the UK’s regulator, MHRA, is that NICE evaluates whether a product is both cost-effective and clinically effective in the context of the NHS treatment pathway.

The ESF sets out how sponsors should demonstrate that a product has economic value in the context of the UK’s healthcare system, as well as the evidence that must be provided to show a technology is of a high quality.

One of the changes included in NICE’s most recent ESF update, according to the organization, was the alignment of its DHT classification system with that of the MHRA.

Johan Ordish, MHRA Head of Software and AI, Innovative Devices Division, told Medtech Insight this week that the regulator has helped support NICE in developing the framework so that it is aligned with regulatory requirements where possible.

“Alignment is important because it means manufacturers can spend their time generating core evidence related to the safety and effectiveness of the product; it reduces regulatory burden, and leads to patients receiving access to safe, effective product more quickly,” he said.

NICE said that it expects the next ESF update to follow the MHRA’s implementation of the Software and AI as a Medical Device Change Programme. (Also see “Time To Comment On ‘Bold New Regulatory Regime’ For UK”, Medtech Insight, 17 Sep, 2021).

As outlined by NICE in its consultation document (see above), some companies have expressed confusion around how the ESF is placed within the broader regulatory and reimbursement setting. NICE said that the document “is set of evidence standards that DHTs should meet before commissioning in the NHS and care system”.

This means that DHTs must comply with regulatory requirements such as the MHRA requirement for UKCA marking, the Care Quality Commission regulations for digital health services and UK General Data Protection Regulation (GDPR) among others, but the framework is not in itself focused on these.