Benefit-Risk Is Front-And-Center In Latest Revision Of International Risk Management Standard ISO 14971

More regulators worldwide are considering the benefits of flawed medical devices when deciding whether they should remain on the market. In response, the International Organization for Standardization (ISO) has strengthened benefit-risk language in the latest revision of its voluntary risk management standard.

That standard, ISO 14971, instructs device-makers on how to best put together a risk management program; it was originally released in 2000 and underwent its first revision seven years later. The third edition of the document builds directly on ISO 14971:2007. (Also see "Draft Revision Of Risk Management Standard Gives Firms More Guidance" - Medtech Insight, 15 Jan, 2007.)

In the works since 2016, the ISO 14971:20XX Draft International Standard (DIS) was given a thumbs-up by members of ISO and the International Electrotechnical Commission (IEC) in November. The Final Draft International Standard (FDIS) is currently undergoing last-minute edits and is being translated into French by ISO Technical Committee 210, Joint Working Group 1, the ISO/IEC subcommittee that oversees ISO 14971 revisions.

That's according to Jos Van Vroonhoven, convener of TC210/JWG1, and a senior manager for standardization for device giant Philips Healthcare.

The FDIS "is expected to go out for an eight-week final ballot in April, with publication targeted for Q3 2019," Van Vroonhoven confirmed in a Jan. 4 email to Medtech Insight.

In a separate interview, Van Vroonhoven touted the draft standard's bolstered emphasis on benefit-risk, which regulators – including US FDA – are increasingly using to weigh product availability and regulatory compliance issues. (Also see "'A Sea Change': Device Center Compliance Chief Touts US FDA's Benefit-Risk Concepts – But Will Manufacturers Buy In?" - Medtech Insight, 7 Aug, 2017.)

And FDA doubled-down on benefit-risk in the pre-market space last September when it released a final guidance explaining how the agency evaluates benefit and risk questions for 510(k) substantially equivalent products with differing technological characteristics.  (Also see "FDA Guidance Looks At Benefit Vs. Risk In Certain 510(k)s" - Medtech Insight, 24 Sep, 2018.)

Thanks to the revision, ISO 14971 will include, for the first time, a definition of "benefit."

"When revising the standard, we wanted to keep as much of it as possible the same. So, there are no changes to the risk management process, but there is more clarification and alignment with regulatory requirements that have changed," Van Vroonhoven said. "One of those changes by regulators is more emphasis on benefit-risk."

That's why ISO 14971 will include, for the first time, a definition of "benefit": "a positive impact or desirable outcome of the use of a medical device on the health of an individual, or a positive impact on patient management or public health."

That definition "is derived from MEDDEV guidance in Europe and FDA guidance," Van Vroonhoven explained.

The draft standard goes on, noting that "benefits can include positive impact on clinical outcome, the patient’s quality of life, outcomes related to diagnosis, positive impact from diagnostic devices on clinical outcomes, or public health impact."

ISO 14971:2007 does discuss benefits, but "it's hidden a bit. When you read the standard carefully, you can see that benefits are well covered, but not always obvious," Van Vroonhoven said. That's why the newly revised standard "puts more emphasis on balancing the benefits against the risks, or the risks against the benefits. And of course, the balance must be on the benefits."

The current version of the standard includes a short clause on "Risk/Benefit Analysis," which was renamed "Benefit-Risk Analysis" in ISO 14971:20XX to better align with regulator-speak.

The updated benefit-risk clause is mostly aligned with what is already in ISO 14971:2007, but the draft standard adds that device-makers can modify a product or its intended use if its residual risks outweigh the benefits.

The standard defines "residual risk" as "risk remaining after risk control measures have been implemented."

Many ISO 14971 annexes have been lifted out of the standard and folded into Technical Report 24971.

Additional information on applying benefit-risk, currently found in Annex D of the 2007 standard, has been lifted out of ISO 14971 and folded into TR 24971 as part of the revision. TR 24971 is an ISO Technical Report that offers manufacturers guidance on the risk management standard.

"The current edition of ISO 14971 has many annexes that number more than 60 pages," Van Vroonhoven said. "Several of those annexes have been moved to the Technical Report and have been updated and supplemented with new information."

TR 24971 was first published in 2013; it's being revised concurrently with ISO 14971. (See sidebar story below for more on TR 24971.)

"ISO 14971:20XX provides the possibility for the manufacturer to perform a benefit-risk analysis for those risks that are not judged acceptable using the criteria established in the risk management plan and for which further risk control is not practicable," the revised TR 24971 tells firms in Clause 7.4.1.

"In some instances, risks can be justified if they are outweighed by the expected benefits of using the device," the draft adds. "In general, the benefit-risk analysis should not include purely theoretical risks and benefits, but rather be supported by objective evidence. The benefit-risk evaluation can be done on individual residual risk or on the overall residual risk."

Aside from that type of general information, the updated TR 24971 offers valuable guidance on estimating benefits, determining the criteria for benefit-risk judgements, and comparing benefits and risks. It also throws in a few examples of benefit-risk decisions.

In an interview with Medtech Insight, Don Powers, a member of ISO TC210/JWG1, pointed out that benefit-risk is also featured prominently in Clause 10 of the revised TR 24971. That portion of the guidance is dedicated to production and post-production activities and aligns with Clause 10 in ISO 14971:20XX. (See "ISO 14971:20XX: A Walk-Through" below for more on Clause 10.)

"There is a lot of helpful information in the post-production clause [in TR 24971:20XX] that talks about benefit-risk and talks about monitoring benefits, as well, because if your decision is based on a benefit-risk analysis, then you must make sure the benefits aren’t changing," said Powers, a longtime device industry consultant.

Indeed, draft TR 24971 advises under Subclause 10.2.2: "Regardless of whether the risk assessment indicates that the risk is acceptable or unacceptable, the manufacturer may need to assess whether the probable benefit from using the medical device has changed."

The guidance continues: "If the benefits from using a medical device change while the risk remains the same, the benefit-risk balance will be altered, and the benefit-risk analysis needs to be updated. If the benefit is reduced significantly, then patient expectations based on the intended use may influence risk acceptability."

A benefit-risk balance could change if, the revised TR notes, there are changes in a medical practice, clinical data confirms additional benefits for patients, there is a change in the patient population that's using the device, or other devices are introduced to market that have the same intended use but have different risks or benefits.

TR 24971 goes on to say that device-makers assessing a change in benefits should consider "individually and in the aggregate" the expected benefits and their magnitude, the probability that a patient will experience identified benefits, and the length of time a patient will receive the benefits.

Powers summed up the revision process this way: "When companies look at their overall residual risks, they’re always doing it in comparison to the benefits, regulatory requirements, and so on. The updated ISO 14971 and TR 24971 help manufacturers along the road to doing just that."

Modern-Day Regulatory Emphasis On Risk Management A Motivator For ISO 14971 Redo

Van Vroonhoven told Medtech Insight it was necessary to update ISO 14971 because "people needed more guidance and clarification of the requirements" of the standard. The document also needed a revision, he noted, "in view of changing regulations that have become stricter with regard to overall risk management."

Indeed, there is a much stronger emphasis on risk management by regulators and device firms in 2019 than when ISO 14971 first appeared on the scene nearly two decades ago.

To wit: FDA's Quality System Regulation – written in the early-to-mid-1990s – makes only a passing mention of risk analysis despite the agency considering the overall concept to be an inherent part of the decision process throughout the regulation.

But an ongoing FDA plan to harmonize its rule with international standard SO 13485 means risk management will most certainly play a much bigger role in a revamped QSR. (Also see "QSR Author Kim Trautman Predicts What A Mash-Up Of FDA's Quality System Regulation And ISO 13485 Might Look Like" - Medtech Insight, 15 Aug, 2018.)

That's because risk management is required by ISO 13485:2016, which is used to ensure quality systems compliance with regulators in a variety of countries, including Canada, Japan, Australia and the 28 member states of the European Union. For guidance on how to perform risk management, ISO 13485 points device-makers to ISO 14971.

Currently, FDA cannot require manufacturers to implement ISO 14971, although the agency strongly endorses the standard's risk management guidelines.

And in the EU, the new Medical Device and IVD Regulations – which go into effect in May 2020 and May 2022, respectively – address the importance of having a well-oiled risk management process. (Check out Medtech Insight's Interactive Timeline to stay abreast of global regulatory deadlines.)

"We see a lot more security risks now than, say, 10 or 15 years ago, because when you hook up your medical device to the internet, it needs to be secure," TC210/JWG1 convener Jos Van Vroonhoven says.

Van Vroonhoven stressed that no requirements were removed from ISO 14971 in the redo. Rather, the revision "clarifies existing requirements and adds a few more requirements where we found that to be necessary," he said. "The standard is enhanced by clarifying all of the steps firms should take and offers better alignment in the wording, with an eye on regulatory requirements. And that, we expect, will facilitate recognition by regulators."

He also emphasized that ISO 14971 can be used to assess any type of device-related risk. The standard defines "risk" as the severity of harm that could come to a patient or user of a device and the probability of that harm occurring. Hazards can occur due to user error, environmental conditions or problems with the device itself – just to name a few things that could go wrong.

"Risk can be use-related or user-related. It can also be data- and system security-related," Van Vroonhoven said. "We see a lot more security risks now than, say, 10 or 15 years ago, because when you hook up your medical device to the internet, it needs to be secure." (The revised TR 24971:20XX includes guidance on cybersecurity in Annex F; see sidebar story above.)

But while "the process provided by ISO 14971 can be used for any kind of risk, that does not preclude that you may need to use some additional standards for specific risks or specific solutions," he added.

Van Vroonhoven pointed to IEC 62366:2015, which focuses on applying usability to devices. That international standard shows firms how risk management and usability work hand-in-hand; it refers often to ISO 14971.

"Another example would be the IEC 60601 series of standards for medical electrical equipment, on how to deal with electrical risks and mechanical risks – especially for moving parts," he said.

ISO 14971:20XX: A Walk-Through

Van Vroonhoven walked Medtech Insight through the revised ISO 14971, pointing out specific sections of the revised standard that device-makers should keep an eye on.

Clause 3, "Terms and Definitions." "Benefit" isn't the only important new addition to the definitions section of ISO 14971:20XX. Also notable is the definition of "reasonably foreseeable misuse," which is "use of a product or system in a way not intended by the manufacturer, but which can result from readily predictable human behavior."

The revised standard explains that "readily predictable human behavior includes the behavior of all types of users, e.g. lay and professional users," and notes that "reasonably foreseeable misuse can be intentional or unintentional."

"'Reasonably foreseeable misuse' is a term that is specifically developed for risk management purposes," Van Vroonhoven said. "It is different than 'use error,' which is a term used in usability engineering.

"But use error as it is used for usability engineering is different than reasonably foreseeable misuse," he continued. "Use errors are mistakes that can naturally happen, but [reasonably foreseeable misuse] covers the intentional use of a device for other purposes."

The revised ISO 14971 defines a "use error" as a "user action or lack of user action while using the medical device that leads to a different result than that intended by the manufacturer or expected by the user."

"Risk control measures can be inside a user interface, but they can also be outside a user interface – and that is not covered by the usability engineering process," Van Vroonhoven said. "And that’s why we have a different term: 'reasonably foreseeable misuse.'"

"Reasonably foreseeable misuse" is mentioned in ISO 14971:2007, but only three times. The term is much more prevalent in the revised standard.

A third addition to the terms and definitions clause in ISO 14971:20XX is "state of the art." The draft defines "state of the art" as "developed stage of technical capability at a given time as regards products, processes and services, based on the relevant consolidated findings of science, technology and experience."

"We copied the definition of 'state of the art' from ISO/IEC Guide 2:2004," Van Vroonhoven said. "It’s not really a big deal" – but firms should still be aware that the new term is there.

Clause 3 in ISO 14971:20XX maps to Clause 2 in the current 2007 version of the standard.

Clause 5, "Risk Analysis." Van Vroonhoven said this section, which maps to Clause 4 in ISO 14971:2007, was revised to give it "a more logical order."

Clause 5.1 describes the general risk management process and notes that device-makers must record risk management results in a risk management file, among other directions.

"We recognize that you need to consider intended use at the start of the risk management process," Van Vroonhoven says.

And Clause 5.2 goes into some detail about intended use and reasonably foreseeable misuse. It says "the intended use should take into account information such as the intended medical indication, patient population, part of the body or type of tissue interacted with, user profile, use environment, and operating principle."

That language "is aligned with usability engineering standard IEC 62366," Van Vroonhoven said. "We recognize that you need to consider intended use at the start of the risk management process. And we also recognize that when you are developing a medical device and doing your risk management, you may decide to change the intended use; for example, to exclude pediatric applications."

Meanwhile, Clause 5.3 "talks about the identification of characteristics related to safety, while Clause 5.4 focuses on the identification of hazards and hazardous situations," he said. "And Clause 5.5 is all about risk estimation.

"So, we moved some text around compared to the current edition of the standard, all with the intention of clarifying the risk analysis process – to make clear which steps need to be taken."

Clause 8, "Evaluation of Overall Residual Risk." "The evaluation of overall residual risk was not expressed very clearly in the current edition of the standard," Van Vroonhoven said.

Clause 8, which maps to Clause 7 in the 2007 standard, "says the manufacturer must evaluate the overall residual risk. When the residual risk is unacceptable, you can do a benefit-risk analysis on the overall residual risk," he explained.

TR 24971:20XX goes further when guiding device-makers on Clause 8, noting that there is no preferred way for firms to evaluate residual risk, and that manufacturers are responsible for establishing their own criteria for risk acceptability.

"A few small risks can be acceptable. But if you have too many of those small risks, then you're at risk of dying by a thousand cuts," Van Vroonhoven says.

"That criteria and any associated methods must be included in the risk management plan," Van Vroonhoven noted. "It is an addition to the risk management plan that you not only define the criteria for acceptability of individual risks, but also for the overall residual risk, and how you evaluate the overall residual risk, with all the contributions of all individual risks together."

He warns device-makers, though, that accepting too many "small risks" could cause one big risk.

"A few small risks can be acceptable," Van Vroonhoven said. "But if you have too many of those small risks, then you're at risk of dying by a thousand cuts."

Clause 10, "Production and Post-Production Activities." This clause takes one large section in ISO 14971:2007 – Clause 9, "Production and Post-Production Information" – and restructures it into three bite-sized sections.

"We inserted Subclauses 10.1 ["Information Collection"], 10.2 ["Information Review"] and 10.3 ["Actions"] to clarify the three steps," Van Vroonhoven said.

"First, you collect information, and we indicate which information needs to be collected. The second step is reviewing that information, where we added a requirement for firms to determine whether the generally acknowledged state of the art has changed," he said.

Indeed, more detail is given in Clause 10.1 on the type of production and post-production information to be collected, including data:

• Generated during production and monitoring of the production process;

• Generated by the operator and/or the user;

• Generated by those accountable for the installation, use and maintenance of the medical device;

• Generated by the supply chain; and

• Related to the generally acknowledged state of the art.

"And finally, we clarified the subclause on actions [10.3], including the need to take considered actions regarding medical devices already on the market. This is a new requirement, as well," Van Vroonhoven said.

"We felt it was necessary to include that because when you have medical devices out on the market and you discover something is wrong, you may consider a recall action, for example," he added. "That’s what’s not explicitly mentioned in the current edition of the standard."

From the editors of The Gray Sheet