ICER Debuts Clinical Trial Diversity Assessment Framework

New medical product sponsors should expect to have the diversity of their clinical trials scrutinized as part of the US Institute for Clinical and Economic Review's clinical and cost-effectiveness evaluations using a new in-house tool.

All new ICER reviews now will include an analysis of clinical trial diversity based upon a framework created by the health technology assessment (HTA) body. The assessment will rate the diversity of pivotal trials based on race/ethnicity, sex and age as good, fair or poor relative to disease-specific prevalence estimates.

However, the assessments will not factor into ICER’s cost-effectiveness conclusions, VP-Research Foluso Agboola said in a recent interview with Medtech Insight sister publication the Pink Sheet.

“The goal here is to be able to answer the question of clinical trial diversity," because lack of diversity is reflective of other issues in the health care system, such as health disparities and lack of access to trial sites, Agboola said, adding  that “lumping that to the cost-effectiveness piece sort of blurs” the diversity issues.

No Consistent Methods For Measuring Diversity

As an HTA organization, ICER often finds itself trying to answer questions not only about clinical and cost effectiveness, but also about ethical and contextual issues related to how new treatments and diagnostics are developed, Agboola said in explaining why ICER became interested in assessing clinical trial diversity.

Despite the growing attention being paid by the US Food and Drug Administration and other regulators, as well as sponsors and patient groups, to the issue of adequate representation in clinical trials, the methods used to evaluate study diversity have been inconsistent, according to ICER.

“There is no agreed benchmark in terms of how we're doing and how we're improving, or maybe we're not at all,” Agboola said. “And so this was some of the rationale behind why we thought it would be important that even if we want to do this as part of ICER evaluation, how can we consistently do this across every review?”

ICER developed the Clinical trial Diversity Rating (CDR) framework to provide an objective and transparent approach to evaluating trial diversity. Agboola was lead author of a 22 February article in the Journal of Clinical Epidemiology that described the framework’s development and testing. 

ICER VP-Research Foluso Agboola ICER

The first step in development involved conducting a scoping review of studies that evaluated trial diversity over the past five years, which was intended to address questions about the characteristics used to define diversity, the benchmarks used to evaluate diversity, and what is considered adequate representation, the article states.

ICER then conducted a cross-sectional study to evaluate the diversity of 208 clinical trials that informed its assessments conducted from 2017 to 2021.

“The cross-sectional study allowed us to test the approaches we identified in the literature,” the article states. “Lessons learned from the scoping review and the cross-sectional study and feedback from the views of an advisory group of diverse international experts were incorporated into the development of the tool framework.”

Disease-Specific Prevalence Estimates

The CDR framework was designed to use disease-specific prevalence estimates, which align with the diversity expectations set by the FDA, ICER said.

In an April 2022 draft guidance on race and ethnicity diversity plans for clinical trials, the FDA said study enrollment should reflect the epidemiology of the disease across racial and ethnic populations. If limited prevalence data make it challenging to set enrollment goals based on disease epidemiology, sponsors are encouraged to leverage various data sources, including published literature and real-world data.  (Also see "FDA Draft Guidance Takes Aim At Racial, Ethnic Disparities In Clinical Trials" - Medtech Insight, 13 Apr, 2022.)

The 2022 draft guidance does not provide specific recommendations for setting enrollment goals by age and sex. However, the diversity action plans that eventually will be required under the 2022 Food and Drug Omnibus Reform Act should include enrollment goals broken down by age group, sex and potentially other demographic characteristics, in addition to race and ethnicity.  (Also see "FDA Details Preferred Approach To Collecting Diversity Data" - Medtech Insight, 9 Feb, 2024.)

The FDA missed the December 2023 deadline to issue new draft guidance on the diversity action plans required under FDORA. However, that guidance is currently undergoing review by the Office of Management and Budget.

Experts highlighted the importance of using current prevalence or disease incidence data by subgroup to set enrollment goals, rather than relying on subgroup representation in prior trials, during a recent meeting convened by the agency and Clinical Trials Transformation Initiative. 

Although the FDA has not established specific cut-points for representation in studies, “one of the things that is clear from all of those guidelines is that there is a big focus on the epidemiology of the disease,” Agboola said. If 20% of the patient population with the disease is Black and there are no Black patients in the trial, “you're not really thinking about the epidemiology of the disease.”

“Based on what we've seen in the FDA, we use the epidemiology of the disease as the backbone of this tool, and so we structured the tool in a way that you are evaluating the participation relative to the prevalence of the condition,” Agboola said. Whatever the FDA comes out with in terms of new guidance, “our tool will be a good one that can be used complementarily to evaluate if [sponsors are] aligning therefore well with what FDA is expecting.”

The framework assigns a “representation score” between 0 and 3 for each demographic category based on Participation-to-Disease-prevalence Representation Ratio (PDRR) thresholds. Based on previous studies, a PDRR cut-off of ≥0.8 was defined as the threshold for adequate representation. The CDR framework assigns a rating of “good,” “fair” or “poor” for each demographic characteristic evaluated based on the cumulative score in each category.

The story continues after the table ….

For multinational trials, ICER’s plan is to evaluate racial and ethnic diversity based only on the subpopulation of patients enrolled from the US.

COPD Review Is First Test Case

ICER’s 10 April draft evidence report on Verona Pharma plc’s ensifentrine for maintenance treatment chronic obstructive pulmonary disease reflects the first application of the CDR framework to an ongoing review.

The new drug application, which has a 26 June user fee goal date, is based on two Phase III, placebo-controlled trials (ENHANCE-1 and -2).

Since these were multinational trials, ICER requested information on the subpopulation of US patients for its evaluation of racial and ethnic diversity. Verona did not provide US-specific enrollment data and, consequently, the trials were rated using the full sample, ICER said.

“The goal now is that for every ICER review, we are going to consistently evaluate the clinical trial diversity and give a rating of either good, fair or poor on clinical trial diversity.” – ICER’s Foluso Agboola

Both trials were rated as “fair” on racial and ethnic diversity, with an adequate representation of White individuals compared to the disease prevalence, but an under-representation of Black or African American individuals. In addition, Asian individuals were under-represented in ENHANCE-2, while Hispanic individuals were under-represented in ENHANCE-1.

ENHANCE-2 got a “good” rating on diversity by sex, but ENHANCE-1 mustered only a “fair” rating due to an under-representation of females.

Older adults were under-represented in both trials (50% of trial participants versus 80% of patients with COPD), which resulted in a “fair” rating for diversity by age.

The draft report includes tables with COPD prevalence estimates broken down by race, ethnicity, sex and age, as well as the PDRRs and scores for each demographic category in both studies.

“The goal now is that for every ICER review, we are going to consistently evaluate the clinical trial diversity and give a rating of either good, fair or poor on clinical trial diversity,” Agboola said. “All of our ongoing reviews right now, we're using this tool.”

Broader Uses

The CDR provides a framework for other demographic characteristics, such as socio-economic factors, that could be evaluated to assess whether representation in a given trial was adequate, Agboola said.

The framework could be used by other HTAs, Agboola said, adding, “We are the first HTA body that will be doing this.”

The framework also could be employed by clinical trial regulators, policymakers, journal editors and individual researchers to enhance transparency and accountability on clinical trial diversity, ultimately promoting equity in such studies, the journal article states.

“It can be broadly applied to assess the demographic diversity of individual trials, assess change over time, or guide discussion on specific areas of under-representation in the US and other countries,” according to the article.

While the CDR provides a way of quantifying and evaluating clinical trial diversity by demographic categories, this is not the end goal of the many ongoing efforts to improve representation in clinical studies, Agboola said.

“All stakeholders bear the responsibility of creating innovative solutions that are within their power to drive change on a broader level when evaluation shows poor representation,” the journal article states.