Medtech Monthly, Ep. 3: Breakthrough Device Analysis with Bradley Merrill Thompson

Thompson: Well I have to be honest with you, it'll probably sound like I was biased during the analysis that I did. But my initial reaction to the program was it was a negative one. It was it was negative for a couple of reasons.

First, as I read through the materials that describe the program, there was really no commitment by FDA as to what they would do in exchange for a company applying for and being admitted to the program. They didn't commit to how much faster the review process would be, they didn't commit to making sure that the company would only be expected to meet the least burdensome evidentiary obligations. There just really wasn't much substance there in the way of FDA committing to do something for the company.

But the other thing that bothered me, quite honestly, is that the program sort of positioned FDA as this almost consultant who would come in and work sort of side by side with the company in order to move the development process forward. And that just runs counter to all of my experience with FDA. And I'm not being at all, I hope, I mean, in saying that, we all play the role we're responsible for playing, and FDA plays a very important role, their role is protecting the American citizens.  They are the government; they are there to make sure that unsafe or ineffective products don't reach the market. And that's a very different role than sitting on the other side of the table where your job is to try to figure out how to get the best possible safe and effective devices to market in the quickest amount of time. And so while those two missions overlap, they're not identical.

And FDA can be structurally far more conservative in its decision making than someone else would be so. So to have the FDA sort of hyper-involved in the development process with that sort of acknowledged mindset that was concerning to me.

So I'll be honest with you, I had a lot of clients who came in my door and said, you know, we want to be part of this breakthrough program. And I talked him out of it. I suggested to them that, that the benefits were  dubious. But all that said, I hope I kept an open mind when I did the data analysis. And I tried to do it in a transparent way so that if I have a bias, the reader can can figure out maybe where I went wrong. So I tried to show my math, as it were, to explain how I did it.

Thompson: Sure. So I'm used to sitting across the table with FDA, where we're talking about, for example, the evidence that will be required for a submission, whether it's a 510(k) or de novo, or whatever it might be, and we might come in there with a proposal as to the kind of evidence that we want and FDA inevitably takes the view that that they want more than that they want.

It's natural, as I said, it's part of their job. But they'll say, you know, we think the clinical trial is, is really underpowered, we'd like to see a lot more subjects in it. We'd like to see a more complex design where we've got additional controls that provide us even greater assurance of safety and effectiveness. They'll just sort of pile on or add on to the evidence that will be ultimately required in order to get through the agency.

So I've made it a habit, honestly have tried to not involve FDA unless it's absolutely necessary, because I know that I'll get that conservative advice. And I think if you talk to consultants across the industry, and I've talked to several, they will say, if you make a good-faith determination of what the clinical trial ought to look like, and it's scientifically justified, and you do all your homework and you go into present it, inevitably, you can get something through the agency review process with less data than if you went in right from the get-go and just said to FDA, “Well, what do you want? What do you want us to do?”

It's cheap and easy for them really specify a very burdensome set of evidentiary requirements. There's an art to getting something through the agency, and the agency isn't there to help you identify that quickest pathway?

Thompson: I share that concern. And there really wasn't enough data in what was released for me to analyze what might be the root cause. But when you think about it, there's sort of two different phases here, there's the phase of pre-submission development, where the company is working to develop the evidence, whether it's clinical trial data, or bench trial, or whatever it might be, in order to prove that its product works well enough and is safe and effective enough to be cleared. And then there's the submission process itself, where you've written the submission, you've sent it to FDA, and they're going through their review cycle. Somewhere along the line, a lot of products are not making it through.

And we don't have the data to know where that problem is, whether it's in the pre-submission phase or the submission review phase. But the 54 out of 693 is really appallingly low. Because another graph that FDA provides is a timing graph that tells you in what fiscal years these submissions came in. And they've been coming in since 2015. But they really been coming in in earnest since about 2019. And 2019, there were 110 submissions. If you think about it, that's three years ago. This is early September, and the end of September will be the end of fiscal year 2022.

So at the end of three years ago, there were a whole bunch of these submissions in the queue in the breakthrough program, and still only 54 at this juncture have gotten through it so. So they're dying, or they're taking incredibly long. But I think it’s more likely they're dying. Maybe it’s because the technology maybe doesn't prove to work – although typically by the time you got to FDA, you've kind of proven that the technology works reasonably well.

Maybe the business case doesn't pan out, maybe investors don't keep the money flowing. But more typically, it's that you work with FDA, and you find out that their expectations are beyond what you think is reasonable or beyond what your technology will, will allow. And so that's really discouraging. And again, I've talked to my colleagues in the industry – it's a close-knit community. And the prevailing view is a lot of products, just struggle through the program, where the same products, not through the program, could sail through the FDA [approval] process. It's definitely a scary thing.

 Thompson: Well, again, we really don't have any data, we have the high-level data that only 54 products have succeeded out of 693. We're sort of left to guess what the cause is, but a reasonable person would look at that and say there's a problem.

Thompson: There are differences across those pathways. One of the things that my blog post kind of focused on was the different review times between 510(k) and de novo, for example. And what I found was when I compared the review times of the 510(k) breakthrough devices with similar non breakthrough devices – and when I say the word similar, I'm allowed by the data to look at product codes, so I can compare a breakthrough device and a given product code with other devices in that same product code, so they have the same general technology and intended use. And when I look at the review time, it's basically dead even. The breakthrough program review times are essentially identical to the review times for non-breakthrough products. And that's not very encouraging. Because you go through all this breakthrough process and you expect an improvement, FDA promises an improvement, and there isn't an improvement.

But in de novo, that experience is a little bit different. With the fresh data that FDA released about a month ago for the third quarter of the fiscal year, I calculated the improvement in review time for de novo products at about 64 days over the average for all de novo products. So I can't really do it on an apples-to-apples basis, because basically, de novo is our first of a kind. So by definition, there aren't other products out there to which you can compare them. So I just compare it to the average of all first of a kind. And in that respect, it shows an improvement.

But part of me thinks that that improvement isn't so much based on a quicker review, quicker work ethic effort, but rather  the fact that maybe the agency spent enough time working with a company in the pre-submission phase that they had gotten sorted up the educational curve, had learned about the product, had learned about the studies and so forth, so that naturally the review could proceed a little bit quicker.

The only reason I'm focusing on review time is because it's the only thing I have data on the bigger picture. And the thing that we don't have data on is the development time. And that's where I'm scared that that the breakthrough program actually suffers you actually takes you longer to get through the pre submission phase for a breakthrough rather than a non–breakthrough.

Thompson: So what I did is FDA presented on its website, the breakdown of the different clinical types of devices that entered the breakthrough designation program. And they have, for example, cardiovascular 163, as the most popular clinical area,  neurology as the second most popular with 126, and so forth. And I thought, well, that's the that's what goes into the funnel, what comes out of the funnel.

Thompson: So I did the exact same analysis of the 54. Actually, I only did 52, because I focused on [the Center for Devices and Radiological Health], and two were [Center for Drug Evaluation and Research] devices. I took the I took the CDRH devices, and I created the exact same bar chart. And it showed that neurology was actually the most successful those products coming out of the pipeline came out at a higher rate and cardiovascular was second.

So cardiovascular from 163 entered the breakthrough program to eight have exited the breakthrough program and have actually been authorized by FDA. Well, that's a huge differential – 163 entering the program, a grand total of eight being authorized out of the program. And there's a couple like that. Urology didn't fare as well as it should have based on the number that were going into the program. And at the other end of the spectrum pathology devices seem to do a little bit better in getting authorization based on the on the numbers going in versus the numbers coming out.

So different areas have different success rates. I would note that the cardiovascular products were by and large PMA devices, they were higher risk PMA devices, where the neurology and orthopedics and so forth were either 510(k) or de novo. So they were going different routes, as well. But I find that interesting. That's either encouraging or discouraging depending on what what particular clinical area you're in.

Thompson: One of the things that bothered me is the fact that there is additional data that would be very useful that FDA hasn't released the data I wish they would release, and I'm planting the seed with you. Because you in the media have a much better chance of getting FDA to release data than just a plain old private citizen like me. FDA will respond to a FOIA request from a guy like me about the time an oak tree is growing over my grave. 

It would be very important for FDA to release data on when the first encounter occurs between an applicant and the FDA.  So they have data on when the breakthrough status was requested and when it was granted, and they can easily release that. And then on the non-breakthrough products, they have a very well-developed pre-submission meeting process, a Q-sub process, where folks request a meeting, they go in with their questions, they get their questions answered, and then either they'll ask for another meeting down the road, or though they'll go to a submission or not.

But it's easy to tie those data together. And if we had those data, we could tell amazing insights into the time that it takes for companies to develop the data necessary to submit to FDA. I don't want to overstate it because people consult FDA at different phases, right? Some will consult FDA real early; some will consult FDA late. I get that. But there's a meaningful average. And there's an average across the breakthrough devices. So we could handle the data responsibly and draw only appropriate conclusions from it.

But boy, if you could get FDA to release data on that, that would be huge for  the industry, it'd be huge for frankly, patients and doctors and others who care about the FDA process, because it would really give us insight into how long the development process works. But that's, that's my big area of concern is not knowing how long that development process takes. And whether it's actually substantially longer for breakthrough products than it is for non-breakthrough products.

 Thompson: Well, as I said, I have to confess even before I had the data, I was discouraging people from pursuing the program. I think now that I have the data, I would definitely discourage anyone who has the possibility of filing a 510(k) instead of a de novo, I would discourage them from the program. Because if you can fit in the 510(k) process, you really don't even need that much in the way of consultations with FDA, because it means you have a predicate device, and you can look and see what the what the roadmap looks like for that company. And you can kind of figure out what you need to do and proceed.

In the de novo, it's about a 64-day advantage in terms of the review cycle. That review cycle has to be viewed in the context of the development cycle. And we don't have data on that. So it kind of depends on just how difficult or problematic I think the questions are going to be that need to be answered.

If I think we can construct a reasonable case for how to put it together, I think I'd still try and avoid the breakthrough without more compelling evidence that the breakthrough program actually offers an advantage. But, you know, I would ultimately lay it out there and say, it's very uncertain.

And even within the program, and I tried to make this point a couple of times in the blog post, the experience of companies is really uneven.  Some go through really quickly, some go through really slowly. The blog post really only looks at averages; it doesn't really account for all that variability. But for some companies, the breakthrough program could be absolutely horrible. So I just need to have that conversation with clients. And ultimately, it's their choice as to whether they want to do it.

I will say that some clients are persuaded that there's a marketing advantage to being in the FDA breakthrough program. And that may well justify it.

When I say marketing, really, what I'm talking about is is their ability to raise capital from private equity and others. They think that if they're in the breakthrough program, that's kind of a badge of honor, and that aventure capital folks and others will be more readily interested in investing in a company that makes a breakthrough product. So they really aren't motivated by the FDA benefit, if any, they're motivated by that marketing benefit. And that may or may not be true. I don't have data to suggest whether it is or not.

Thompson: One of the things that I initially reacted to when I first looked at the program was the lack of a commitment. And there are certain things FDA could do to firm up its commitment. On the development side, there's not a lot they can say other than sort of reaffirm the overarching objective of identifying the least burdensome; I would hope they're doing that but it'd be helpful if they said it out loud.

In terms of the review process, in the context of user fees, FDA has made commitments as to average target goals for reviews, they could do something similar for the breakthrough program and say, you know, our average goal instead of 90 days for a 510(k)is 60 days.

Honestly, I don't think they want to do that. I don't think they want to be that fast. I don't think they want to make a commitment. But if they were serious about the program, making some sort of commitment along those lines would be beneficial.

But the thing that I would find even more valuable again, I'm just going to sound like a nerd – and I am – making more data available to the public to evaluate the merits of the program would be hugely advantageous. And you might say, “Well, what, what data would that be?” Well, one, it would be the data I mentioned a moment ago data on first encounters, whether it's an application for a breakthrough, or whether it's a pre-submission meeting on a non-breakthrough.

But the other thing they could do very easily is at the end of the process, give the medical device company a survey for them to fill out completely anonymous, where they couldn't identify who answered the survey, because companies would always be worried; they don't want to anger anyone at FDA with a criticism. But the survey could be things like, do you think that overall, the program moderated the amount of evidence, do you think overall, that the development cycle was shorter, given the FDA interaction? Do you think the data requests were reasonable, or whatever it might be, and then at the end, you know, on an anonymized basis, annually, you know, release the experience of the companies that go through it.

To me, that's hugely helpful, because there you get right from the horse's mouth on an aggregate basis. I would think it'd be helpful to FDA in deciding whether they're doing the program the way they want to do the program. But there's no data collection effort by FDA, and so that's a disappointment. But that would be a very easy way to really assess the program from everyone's standpoint.

And then finally, if the program actually offers an advantage … I think they need to tighten up the criteria for what constitutes a breakthrough device. Right now, the criteria are really subjective. And the fear is that it's anyone who's a good friend of FDA would get admitted, while those who FDA doesn't like so much might not get admitted. And there's really no accountability around that.

The first criteria says it has to be a serious or life-threatening disease, that's relatively objective. But the other part – that it's breakthrough – is a very loosely articulated standard. If it really did amount to a substantial savings, that'd be a big competitive dynamic for a company, and they shouldn't just be handing that out willy-nilly, it really ought to be tightened, if this program is advantageous.

Thompson: If I were CMS, that's how I would react to it.

Thompson: Right. And they really have completely different purpose. When it gets to FDA it’s considered safe and effective, but that doesn't mean that CMS wants to make access to it widely available. There are other factors that come into play.

Thompson: Honestly, I would be speculating. But that's also revealing. I've never heard anyone at FDA say that they're concerned about these things. Now, maybe they are and they just haven't publicly said it. That's possible. But the fact is that they haven't publicly said, “We want to examine these things, and we want to make sure that the program is advantageous.” To me, that means that they're not very motivated to improve the program.

I think it's serving FDA’s purpose fine the way it is, and the only reason they would change it is if they had concerned that companies might not be interested in pursuing it, because it's not advantageous. And I think FDA can sort of sit back a little bit with self-confidence and say, “Well, we're the only game in town. So they either do the breakthrough, they do the regular and we frankly don't care which they do.”

Thompson: No, I just wanted to thank you for inviting me to talk about it. It is an important program, the purpose of it is very important, which is to really minimize the time delay of getting important therapies to patients who are waiting, and that's a common mission between the industry and FDA. We start with that common mission which is a very good thing. I'm just not sure FDA looks at the program through industry's eyes to really evaluate whether it’s s accomplishing that. And I wish they would.

And I'm also kind of excited about the possibility that you're willing to think about requesting that additional data because FDA if they shared data, and when those first encounters occurred, would just give us all sorts of insights into how the development process looks. And that's really, at the end of the day, far more important than the review time itself. We companies spent far more time in development than they do waiting for FDA review. And so getting, getting a handle on that and seeing the trends and how it varies from clinical area to clinical area that will just be immensely helpful.