Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clinical failure rates over the decades: yikes

science.org

Post reply on HN
Print view View original

81–90 of 146 posts

Re: Clinical failure rates over the decades: yikes

#81

Post by ozim »

rwmj wrote: post #75
mbnielsen wrote: post #70

Many comments so far seem to try to handwave away the 90 % failure as somehow "optimal" in the system, which seems absurd to me. It is clearly not advantageous for individual companies to keep a drug candidate alive long enough for it to fail in stage III or IV. One obvious question is why they don't and it is very, very tempting to speculate that it's because the problems are getting harder, we are targeting novel m…

Sure, but Pharma A is full of career managers who never kill drugs under development and Upstart B relentlessly culls drugs that don't work early on. Upstart B's failure rate at the final stages is under 50% so they develop 5 times as many drugs, beating the existing company, and indeed all other drug companies. Since this isn't happening, it seems this might not be the explanation.

For me it seems this simplification falls flat right away when you take into account budget constraints.

It is not software development where you can start a project every month see how it goes and drop it or pivot.

I guess they use a lot of computer aided models before they even start serious parts but I believe this discussion is not about failure rates on that stage because then it would be 99%

Re: Clinical failure rates over the decades: yikes

#82

Post by amelius »

perpetuallunch wrote: post #2

This is completely unsurprising, and this: But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground Is an utterly irrelevant comparison. For physical thing we have engineerin…

> For physical thing we have engineering, and the practical application of the trades and craft, trial and error.

> For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.

You could say the same about deep learning. Yet we see improvements every day.

Re: Clinical failure rates over the decades: yikes

#83

Post by mft_ »

fabian2k wrote: post #65
pinkmuffinere wrote: post #56

> But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground - and if you only found that out after spending all the R&D money to build them at full size and trying to fly them…

A large part of that failure rate is in phase 3. Your comparison with prototypes would be more like drug development before even phase 1. Phase 3 is enormously expensive, much more than the earlier parts. And what's even worse, you don't get to learn all that much from failures in drug development in many cases. You can't just fix the problem and try again, you essentially have to try a completely new molecule.

> A large part of that failure rate is in phase 3.

I must ask for some data to support this statement, and also your definition of "a large part".

Re: Clinical failure rates over the decades: yikes

#84

Post by braiamp »

I had to look what "clinical failure rate" means and I think I got the answer in this paper [1] and... I'm not going to say that it is worrisome. Unlike other industries, we can't accurately model in the pre-prototype phase how something will behave in living beings. And to use the author examples, cars and airliners are big and way simpler to model than the complex pathways that you find in biology.

We are chipping away at that problem but it's not like we have it mostly solved as other engineering areas of knowledge. I would argue that due that phase I successes isn't the benchmark, but phase II success should be the actual measure. I'm sure that if someone charts the accumulative success rate for each phase of clinical trials, you will see that phase I is the most brutal one.

1: https://www.nature.com/articles/nrd.2016.136

Re: Clinical failure rates over the decades: yikes

#85

Post by mft_ »

mbnielsen wrote: post #70

Many comments so far seem to try to handwave away the 90 % failure as somehow "optimal" in the system, which seems absurd to me. It is clearly not advantageous for individual companies to keep a drug candidate alive long enough for it to fail in stage III or IV. One obvious question is why they don't and it is very, very tempting to speculate that it's because the problems are getting harder, we are targeting novel m…

> it is generally not a career advancement move for a project manager to kill the drug candidate they oversee.

You're grossly oversimplifying the process. The decision to "kill" a drug is huge, especially if it's already in the clinic (per the article). That decision will be taken by a large group of people, not an individual - and certainly not a "project manager".

Re: Clinical failure rates over the decades: yikes

#86

Post by officialchicken »

braiamp wrote: post #84

I had to look what "clinical failure rate" means and I think I got the answer in this paper [1] and... I'm not going to say that it is worrisome. Unlike other industries, we can't accurately model in the pre-prototype phase how something will behave in living beings. And to use the author examples, cars and airliners are big and way simpler to model than the complex pathways that you find in biology. We are chipping…

> I had to look what "clinical failure rate" means

That should have been the end if the comment; the follow-up by demonstrating such expertise just has me laughing.

Re: Clinical failure rates over the decades: yikes

#87

Post by lolc »

pinkmuffinere wrote: post #56

> But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground - and if you only found that out after spending all the R&D money to build them at full size and trying to fly them…

Yeah why is 10% success bad? What is the tradeoff between spent effort and missed cures if we try to tweak the success rate by killing prototypes earlier in the pipeline? While reading I was expecting a reasoned argument... that was stubbornly not coming around.

Re: Clinical failure rates over the decades: yikes

#88

Post by [deleted] »

officialchicken wrote: post #86
braiamp wrote: post #84

I had to look what "clinical failure rate" means and I think I got the answer in this paper [1] and... I'm not going to say that it is worrisome. Unlike other industries, we can't accurately model in the pre-prototype phase how something will behave in living beings. And to use the author examples, cars and airliners are big and way simpler to model than the complex pathways that you find in biology. We are chipping…

> I had to look what "clinical failure rate" means That should have been the end if the comment; the follow-up by demonstrating such expertise just has me laughing.

[deleted]

Re: Clinical failure rates over the decades: yikes

#89

Post by braiamp »

lolc wrote: post #87
pinkmuffinere wrote: post #56

> But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground - and if you only found that out after spending all the R&D money to build them at full size and trying to fly them…

Yeah why is 10% success bad? What is the tradeoff between spent effort and missed cures if we try to tweak the success rate by killing prototypes earlier in the pipeline? While reading I was expecting a reasoned argument... that was stubbornly not coming around.

It's mostly that we have too many false findings pre-clinical trials, most drug targets validated in models that don't actually hold up in humans, so a huge share of the 90% failure is money and years spent testing candidates that were never going to work. That inflates the number of potential candidates that are likely wrong, due how we select them.

There are ways used right now that are working towards reducing those false findings by looking at actual humans, their biomarkers and whenever or not there's an associated molecule to the condition that we would like to pass onto others. It will not kill prototypes, instead it will discourage us from going through a prototype at all by going for better candidates instead.

Re: Clinical failure rates over the decades: yikes

#90

Post by estearum »

perpetuallunch wrote: post #17
estearum wrote: post #13

Earlier quoted context omitted.

> For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that. Why do you say that? What's the evidence? We continue to have virtually no clue how to make drugs, per TFA.

Targeted gene therapy, cancer survival rates, trauma care, the advancements in hip and knee replacement, all sorts of surgery, HIV is now a non-issue with the right care. The list goes on. Drug development is a hard problem because the solution space is poorly constrained: biochemistry is complex and messy, expecting one chemical substance to have narrow positive effects is probably hopeless.

...?

So the claim is that we're out of the wild west because there's a set of things we've made advancements on, despite drug development getting harder and harder?

Post reply on HN