Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clinical failure rates over the decades: yikes

science.org

Post reply on HN
Print view View original

121–130 of 146 posts

Re: Clinical failure rates over the decades: yikes

#121

Post by preg_match »

perpetuallunch wrote: post #2

This is completely unsurprising, and this: But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground Is an utterly irrelevant comparison. For physical thing we have engineerin…

And, moreover, the human body is much, much more complex than any plane or car. We know how cars work and how planes work. For many parts of the human body, we don't know exactly how they work. We have multiple hypothesis, maybe, but we can't say "oh yes this is how serotonin works in the brain". We might know the small scale interactions, but why do SSRIs work in terms of the entire brain? We don't know. Why and how does anesthesia work? We also don't know.

Re: Clinical failure rates over the decades: yikes

#122

Post by preg_match »

estearum wrote: post #90
perpetuallunch wrote: post #17

Earlier quoted context omitted.

Targeted gene therapy, cancer survival rates, trauma care, the advancements in hip and knee replacement, all sorts of surgery, HIV is now a non-issue with the right care. The list goes on. Drug development is a hard problem because the solution space is poorly constrained: biochemistry is complex and messy, expecting one chemical substance to have narrow positive effects is probably hopeless.

...? So the claim is that we're out of the wild west because there's a set of things we've made advancements on, despite drug development getting harder and harder?

Consider drug development becomes harder because we've gotten so good at it. Basically all the low-hanging fruit is long gone. We have very reliable and effective drugs and vaccines for illnesses which DESTROYED humanity before.

The only reason people are even dying from cancer at such high rates is because we solved other easier illnesses, and people live way longer as a result. People 100 years ago weren't dying to cancer like they are now, because they were already dead from other more common, simpler, diseases.

Something like cancer is extraordinarily complex. It's nothing like a bacteria or a virus. I mean, think about it, we have to devise methods of targeting cancer cells without injuring human cells around them - when the bodies own immune system cannot differentiate them.

And we have devised those methods! Chemotherapy, immunotherapy, radiation therapy. It's truly incredible.

Re: Clinical failure rates over the decades: yikes

#123

Post by db48x »

peyton wrote: post #47
db48x wrote: post #46

Earlier quoted context omitted.

These are examples of ambitious drugs that wouldn’t have been attempted even a few decades earlier. But technology improved and the money was there so they tried anyway, and found success. But for any 3 successes there are still 27 failures because those 27 were equally ambitious.

Feels like a just-so story. How to quantify “ambitious”? If the goal were flight, would jumping off cliffs with wings like birds be “ambitious”? The most parsimonious explanation is either our models or methods (or both) are garbage. Something’s missing. The failure rate is insane, and writing it off as ambition or “biology is hard” rather than digging in does us no favors.

You’ve misunderstood. Nobody here has said that we shouldn’t try to improve the success rate, only that improving the success rate will make people try harder problems which worsens the success rate. If this is true, it just means that any effort you make to improve the success rate really has the effect of improving the quality of our medicines rather than reducing the failure rate.

Keep in mind that we’re talking about the success rate of the last step in the process: human trials. For every human trial there were millions of drug candidates that were considered and rejected. Tens of thousands of those candidates were actually synthesized and tested in animal models or in cell cultures. In the 80s these numbers were all lower. Back then only thousands of candidates would be considered, and only hundreds would be synthesized for actual testing.

Worse, for every candidate that gets to the human trial stage there were multiple projects that rejected all of their candidates and were discontinued. The numbers here are pretty vague, but past surveys have found that the number of failures at this stage is really high too, maybe as high as 90%.

Pharma companies are hardly unaware of this problem. They regularly spend years (even decades) and hundreds of millions of dollars on drug projects that never reach human trials. They spend even more on the ones that reach human trials and then fail. As a result they have spent billions on new methods, new systems, new techniques, etc, all intended to reduce that risk. It appears that all they have managed to do is keep pace with the increasing difficulty of drug development.

I don’t necessarily agree that this is inevitable, but it is understandable.

> The most parsimonious explanation is either our models or methods (or both) are garbage.

I disagree with that. In spite of the difficulties we have managed to get around 50 new drugs approved every year for the last several decades. Sometimes more, sometimes less, of course, but as far as I know there’s been no obvious trend upwards or downwards. That doesn’t seem like garbage to me.

> If the goal were flight, would jumping off cliffs with wings like birds be “ambitious”?

Not any more, no, but it was quite ambitious back in the 1890s. Back then there was a single scholarly study on the lift generated by wings, and it was completely wrong. The Wright brothers built their own wind tunnel and ran their own tests to get reliable data to base the Flyer’s wings on.

Re: Clinical failure rates over the decades: yikes

#124

Post by mmooss »

refurb wrote: post #57
mmooss wrote: post #44

Earlier quoted context omitted.

It's interesting that you frame it as the FDA increasing regulation and not that science has increased its foreknowledge of problems.

It’s a similar problem you see with all safety related decisions - as the large risks are eliminated you become more concerned about the smaller risks. Plenty of successful drugs from the 50’s had hERG activity, yet the impact on safety was marginal. Today plenty of programs are killed over it. Increase safety scrutiny does increase safety, but at a cost.

> Plenty of successful drugs from the 50’s had hERG activity, yet the impact on safety was marginal.

Is there some evidence of these two claims? Obviously many experts think otherwise.

Re: Clinical failure rates over the decades: yikes

#125

Post by EA-3167 »

calf wrote: post #39
EA-3167 wrote: post #9

Earlier quoted context omitted.

How? He’s saying that he can’t think of any good ones, and in my experience that’s true of most informed people. This is one of those hard problems that needs solutions, and so far the present system is the best anyone has managed.

I don't care what he's "saying that" which is just your biased interpretation. His essay does not warrant the final concluding sentence which would require a different essay thesis. It's just basic high school reading and writing skills.

It’s a shame we aren’t a basic high school reading class, that might help this conversation along to the inevitable conclusion that your opinions aren’t interesting without something substantial behind them. They’re still “valid”… but only to you.

Re: Clinical failure rates over the decades: yikes

#126

Post by estearum »

preg_match wrote: post #122
estearum wrote: post #90

Earlier quoted context omitted.

...? So the claim is that we're out of the wild west because there's a set of things we've made advancements on, despite drug development getting harder and harder?

Consider drug development becomes harder because we've gotten so good at it. Basically all the low-hanging fruit is long gone. We have very reliable and effective drugs and vaccines for illnesses which DESTROYED humanity before. The only reason people are even dying from cancer at such high rates is because we solved other easier illnesses, and people live way longer as a result. People 100 years ago weren't dying to…

Sure I agree with all of that. "We've made progress" is not what comes to mind when I read "we're just coming out of the wild west."

Re: Clinical failure rates over the decades: yikes

#127

Post by preg_match »

estearum wrote: post #126
preg_match wrote: post #122

Earlier quoted context omitted.

Consider drug development becomes harder because we've gotten so good at it. Basically all the low-hanging fruit is long gone. We have very reliable and effective drugs and vaccines for illnesses which DESTROYED humanity before. The only reason people are even dying from cancer at such high rates is because we solved other easier illnesses, and people live way longer as a result. People 100 years ago weren't dying to…

Sure I agree with all of that. "We've made progress" is not what comes to mind when I read "we're just coming out of the wild west."

I would agree, I guess it's a matter of perspective. I'd put modern medicine at maybe a 100 year timeline. That's short as compared to, say, industrialization. Which is then short as compared to, say, agriculture, at 10,000 years. And the shortest of all is computerization.

Re: Clinical failure rates over the decades: yikes

#128

Post by s1artibartfast »

bonsai_spool wrote: post #117
s1artibartfast wrote: post #107

Earlier quoted context omitted.

I dont know why you are being rude about a topic you know nothing about. As some who actually does this for a career, they are largely correct. Dose frequency and side effects especially are major reasons. I would add route of administration and patient convenience as well. Anti-vegf treatments for blindness are a good example if you want to research. They all have basicly the same effect in terms of letters preserve…

Not being rude, and I definitely know about medicine. I explained in sibling that I understood the Lowe article to be about new research (as a researcher, I don't often think about how hard it is to make a GLP survive the stomach but I do think about how incretin biology works). I guess these VEGF agents you list feel like a very straightforward engineering question (can we make an antibody that targets VEGF, humaniz…

I dont think you have an accurate model of how US prescribers and patients operate.

They want the best, and are willing to sepnd 10x the price or switch treatment to secure that 1% survival, extra week of dosing interval, or the ability to inject at home via mail instead of some sweaty infusion clinic.

I used to think like you before transitioning from medical devices to pharmaceutical development.

The USA is the only real Market that matters for drug development. Unlike the with marginal benefit pricing and National negotiation, the US market is consumer Centric, with patients and their providers wanting the best, even if it is marginal. That 1% means that you can either take the market share or 10x the price.

> I guess these VEGF agents you list feel like a very straightforward engineering question

Each of those straightforward engineering questions takes about 100,000 labor-years and a billion dollars to get approval.

Re: Clinical failure rates over the decades: yikes

#129

Post by bibouthegreat »

People get confused about all the types of clinical trials. The worst part happened in 1997 [1] Before this time, you could p-hack your studies. You could get a molecule and just look at some random effect that happens in any case because of sampling issues. Since 1997, you must register the full protocol to the FDA before the study. Since then, almost no study with a full protocol brought a really new medicine.

[1] https://en.wikipedia.org/wiki/Food_and_Drug_Administration_M...

Re: Clinical failure rates over the decades: yikes

#130

Post by tim333 »

>It’s a nerve-wracking way to live, but there’s no alternative in sight.

There's always AI simulation on the way. Deepmind's Isomorphic labs is working on it https://www.isomorphiclabs.com/articles/the-isomorphic-labs-...

And an AI designed drug for idiopathic pulmonary fibrosis from Insilico Medicine is going to clinical trials https://www.artificialintelligence-news.com/news/insilico-me...

Post reply on HN