Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning is booming in medicine, but also facing a credibility crisis

statnews.com

Post reply on HN
Print view View original

71–80 of 100 posts

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#71

Post by antipaul »

debarshri wrote: post #4
dkrysl wrote: post #3

I've been at a few startups and ML has typically meant shoddy rules-based logic or some out of the box model. Only one company applied it with rigor and even then it was a slog - lackluster results, tinkering with different models, poring through research papers to figure out where the cognitive gap came from. The rest of the company thought we were brilliant as did prospective clients. Funny thing is it's possible (…

Shoddy rules-based logic is ok, as long as it created 10x value. When you masquerade it as AI/ML, you are doing disservice to your own company and clients. My question is very simple, at what point Shoddy rules-based engine becomes "AI-driven approach"?

When you are making the PowerPoint

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#72

Post by redsymbol »

bayesian_horse wrote: post #12

I would really like to get a job in medical ML, but without a related degree it seems impossible. I'm "only" a Veterinarian and I don't even get interviews...

Reading some of your replies, you are good at coming up with reasons you cannot be successful.

A friendly suggestion: this attitude may be holding you back more than you realized before now.

What to do instead, is repeatedly ask yourself: "how can I do this?"

Apply this to many specific areas:

- How can I change my CV so it comes across differently?

- How can I begin a small startup without money or other resources?

- How can I move towards my goal without any new degree?

- How is my training in veterinary medicine an asset?

- How can I do medical ML for veterinary medicine?

- Et cetera...

You have a choice now to dig into your heels and say to yourself reasons why none of what I say above will work, and why your situation is truly and utterly hopeless.

OR you can say different things to yourself, maybe get a different result.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#73

Post by spoonjim »

buildingmateri wrote: post #31
spoonjim wrote: post #20

> By far the biggest problem — and the trickiest to solve — points to machine learning’s Catch-22: There are few large, diverse data sets to train and validate a new tool on, and many of those that do exist are kept confidential for legal or business reasons. This is why China will win the AI Age.

Pretty doubtful. The top innovating nations are such because they attract the top talent from around the world, and almost no talent at all moves to China. There's a lot of breakthroughs in AI left and whoever gets the most talent is going to have the advantage.

Norvig’s article “The Unreasonable Effectiveness of Data” suggests that it will be those with the most data who defeat those with the most talent.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#74

Post by zepto »

PeterisP wrote: post #70
BiteCode_dev wrote: post #10

Unless the AI can talk tot the patient to get some context, it's going to be taking decisions with only very partial information, no matter how good it is. Still, I'm thinking that as it improves, it's going to show that doctors are not that good at their job on average, and that's going to be fun to watch.

Radiologists aren't talking to the patient either and don't have earlier scans available, they "just" comment on the image; so the doctor who does have the patient, their history and the radiology report(s) might as well receive the report from a ML system instead of a human radiologist.

That’s how it is in the US, and it’s terrible.

In other places, e.g. Mexico, India, the radiologist can very much be talking to the patient, and can have information from earlier scans.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#75

Post by TylerLives »

randomopining wrote: post #68
spoonjim wrote: post #20

> By far the biggest problem — and the trickiest to solve — points to machine learning’s Catch-22: There are few large, diverse data sets to train and validate a new tool on, and many of those that do exist are kept confidential for legal or business reasons. This is why China will win the AI Age.

So China's (1.4 bil and dropping) smart people vs the rest of the world's smart people (6-7 bil). And China doesn't get easy stepping stones when it wants by stealing secrets from western companies, uni's, and the us military. Lets go.

It doesn't seem to me like the rest of the world is united against China. Even the US companies and universities are very friendly with them.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#76

Post by randomopining »

TylerLives wrote: post #75
randomopining wrote: post #68

Earlier quoted context omitted.

So China's (1.4 bil and dropping) smart people vs the rest of the world's smart people (6-7 bil). And China doesn't get easy stepping stones when it wants by stealing secrets from western companies, uni's, and the us military. Lets go.

It doesn't seem to me like the rest of the world is united against China. Even the US companies and universities are very friendly with them.

Yeah because of their greed. CCP was smart and used our greed against us.

Universities get these students to pay full tuition. Professors get graduate workhorses for research. Companies get cheap labor, low environmental standards, etc so they get a larger profit margin on their widget.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#77

Post by dekhn »

I was a moderately successful scientist who reorganized his career around applying machine learning to medicine. After working in the field for a few years, I noticed that all the folks getting attention for their papers were basically publishing bullcrap. I have a high bar for publishing, I don't publish stuff that is bullcrap. But my competitors were, and they had a serious advantage over me (because most people couldn't recognize the bullcrap).

ML folks today are under immense pressure to show small-digit improvements over existing methods and they're using all the wrong techniques (p-hacking, massive hyperparamter runs, press releases that overstate the impact). It's really sad to see all the snake oil.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#78

Post by FL33TW00D »

No wonder it’s facing a credibility crisis. Almost every single Machine Learning for Neuroimaging paper I see is reporting 0.9+ AUC whilst using With the rampant lack of statistical rigour it blows my mind they get published. This great paper shows that reported accuracy plummets as dataset size increases: https://hal.inria.fr/hal-01545002/document

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#79

Post by Eridrus »

klodolph wrote: post #18

> By far the biggest problem — and the trickiest to solve — points to machine learning’s Catch-22: There are few large, diverse data sets to train and validate a new tool on, and many of those that do exist are kept confidential for legal or business reasons. This is exactly my sense of the problem. Not saying that this is the only problem, just that this seems like the biggest immediate problem. There are a bunch of…

I came away from this convinced that somebody needs to build a data sharing system for medical images.

If there was a system for sharing images that had existed before Covid and enough data had been contributed that reviewers could demand evaluation on some predefined test sets, then a lot of these inconsistent evaluation issues could be weeded out.

The article mentions federated learning, but I feel like that's solving a different problem than reliable evaluation.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#80

Post by Der_Einzige »

TylerLives wrote: post #75
randomopining wrote: post #68

Earlier quoted context omitted.

So China's (1.4 bil and dropping) smart people vs the rest of the world's smart people (6-7 bil). And China doesn't get easy stepping stones when it wants by stealing secrets from western companies, uni's, and the us military. Lets go.

It doesn't seem to me like the rest of the world is united against China. Even the US companies and universities are very friendly with them.

Most of the world is united against china. Every single neighbor of china hates them and hates them far more than they hate other foreigners. Vietnamese government is extremely friendly to america now because of the shaded rivalry with china.

China does have a sphere of influence but the total population of china + friends is well below 3 billion people and will continue to drop as a percentage of the world population.

Post reply on HN