Machine learning is booming in medicine, but also facing a credibility crisis
1–10 of 100 posts
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#2This is true for every other domain. When every company calls itself a AI company these days. Therefore value of AI/ML and credibility gets diluted.
It becomes very difficult to classify real orgs using hardcode stats from fake ones. When I hear "we are solving x with AI" or "AI driven", it gives me jitters.
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#3I've been at a few startups and ML has typically meant shoddy rules-based logic or some out of the box model. Only one company applied it with rigor and even then it was a slog - lackluster results, tinkering with different models, poring through research papers to figure out where the cognitive gap came from. The rest of the company thought we were brilliant as did prospective clients. Funny thing is it's possible (though less common) to get paid just as much if not more doing data analysis / engineering than vaunted ML/AI work. Businesses are swimming in data but it's siloed or dirty. I don't really see that and the actual analysis being automated away - too much messiness (human error inputting data into systems like Salesforce, ETL breaks in prod leading to gaps, etc).
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#4I've been at a few startups and ML has typically meant shoddy rules-based logic or some out of the box model. Only one company applied it with rigor and even then it was a slog - lackluster results, tinkering with different models, poring through research papers to figure out where the cognitive gap came from. The rest of the company thought we were brilliant as did prospective clients. Funny thing is it's possible (…
Shoddy rules-based logic is ok, as long as it created 10x value. When you masquerade it as AI/ML, you are doing disservice to your own company and clients.
My question is very simple, at what point Shoddy rules-based engine becomes "AI-driven approach"?
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#5This is true for every other domain. When every company calls itself a AI company these days. Therefore value of AI/ML and credibility gets diluted. It becomes very difficult to classify real orgs using hardcode stats from fake ones. When I hear "we are solving x with AI" or "AI driven", it gives me jitters.
The elephant in the room is that AI does not exist, even in "hardcore" orgs. A transformer model is not AI.
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#6This is true for every other domain. When every company calls itself a AI company these days. Therefore value of AI/ML and credibility gets diluted. It becomes very difficult to classify real orgs using hardcode stats from fake ones. When I hear "we are solving x with AI" or "AI driven", it gives me jitters.
The elephant in the room is that AI does not exist, even in "hardcore" orgs. A transformer model is not AI.
The classical definition of AI in CS is so broad as it can even include expert systems. I think using that term when the general public sees it differently has done a great disservice to the fields credibility.
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#7This is true for every other domain. When every company calls itself a AI company these days. Therefore value of AI/ML and credibility gets diluted. It becomes very difficult to classify real orgs using hardcode stats from fake ones. When I hear "we are solving x with AI" or "AI driven", it gives me jitters.
The elephant in the room is that AI does not exist, even in "hardcore" orgs. A transformer model is not AI.
What I realized recently is the AI is subjective thing. AI for an Investor might be simple math for an engineer.
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#8Some radiologists think that AI will be really good for filtering out normal images, so that they only have to review anomalies. But I don't think that a model that detects diseases will be too successful, even if they manage to make it really work.
For one, it's actually difficult to interpret and find signs in radiologic images. Obvious signs are obvious, but there are others that could be image artifacts, or just indolent variations, or point to something serious. Even with a generally good accuracy, it'll be hard that a general model performs well on those anomalies with low prevalence.
Second, radiologic signs are just signs. Most diseases are diagnosed with more than just radiologic signs. Most signs are compatible with a lot of diseases. If you see a model that pretends to diagnose a certain disease, well, they're looking at it wrong.
Third, you need a way to have responsibility for diagnoses, and a way to find and correct errors. I don't think that's possible with an unsupervised AI, you'll always need a doctor there to check the image and verify the output. There won't be much savings there. Whenever someone says "AI is going to revolutionize medicine", it's really hard to believe them. I mean, you just have to look at EKGs, modern machines can detect anomalies but doctors still learn how to interpret EKGs and double check what the machine says. It's a help, but not a replacement.
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#9I've been at a few startups and ML has typically meant shoddy rules-based logic or some out of the box model. Only one company applied it with rigor and even then it was a slog - lackluster results, tinkering with different models, poring through research papers to figure out where the cognitive gap came from. The rest of the company thought we were brilliant as did prospective clients. Funny thing is it's possible (…
> Funny thing is it's possible (though less common) to get paid just as much if not more doing data analysis / engineering than vaunted ML/AI work
Which makes me wonder how much of the supposed magical 10x value of AI actually just comes from getting your shit together in terms of ETL and data pipelines.
Re: Machine learning is booming in medicine, but also facing a credibility crisis
#10Unless the AI can talk tot the patient to get some context, it's going to be taking decisions with only very partial information, no matter how good it is.
Still, I'm thinking that as it improves, it's going to show that doctors are not that good at their job on average, and that's going to be fun to watch.