Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

inflecthealth.medium.com

Post reply on HN
Print view View original

211–220 of 368 posts

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#211

Post by kelnos »

Shorel wrote: post #192
gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

It makes predictions based on incomplete and or biased information. Just as doctors do. The funny thing is not that we are holding AI to a high standard. The funny thing is that we normally do not do the same for doctors, marketers, politicians, etc.

Sure we do, for doctors at least. A doctor that got ER diagnoses wrong 50% of the time would get fired.

(And the AI here didn't even get it right 50% of the time, just 50% of the time it presented the right answer as one of several options. If pressed, it may not have made the one single correct answer for each of those patients.)

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#212

Post by blibble »

gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

> This article and the comments in this thread are right up that alley. I mean, can you imagine say 1 or 2 years ago saying we'd have a readily accessible system that you could feed it the symptoms a patient is experiencing (in any language!) and out would split a well-described explanation of the diagnosis (in any language!) around half the time?

we've had this for years (bar the "any language")

they're traditional expert systems with the same flowchart a GP (family doctor) would use to ask questions

and they do a lot better than 50%

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#213

Post by seydor »

There 's 2 main questions about what current AI systems (or whatever one thinks they are) : (1) can it be improved to learn to be ~100% correct via reinforcement learning or otherwise? It seems like the answer is Yes (2) Will people become addicted and dependent on AI to the point where it may become problematic? Also yes.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#214

Post by dzdt »

civilized wrote: post #37
gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

What I didn't predict is so many AI boosters being so miffed at regular folks pointing out the obvious: a half-working system may be tremendously exciting to AI enthusiasts, but it's not that useful to society. Recently we've had all these people complaining of moving goalposts, when in fact most people never had goalposts for AI in the first place, and certainly never had goalposts that remotely resembled those that…

The half-working AI is like 95% of the progress to a better-than-the-best-human AI.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#215

Post by badcppdev »

paraxion wrote: post #127

I wonder if, instead of asking ChatGPT for a diagnosis, he could've got it to prompt for further questions he could ask? My thinking is that given the nature of LLMs of connecting related information, it might be a good way to figure out the gaps in the diagnostic process, rather than actually provide one.

Or the doctor could feed their answer into ChatGPT before they give it to the patient and ask if there are any possible errors

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#216

Post by aflag »

ekidd wrote: post #79
aflag wrote: post #51

Earlier quoted context omitted.

Not sure if I'd compare ChatGPT with a junior programmer. In my experience junior programmers tend to be builders. They will tend to code a lot of stuff and usually get reasonable results, but making some bad decisions that more experienced developers have already gone through. Inexperienced developers need supervision because otherwise they will just create heaps of code that will be hard to maintain later. ChatGPT…

> ChatGPT just doesn't do anything on its own and will never follow through with anything. Yeah, I should be clear: the specific context in which I often see junior developer quality performance is when I'm essentially "pair programming" with it. I can say, "Write a program that does X. No, there's a closed-form solution for that, so we shouldn't write it recursively. OK, now write some tests similar to the following…

Yes, I see what you mean. But just like you're being less productive when you try to instruct someone else on how to do things, if you follow that path with ChatGPT you'll likely be less productive too. I think ChatGPT adds more value when you ask things you don't quite know or figured out yet. Sometimes it hallucinates, which you can usually figure out quickly, but sometimes it does give good insights which would take longer for you to get than the time you lose by checking hallucinations. It's a decent advisor. It's a rubber duck on steroids.

I guess copilot is a bit different in that regard though, I haven't used that much, but I heard it's good at writing unit tests.

PS: you should probably not refer to a someone else as "it", specially if you are more senior than them.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#217

Post by vidarh »

civilized wrote: post #37
gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

What I didn't predict is so many AI boosters being so miffed at regular folks pointing out the obvious: a half-working system may be tremendously exciting to AI enthusiasts, but it's not that useful to society. Recently we've had all these people complaining of moving goalposts, when in fact most people never had goalposts for AI in the first place, and certainly never had goalposts that remotely resembled those that…

A half-working system that hasn't been trained specifically for this purpose is a strong indicator that a system that can handle this use-case far better is within reach. That is what is exciting about it. I'm constantly mystified by those who expect a system trained only on publicly accessible data pulled from the web to be an expert in everything rather than seeing it as an indication of what can be possible if enough specialised training data is added.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#218

Post by apwheele »

gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

I think people are more impressed with the "well-described explanation", which is a (mostly bullshit) interpolation from prior text on the internet. If you just focus on the "feed a narrative and get out a probability of a particular diagnosis" -- the ability to do that with other machine learning models has already existed.

If you give me a sample of 100k ER text records and final diagnoses, I would be pretty confident I can estimate a model that gets you +50% accuracy on my laptop using open source tools.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#219

Post by lumb63 »

The tail end of this article, where the author talks about how many more patients he could see in his life if he had AI assistance, made me realize that part of healthcare cannot be solved by AI. The goal is not to see more patients; the goal is to help more patients get better. For a lot of patients who are frustrated and have not had their problems validated and have been to many doctors and seen no results or poor results, having a real, physical, human doctor validate their condition, and work with them to solve it is part of the treatment. Doctors can prescribe whatever medicines and do whatever surgeries they want, but only the patient’s body is capable of healing itself. I worry that a tendency to plug symptoms into an AI that “diagnoses” the patient, that the patient doesn’t trust, will hurt outcomes. The patient benefits greatly from understanding the doctor’s methodology and thought process.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#220

Post by MrPatan »

apwheele wrote: post #218
gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

I think people are more impressed with the "well-described explanation", which is a (mostly bullshit) interpolation from prior text on the internet. If you just focus on the "feed a narrative and get out a probability of a particular diagnosis" -- the ability to do that with other machine learning models has already existed. If you give me a sample of 100k ER text records and final diagnoses, I would be pretty confid…

What percent of human doctors diagnoses is also well-decribed bullshit, though?
Post reply on HN