Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

inflecthealth.medium.com

Post reply on HN
Print view View original

41–50 of 368 posts

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#41

Post by chimeracoder »

tedunangst wrote: post #18
chimeracoder wrote: post #11

> So after my regular clinical shifts in the emergency department the other week, I anonymized my History of Present Illness notes for 35 to 40 patients — basically, my detailed medical narrative of each person’s medical history, and the symptoms that brought them to the emergency department — and fed them into ChatGPT. It's quite shocking that the doctor would openly admit to violating HIPAA in such a brazen way. HI…

And yet medical journals are filled with articles with sufficient detail that other doctors can even learn to make diagnoses from reading them.

> And yet medical journals are filled with articles with sufficient detail that other doctors can even learn to make diagnoses from reading them.

This would be an apt analogy, if medical journals involved no oversight from the covered entity at which the patient presents, if there were no editorial intermediary, and if the entire publication timeline happened in weeks, allowing for no data redaction and review, rather than years.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#42

Post by jjcon »

zzzeek wrote: post #6

[flagged]

> because this doctor is not practicing in Texas where such a procedure might get you arrested

https://texas.public.law/statutes/tex._health_and_safety_cod...

Please don't spread misinformation, there is enough confusion out there already. Texas law specifically allows for the removal of ectopic pregnancies.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#43

Post by fzzzy »

Maursault wrote: post #33
gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

> as "true" AI begins to emerge What, exactly, is that?

The definition has always been the Turing test. Nobody has been able to provide a more precise answer.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#45

Post by jacquesm »

suddenclarity wrote: post #12

It does worry me what data people are sharing without seemingly much though. He claims it anonymised but I'm a bit sceptical when you input the medical history of 40 people. It's easy to slip up.

With a rare enough disease the anonymized file would still be enough to ID the patient given where the doctor is located.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#46

Post by flir »

This is how I played doctor with ChatGPT.

> Do you know what an Expert System is?

[describes expert system]

> I'd like you to emulate an expert system while I play the role of a patient. Expert systems ask questions in order to produce a diagnosis. I would like you to ask me a series of questions, and respond with a diagnosis when you feel confident.

Alright, I'll do my best to emulate an expert system. Please keep in mind that I'm an AI language model and not a certified medical professional. For any serious concerns, it's essential to consult with a healthcare provider. Let's begin.

What is your primary complaint or reason for seeking help today?

I fed it a symptom my doctor had already diagnosed, and it did ok - it got it down to three possible causes, one of which was the correct one. All along the way it was warning me that I really should see a real health professional and it's just a chatbot.

What really interested me is that I said "please emulate an expert system" and it did. Once upon a time, expert systems were an entire branch of AI, and here it is just emulating one off the cuff.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#47

Post by weaksauce »

famouswaffles wrote: post #31
gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

It's interesting for sure. There's also the fact that he didn't use the sota (GPT-4). I'm not saying that would be perfect. But i would be surprised if it wasn't big leap over 3.

it wasn't mentioned anywhere in there which version of the model he was using... how are you so sure?

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#48

Post by jcims »

I know everyone scoffs at the concept of 'prompt engineer', but it really is an essential craft that we're going to have to come to terms with when interacting with large language models.

Seeking suggestions on a more comprehensive prompt: https://sharegpt.com/c/sckAPvV

Trying it out: https://sharegpt.com/c/LbpEIxi

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#49

Post by viraptor »

JacobDotVI wrote: post #21
Turskarama wrote: post #7

I'm not really sure what he expected here, ChatGPT was not trained to be a doctor, it is far more general than that. Asking ChatGPT for medical advice is like asking someone who is very well read but has no experience as a doctor, and in that context it's doing very well. He also brings up one of the most salient points without really visiting it enough: ChatGPT does not ask for clarification, because it is not a kno…

My hunch is that this is exactly what he was expecting. There is a lot of hype around ChatGPT passing the medical exam and this exercise is a counter point to that.

The medical exam has very specific questions which define what you're expected to include in the answer. The question asked in this case was nowhere near that detailed, do I don't think they're comparable. To really evaluate something beyond the "random generic user" level, you need to be familiar with the tech as well.

The article really tells us more about the experience of someone with no chatgpt knowledge checking their own symptoms rather than its usability for emergency diagnosis.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#50

Post by messe »

asdff wrote: post #34
gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

But at the same time consider the problem, and if AI is even the tool for this job or if its total overkill. We are just matching symptoms to a diagnosis, you could get this utility from programming a simple dictionary with a fraction of the compute. Maybe even with a confidence estimate of the diagnosis this time. There's an old adage that rings true here I think: "keep it simple, stupid."

But I think that's missing the "magic" of it—for lack of a better word. In the situation you describe, you have to build a specific tool for the task. With LLMs, we appear to have stumbled on a general purpose tool for a myriad of situations. Yes, they're not perfect; but the fact that they work remotely as well as they do even at these early stages is astonishing.
Post reply on HN