I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

>is bad at understanding the atypical.

To be fair I've found most doctors require a lot of convincing if your problem atypical as well.

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety.

This article and the comments in this thread are right up that alley. I mean, can you imagine say 1 or 2 years ago saying we'd have a readily accessible system that you could feed it the symptoms a patient is experiencing (in any language!) and out would split a well-described explanation of the diagnosis (in any language!) around half the time? And now it is suddenly here, and so many people are like, "yeah, but it isn't perfect" or "we can't completely trust it" or "we have to fact check it because it can be wrong".

It is all so weird and fascinating and just bigger than I can describe.

But here we are!

> only reflecting back to me the things I thought were obvious — enthusiastically validating my bias like the world’s most dangerous yes-man.

This is why it's exciting: we're seeing that awkward stage of impressive (for entry level/passing the bar) but still requires (expert?) supervision. Any worse and the novelty would wear off - any better and we'd be having (warranted) AI panic.

Could go either way tbh

> ChatGPT does not ask for clarification, because it is not a knowledge base trying to find an answer. All it does is figure out what character is statistically most likely to come next, it has no heuristic to know that there is a task it hasn't fully completed.

It absolutely can, but it requires the right prompts. If you just say "give me a diagnosis" it won't, but if you prompt it with a "you are a doctor and I am a patient" kind of prompt, it will.

I'll agree that it tends to "close off" conversations without a lot of open endedness - I role-played through a session using myself as a patient with a chronic condition I suffer from and it correctly identified the problem, but I had to push it a little to get it to try to narrow down the problem (instead of "here's a bunch of things it could be" and ending the convo there).

This is a republication of an older article that was published just when ChatGPT 4 came out, and the date was changed.

I personally had seen good and bad parts of diagnosing with ChatGPT 4, and what I would interested in is if the doctor tries using multiple questions and finds out how to use the tool well.

I believe he could have improved the tool significantly if he puts in the time to experiment with it.