This „study“ is missing a control group. He should have given his data also to some humans and see how they would do compared to ChatGPT.
I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
241–250 of 368 posts
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#242Earlier quoted context omitted.
Translators who've been picking up after machine translation for the last decade will tell you that it is not, in fact, useful to them. They get paid half as much as before, but correcting the machine output is as much work as translating from scratch, and sometimes more, since the machine can give results that look good at first glance but are fundamentally incorrect.
I'm a translator, and that might be true for Google Translate, but not GPT-4. GPT-4 produces translations that are generally accurate, and when instructed to copyedit its own work, well-written. In most cases, all I need to do is fix the occasional error and do some light copyediting. This is much less work than translating from scratch and results in superior quality, since it combines my own skills with the AI's. I…
> It is also useful for researching equivalents to obscure terms across languages.
I'm super curious to learn how you use AI to do that. Thanks!
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#243One complaint he has is "It doesn't know to ask the right questions". Well, the prompt was to give diagnoses, not questions. Ask GPT for the follow up questions first, then the diagnoses. This is fascinating in that, because now the machine speaks human, we subconsciously ascribe human agency to it. This guy was instictively treating it like a colleague, who would naturally ask follow up questions unprompted. But you…
Doctors are done? Can the chat bot take out a colon or gall bladder or fix an aneurysm? Hilarity.
Most doctors aren’t surgeons.
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#244Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#245"If my patient notes don’t include a question I haven’t yet asked, ChatGPT’s output will encourage me to keep missing that question."
This is the point we have to help people understand and I'm not sure AI will catch up with this anytime soon; questions are the key to knowledge and intelligence. I haven't seen an AI ask interesting questions. Maybe it's possible with the right training set and weighting of factors to encourage enquiry, but this will be a gap in AI's ability for at least the near term.
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#246One complaint he has is "It doesn't know to ask the right questions". Well, the prompt was to give diagnoses, not questions. Ask GPT for the follow up questions first, then the diagnoses. This is fascinating in that, because now the machine speaks human, we subconsciously ascribe human agency to it. This guy was instictively treating it like a colleague, who would naturally ask follow up questions unprompted. But you…
Other commenters take offense at your comment "doctors are done" and point to doctors doing physical things.
GPT can already write robot control and machine vision code (poorly).
The real questions are harder.
For example, what about patients who lie or are too stupid to answer correctly? They can still be helped by a real human doctor, but GPT would be clueless.
Another angle: we didn't have radiation sickness, HIV, popcorn lung, or computer induced carpal tunnel 100 years ago. Will GPT be clever enough to diagnose radically new problems?
What happens when old diagnoses and treatments are superseded? Will it tell you you need a lobotomy when you instead need antipsychotics?
GPT right now will happily roleplay a lawyer who cites legal precedent that was superseded 40 years ago.
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#247"If my patient notes don’t include a question I haven’t yet asked, ChatGPT’s output will encourage me to keep missing that question." This is the point we have to help people understand and I'm not sure AI will catch up with this anytime soon; questions are the key to knowledge and intelligence. I haven't seen an AI ask interesting questions. Maybe it's possible with the right training set and weighting of factors to…
I don't remember the context, but I have seen properly-prompted GPT-4 proactively ask questions.
It's also worth noting that the future is multi-layered. The Reason+Act model (https://ai.googleblog.com/2022/11/react-synergizing-reasonin...) should be excellent at getting the LLM to analyse its own output and inquire about missing pieces of knowledge.
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#248Earlier quoted context omitted.
> a half-working system may be tremendously exciting to AI enthusiasts, but it's not that useful to society. A half-working system might actually be worse than useless if people believe it 100% of the time.
And a 98% working system even worse due to it being treated as 100%.
Can you give a single example of a "system" that works 100% well, 100% of the time?
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#249Earlier quoted context omitted.
> GPT-4 passed the medical exam, not ChatGPT running GPT-3 With a subscription you can use GPT-4 with ChatGPT. ChatGPT is just the wrapper to the model.
That's not true, ChatGPT is a model. Quote from the ChatGPT announcement post: > ChatGPT is a sibling model to InstructGPT, which is trained to follow an instruction in a prompt and provide a detailed response.
I’m looking at the ChatGPT interface right now (paid account) and I have a “Model:” drop-down at the start of a new chat that says:
- Legacy (GPT-3)
- Default (GPT-3.5)
- GPT-4
Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients
#250One complaint he has is "It doesn't know to ask the right questions". Well, the prompt was to give diagnoses, not questions. Ask GPT for the follow up questions first, then the diagnoses. This is fascinating in that, because now the machine speaks human, we subconsciously ascribe human agency to it. This guy was instictively treating it like a colleague, who would naturally ask follow up questions unprompted. But you…
Doctors are done? Can the chat bot take out a colon or gall bladder or fix an aneurysm? Hilarity.
No, but a GP can also not do those things and on top of that I've met some pretty terrible GP's :p
So maybe some classes of doctors are done...
Another thing to keep in mind: tons of people don't have access to any type of doctor at all (too expensive, too far away, queue too long, etc). Is current state of AI better or worse than no doctor at all? I don't know, but the answer is not obvious to me.
In those cases AI isn't competing with GP's, it's competing with internet searches.