Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards accurate differential diagnosis with large language models

arxiv.org

Post reply on HN
Print view View original

11–20 of 99 posts

Re: Towards accurate differential diagnosis with large language models

#11

Post by caycep »

fnordpiglet wrote: post #8
guender wrote: post #2

Yikes, do I read this correctly and the LLM alone outperforms clinician + LLM?

This has come up before in similar contexts. Model based decision decisions tend to do better than clinicians alone or clinicians with the model based decision system in general. Despite this it’s been effectively impossible to deploy such systems in a clinical setting due to many issues but one is that clinicians aren’t willing to cede their ground and patients aren’t willing to believe the machine over a human.

granted, I feel like the training of some physicians also doesn't make them that good at diagnosis. Over time, intuitively understanding the narrative history of the illness/symptoms is key; that being said, nowadays, this sort of thing seems scattershot in some younger physicians.

Med schools tend to try and rejigger their curriculum frequently to try and justifiably make the experience friendlier and w/ less workload for the students, but sometimes, it backfires in terms of this.

I had an older school professor who liked to remark, in his day, no such thing as a differential diagnosis, just the right one and a bunch of wrong ones.

Re: Towards accurate differential diagnosis with large language models

#12

Post by zmmmmm »

resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

> If we call the difference between GPT-3 and GPT4 1x, then I would expect to see a 2x-5x improvement in LLM capability within the next few years

it's interesting this is probably the assumption of many people leading to the current hype cycle.

But in reality it's not how progress tends to happen which is a much more punctuated equilibrium type phenomenon. My guess is it's actually quite likely that most of the progress will be made on maturing things and underlying capability will plateau.

Re: Towards accurate differential diagnosis with large language models

#13

Post by nradov »

resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

That is not obvious, and at least in the case of physicians is unlikely to come to pass. They don't just take a written list of signs and symptoms as input, and produce a diagnosis as output. Often the evidence they use to make a diagnosis isn't even written down anywhere. And for most primary care cases, the diagnosis is only a minor part of the workflow. The hard part is in working with the patient to make a practical treatment plan, and then course correcting as needed based on results. This is as much art as science, and relies on tacit knowledge that no LLM can access.

In order to automate most medical care we'll need multiple breakthroughs that go way beyond LLMs. Linear improvements in LLMs won't get us there.

LLMs do have potential to improve other areas of clinical practice such as charting.

Re: Towards accurate differential diagnosis with large language models

#15

Post by lumost »

doctorpangloss wrote: post #9
resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

> If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. While I don't know you personally and I'm not talking about you specifically, "just" feeling very excited about something doesn't Make This About You, the prognostications don't actually help you Be a Part of This. This is the same energy as being really into COVID-19 (what was…

I strongly doubt that the LLM club will remain exclusive for very long, or that the innovation in this space was restricted to 10k people.

At a minimum, I’ve already begun using gpt-4 as a consultant for my fathers medical issues. I can reasonably get confirmation/dis confirmation of whether a medical experts opinion is BS or not. As anyone who has dealt with the medical system in the US can attest - the majority of opinions you get from experts are either BS, outdated, or folks talking their own expertise.

Asking the question to GPT-4 can help you decide when to talk to a different doctor, or try something new entirely.

Re: Towards accurate differential diagnosis with large language models

#16

Post by refulgentis »

doctorpangloss wrote: post #9
resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

> If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. While I don't know you personally and I'm not talking about you specifically, "just" feeling very excited about something doesn't Make This About You, the prognostications don't actually help you Be a Part of This. This is the same energy as being really into COVID-19 (what was…

Apologies, I may misunderstand: there's a lot of Novel All Caps Concepts Even Though Caps Is For Well Worn Cliches. I Think Maybe You Think They're For Accusations? Insults? Big Concepts? For Handwaving At Big Concepts You've Divined Other People Are Thinking?

The last paragraph caught my attention though you were a bit abstract _and_ obtuse, due to the structure being "brief aside for the curious: LLMs don't have bodies"

My job and existence is centered around enabling doctors via LLMs and it absolutely is astounding to and for them. Funnily enough, it's the simplest thing: citations and easy UI to reach them. They perceive it as quick literature review on demand, as far as I can tell, that means they get to skip about 5 minutes across 4-8 articles checking links for relevancy.

I don't think OP was proposing that LLMs will replace doctors as you seem to think. I can't identify anything in their post suggesting that.

Re: Towards accurate differential diagnosis with large language models

#17

Post by skepticATX »

doctorpangloss wrote: post #9
resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

> If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. While I don't know you personally and I'm not talking about you specifically, "just" feeling very excited about something doesn't Make This About You, the prognostications don't actually help you Be a Part of This. This is the same energy as being really into COVID-19 (what was…

I have heard it described as a form of Pascal's wager. If you shout from the rooftops that LLMs are about to spawn a god machine, you get rewarded with attention, and if you're a researcher, job opportunities. There isn't nearly as much room for critics right now.

And if a few years down the line it turns out that you were wrong, everyone else was to, so no big deal.

In fact, you can already see it happening. Scaling up LLMs was surely enough to get to AGI, but now that apparently OpenAI is working on Q*, it's obvious that LLMs alone aren't enough, but LLMs + Q* are!

Re: Towards accurate differential diagnosis with large language models

#20

Post by rafaelero »

guender wrote: post #2

Yikes, do I read this correctly and the LLM alone outperforms clinician + LLM?

That surprised me as well. The case for humans being merely assisted by AI may not be the strongest one. It could be that they work better without human input, which is at the same time amazing and a bit frightening.
Post reply on HN