Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards accurate differential diagnosis with large language models

arxiv.org

Post reply on HN
Print view View original

41–50 of 99 posts

Re: Towards accurate differential diagnosis with large language models

#41

Post by whatshisface »

modeless wrote: post #30
dogmatism wrote: post #27

Earlier quoted context omitted.

Call me when an LLM can ingest a pts medical record from 3 different sources in multiple formats ranging from structured data to free form text stored (I shit you not) as TIFF files, extract the actual information, and present it for review prior to a visit and for a follow up visit, read the prior visit's note and orders, search/track down all the test results (may be at more than one institution with multiple EHR s…

I mean, have you used GPT-4? Free form text is its bread and butter, and yes, it can absolutely ingest images and read text in them, and combine that with other information from multiple sources in structured or unstructured text or image form, no problem! What you are describing sounded decades away last year. Today it's not something you can rely on quite yet if you need accuracy. But if you still believe it's deca…

This reminds me of what happened in self-driving. It went from "decades away," to "it works but you can't quite rely on it for accuracy" in a year and then it stopped cold for some reason.

Re: Towards accurate differential diagnosis with large language models

#42

Post by acover »

corethree wrote: post #35
doix wrote: post #26

This doesn't really surprise me given my experience with doctors. If it's a common problem then doctors do fine. As soon as you're not the common case or have something somewhat rare, it becomes a bit of a crap shoot, at least in the UK when you deal primarily with your GP (general practitioner). I have a friend with Crohn's who was feeling low energy. I was a gym bro at the time and convinced him to take a testoster…

[flagged]

You can make the right call but still be wrong sometimes.

There's a cost to tests of rare diseases.

Re: Towards accurate differential diagnosis with large language models

#43

Post by hackerlight »

In poor countries there aren't enough doctors. Billions of people don't have access to healthcare and that won't change until the country becomes wealthier. Public healthcare that covers everyone can't happen because the reality of poverty makes it impossible. For these billions of people, the valid comparison isn't human doctor vs AI, it's no treatment vs AI. I am extremely optimistic about the positive role of AI in medicine for this reason.

Re: Towards accurate differential diagnosis with large language models

#44

Post by vidarh »

dogmatism wrote: post #27
resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

Call me when an LLM can ingest a pts medical record from 3 different sources in multiple formats ranging from structured data to free form text stored (I shit you not) as TIFF files, extract the actual information, and present it for review prior to a visit and for a follow up visit, read the prior visit's note and orders, search/track down all the test results (may be at more than one institution with multiple EHR s…

So you describe a couple of things LLMs seem near ideal for, and one thing that is a horrific failure of information management that it's shocking if doctors are wasting their time on, and a case for other automation.

At my GP at least, all of this is presented to doctors digitally, all notes are digital, and all requisition of follow up tests are digital, and my initial consultations are all phone or video and only follow up visits are in person and only if a physical examination or tests are needed.

Re: Towards accurate differential diagnosis with large language models

#45

Post by kridsdale1 »

whatshisface wrote: post #41
modeless wrote: post #30

Earlier quoted context omitted.

I mean, have you used GPT-4? Free form text is its bread and butter, and yes, it can absolutely ingest images and read text in them, and combine that with other information from multiple sources in structured or unstructured text or image form, no problem! What you are describing sounded decades away last year. Today it's not something you can rely on quite yet if you need accuracy. But if you still believe it's deca…

This reminds me of what happened in self-driving. It went from "decades away," to "it works but you can't quite rely on it for accuracy" in a year and then it stopped cold for some reason.

Because it kept crushing peoples knees.

Re: Towards accurate differential diagnosis with large language models

#46

Post by vidarh »

dogmatism wrote: post #32
modeless wrote: post #30

Earlier quoted context omitted.

I mean, have you used GPT-4? Free form text is its bread and butter, and yes, it can absolutely ingest images and read text in them, and combine that with other information from multiple sources in structured or unstructured text or image form, no problem! What you are describing sounded decades away last year. Today it's not something you can rely on quite yet if you need accuracy. But if you still believe it's deca…

see my comment above Pt sees Dr in office A, has labs and scans ordered. Labs order is printed out and given to pt, who takes it to lab A. Scan #1 is ordered, and order faxed to facility B, who independently calls, schedules, performs, and interprets scan #1, same for scan #2, but at a different facility. Office A has no electronic link to any of these facilities How does LLM get the data? A human knows the patterns…

What you're describing is a process and information management failure that ought to be addressed first, sure.

I book a video consult. I talk to a doctor, who has an initial description of my symptoms from my booking on her computer. She takes notes on a computer (I have access to them). If needed, she orders labs on a computer. I go in, and there is no paper, just an electronic record, and while there may well be paper in the process in some instances behind the scenes, data entry people ensures it is all added to the electronic record. If labs are not needed, she'll submit my prescription electronically to my local pharmacy.

This is how it's worked when I talk to my doctor for years (UK; not nearly all GPs have gone this far)

It may well "only save the bare minimum" of the actual labour happening behind the scenes, but with a well functioning system, it saves the expensive, resource constrained part of the work.

Re: Towards accurate differential diagnosis with large language models

#47

Post by bradknowles »

doix wrote: post #26

This doesn't really surprise me given my experience with doctors. If it's a common problem then doctors do fine. As soon as you're not the common case or have something somewhat rare, it becomes a bit of a crap shoot, at least in the UK when you deal primarily with your GP (general practitioner). I have a friend with Crohn's who was feeling low energy. I was a gym bro at the time and convinced him to take a testoster…

I have hypertension, as well as diabetes. I wanted my cardiologist to test me for hyperaldosteronism, because I had read it was not uncommon and could be a factor in hypertension that continues to get worse and worse, without an obvious explanation. He refused, saying that it was the job of my endocrinologist to do that test.

So, I talked to my endocrinologist and he said it wasn't a normal test they do, but he was happy to do it if I wanted it. Well, the test came back positive. So they sent me in for a CAT scan of my adrenal glands to see if anything was obviously abnormal there. Which came back negative. So, now they want me to do a salt test to see how bad my hyperaldosteronism is, so that they can give me the right amount of the drug for that condition.

But the key thing that set me off was that original article that said this was now recommended by cardiologists as a standard screening test. Maybe only 10% of patients have this problem, but it's common enough that it's still worthwhile screening for.

So, why didn't my cardiologist screen me for it in the first place? Why didn't my endocrinologist screen me for it without me having to explicitly request it?

I'm pissed.

I don't want to sue anyone for malpractice, or even mention that word to any of my doctors. But I do want to convince them that they really do need to screen for this condition.

Meanwhile, I'm going to continue to follow this diagnosis to see where it goes, and if they can help me get my blood pressure under better control.

Re: Towards accurate differential diagnosis with large language models

#48

Post by staunton »

corethree wrote: post #35
doix wrote: post #26

This doesn't really surprise me given my experience with doctors. If it's a common problem then doctors do fine. As soon as you're not the common case or have something somewhat rare, it becomes a bit of a crap shoot, at least in the UK when you deal primarily with your GP (general practitioner). I have a friend with Crohn's who was feeling low energy. I was a gym bro at the time and convinced him to take a testoster…

[flagged]

> statements like this literally set me off. The patient being right and the doctor ignoring the patient and treating the patient like an idiot

There's a huge number of people who vastly overestimate their medical knowledge, google some symptoms (I'm sure you know the pitfalls of that but many don't), do self-diagnosis as a hobby with friends over coffee, come to absurd conclusions and then bother their doctors about it.

It's not ideal but somewhat understandable that under time pressure experts want to to know the object level (symptoms, history) and don't want to waste time listening to the speculation of non-experts, even if that speculation is sometimes right, and even if some (the doctor can't tell who) of those non-experts actually make good points.

Re: Towards accurate differential diagnosis with large language models

#49

Post by vidarh »

hackerlight wrote: post #43

In poor countries there aren't enough doctors. Billions of people don't have access to healthcare and that won't change until the country becomes wealthier. Public healthcare that covers everyone can't happen because the reality of poverty makes it impossible. For these billions of people, the valid comparison isn't human doctor vs AI, it's no treatment vs AI. I am extremely optimistic about the positive role of AI i…

The irony is that we may see another leapfrog moment similar to analog cellphone networks and Internet infrastructure (where many poor countries largely skipped analog and modems because they had their boom later), where it will be easier to deploy AI for these purposes in countries where the existing healthcare system is underfunded and lacking in political clout than in richer countries even for situations where it delivers better outcomes.

Re: Towards accurate differential diagnosis with large language models

#50

Post by staunton »

whatshisface wrote: post #39
lumost wrote: post #15

Earlier quoted context omitted.

I strongly doubt that the LLM club will remain exclusive for very long, or that the innovation in this space was restricted to 10k people. At a minimum, I’ve already begun using gpt-4 as a consultant for my fathers medical issues. I can reasonably get confirmation/dis confirmation of whether a medical experts opinion is BS or not. As anyone who has dealt with the medical system in the US can attest - the majority of…

This adds a whole new dimension to AI safety, it's one thing to dumb it down so it can't do bad stuff, but when you do that you're making it give worse medical advice to people who are going to take that advice.

It's possible most publicly available models (or at least "models the general public uses") will eventually refuse to give medical advice
Post reply on HN