Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards accurate differential diagnosis with large language models

arxiv.org

Post reply on HN
Print view View original

71–80 of 99 posts

Re: Towards accurate differential diagnosis with large language models

#71

Post by corethree »

evgen wrote: post #65
corethree wrote: post #54

Earlier quoted context omitted.

Then let the person who is wrong foot the cost. The cost isn't just the cost of the test. It's the cost of what occurred because the doctor decided to block a test.

No one is preventing you from going to a private clinic and paying for the test yourself. The doctor is not 'blocking' the test or preventing you from getting it, they are just not authorising it. Sounds like you have a problem with either your insurance provider or your wallet.

https://www.cdph.ca.gov/Programs/OSPHLD/LFS/CDPH%20Document%...

This is a cartel based law. Basically patients can't order tests or even look at results from tests by law.

The "made up" reasoning for why this law exists is because the law makers acquiesced into agreeing with doctors that all patients are fucking idiots. That's the bullshit reason.

But really it's business reasons. Same reason why the doctor supply is made artificially low, all these things are legislated by the AmA to lower the supply of doctors and give them more control.

>Sounds like you have a problem with either your insurance provider or your wallet.

If you're a doctor. My problem is you. How does a doctor even have the gall to say this? It's flat out wrong. So if you're a doctor there's only two logical possibilities for why you said what you said. 1. You're a liar and are therefore untrustworthy. 2. You don't know what you're talking about and are therefore incompetent. Which is it?

Re: Towards accurate differential diagnosis with large language models

#72

Post by IanCal »

dogmatism wrote: post #32
modeless wrote: post #30

Earlier quoted context omitted.

I mean, have you used GPT-4? Free form text is its bread and butter, and yes, it can absolutely ingest images and read text in them, and combine that with other information from multiple sources in structured or unstructured text or image form, no problem! What you are describing sounded decades away last year. Today it's not something you can rely on quite yet if you need accuracy. But if you still believe it's deca…

see my comment above Pt sees Dr in office A, has labs and scans ordered. Labs order is printed out and given to pt, who takes it to lab A. Scan #1 is ordered, and order faxed to facility B, who independently calls, schedules, performs, and interprets scan #1, same for scan #2, but at a different facility. Office A has no electronic link to any of these facilities How does LLM get the data? A human knows the patterns…

> How does LLM get the data?

* Run database query to see where the type of scan is typically ordered (we know they can generate sql pretty alright)

* * Alternatively someone at one point needs to just make a list

* Call the office (voice conversations is clearly possible, it's a feature in chatgpt as an app)

* Receive fax

* gpt-4-vision or OCR then GPT-4 or however you want to interpret the data

It's not like it's not work to build something like this, but all the pieces are right there.

Re: Towards accurate differential diagnosis with large language models

#73

Post by krisoft »

whatshisface wrote: post #41
modeless wrote: post #30

Earlier quoted context omitted.

I mean, have you used GPT-4? Free form text is its bread and butter, and yes, it can absolutely ingest images and read text in them, and combine that with other information from multiple sources in structured or unstructured text or image form, no problem! What you are describing sounded decades away last year. Today it's not something you can rely on quite yet if you need accuracy. But if you still believe it's deca…

This reminds me of what happened in self-driving. It went from "decades away," to "it works but you can't quite rely on it for accuracy" in a year and then it stopped cold for some reason.

> and then it stopped cold for some reason

By stopped cold you mean "there are actual real world self driving cars delivering actual real world passengers in select locations"?

Re: Towards accurate differential diagnosis with large language models

#74

Post by refulgentis »

skepticATX wrote: post #17
doctorpangloss wrote: post #9

Earlier quoted context omitted.

> If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. While I don't know you personally and I'm not talking about you specifically, "just" feeling very excited about something doesn't Make This About You, the prognostications don't actually help you Be a Part of This. This is the same energy as being really into COVID-19 (what was…

I have heard it described as a form of Pascal's wager. If you shout from the rooftops that LLMs are about to spawn a god machine, you get rewarded with attention, and if you're a researcher, job opportunities. There isn't nearly as much room for critics right now. And if a few years down the line it turns out that you were wrong, everyone else was to, so no big deal. In fact, you can already see it happening. Scaling…

This is sort of like reading a 5 year old saying hamburgers are supposed to be good but adults are surprising the truth, the top secret new Big Mac sauce is required to make it edible, not to mention a bun.

Re: Towards accurate differential diagnosis with large language models

#75

Post by doix »

rscho wrote: post #60
doix wrote: post #26

This doesn't really surprise me given my experience with doctors. If it's a common problem then doctors do fine. As soon as you're not the common case or have something somewhat rare, it becomes a bit of a crap shoot, at least in the UK when you deal primarily with your GP (general practitioner). I have a friend with Crohn's who was feeling low energy. I was a gym bro at the time and convinced him to take a testoster…

As almost every comment of this kind on this site, you are missing the wider socioprofessional context. Yeah sure, you may find papers about plenty of things that may affect you without your doc's knowledge, perhaps even guidelines that are to be followed. The problem is that respecting every detail of every guideline all the time is pretty much impossible, and even a debatable stance. There are many committees produ…

> But that doesn't make AI the use-all-do-all, because if everybody starts to test for every little thing, diagnostic probabilities will decrease massively and guidelines would have to adjusted in conséquence. It's a feedback loop. And also, you'd pay 5x the price for healthcare, so not realistic.

It's hard for me to write out all my thoughts on the subject, but I completely agree with this. The paper cherry-picked difficult cases and compared how AI deals with them vs regular clinicians. I think that is something AI will excel at because there is no cost to being "wrong", as long as something it spews out is correct.

In difficult to diagnose cases, that kind of makes sense. Find the answer at "any cost". But you cannot apply that to every case. If you let the AI loose on fairly simple cases, I bet it would over complicate many simple things, because it couldn't handle the nuance of people coming in and just whining about a regular cold. "Patient comes in with a headache, might be brain cancer but might be a cold" ~ type DDx.

Re: Towards accurate differential diagnosis with large language models

#76

Post by Tostino »

ProllyInfamous wrote: post #59
gundmc wrote: post #55

Earlier quoted context omitted.

Part of this is by design - from medical school, doctors are quite literally taught "when you hear hoof beats think horses, not zebras." The progression and flow the decision trees in diagnosis are built to find the most common ailments in the most efficient way. It sucks for people whose conditions are further down the tree, but generally it does what it was designed to do. Hopefully technology like this can help us…

I dropped out of medical school fifteen years ago — entered another [non-tech] profession entirely - and there's one thing I always tell either of the following groups: Said to physicians: "I dropped out and I TELL EVERYBODY I KNOW you cannot pay a doctor enough for all the unnecessary sacrifices they had to make, just to prescribe you an antibiotic." ...to Non-physicians: "While I believe that you cannot pay physici…

Not yet. Your faith in people not being led astray by the LLM is too high right now IMO.

I've seen way too many people who I consider generally intelligent, completely falling for the convincing bullshit. Mostly in my online communities, but also a few times in person.

Re: Towards accurate differential diagnosis with large language models

#77

Post by chaxor »

corethree wrote: post #35
doix wrote: post #26

This doesn't really surprise me given my experience with doctors. If it's a common problem then doctors do fine. As soon as you're not the common case or have something somewhat rare, it becomes a bit of a crap shoot, at least in the UK when you deal primarily with your GP (general practitioner). I have a friend with Crohn's who was feeling low energy. I was a gym bro at the time and convinced him to take a testoster…

[flagged]

They suck the *insurers* dry?

Insurance is one of the major problems in the US, and they're a parasite on all of the US society, so this line of thinking is completely backwards.

However, the idea that physicians should make less is reasonable, in order to get people who are more interested in the real medicine, rather than money seeking people that is often found now. Getting rid of insurance and the structures that involve it essentially solves this issue.

Unfortunately, it's unlikely to occur, so sidestepping everything using capitalism is the only way forward I can see. One good example is making diagnostic tools at the house more common place. It's pretty typical to be able to purchase an entire machine for the price you will be charged at a physician's office.

For instance, a good 12 lead ECG may cost about $1300 to get for your house, but it will also cost that much to get one test done at the physicians office. So it's clear that buying one for the house is the way to go, as you can do it everyday for years for the price of one time through 'the system'. It could also be much cheaper if targeting every household through economies of scale.

This is true for essentially all diagnostic tools. So it's about time people just expand their first aid kits to involve some of these simple diagnostic tools, like urinalysis kits, etc.

The interpretation is of course valuable, as it takes a little while to understand the output of different tools, but much of this is very capable to be automated and sold with the tool or plugged into a foss solution.

Perhaps it's not ideal to some, but it's the easiest to occur in capitalistic settings and where insurance keeps costs at institutions so high.

Re: Towards accurate differential diagnosis with large language models

#78

Post by rscho »

corethree wrote: post #71
evgen wrote: post #65

Earlier quoted context omitted.

No one is preventing you from going to a private clinic and paying for the test yourself. The doctor is not 'blocking' the test or preventing you from getting it, they are just not authorising it. Sounds like you have a problem with either your insurance provider or your wallet.

https://www.cdph.ca.gov/Programs/OSPHLD/LFS/CDPH%20Document%... This is a cartel based law. Basically patients can't order tests or even look at results from tests by law. The "made up" reasoning for why this law exists is because the law makers acquiesced into agreeing with doctors that all patients are fucking idiots. That's the bullshit reason. But really it's business reasons. Same reason why the doctor supply is…

I am a doctor, and I encourage you to learn about how diagnostic test performance is modelled, and the consequences linked to overtesting and testing without contextual knowledge. It may change your perspective and lead you towards less extreme opinions.

Re: Towards accurate differential diagnosis with large language models

#79

Post by corethree »

chaxor wrote: post #77
corethree wrote: post #35

Earlier quoted context omitted.

[flagged]

They suck the *insurers* dry? Insurance is one of the major problems in the US, and they're a parasite on all of the US society, so this line of thinking is completely backwards. However, the idea that physicians should make less is reasonable, in order to get people who are more interested in the real medicine, rather than money seeking people that is often found now. Getting rid of insurance and the structures that…

Insurance is parasitic but they are the symptom not the root of the problem. They suck insurance dry so insurance has to suck you dry. The root is low supply of doctors. Thus unhealthy competition and thus doctors and the whole industry can charge exorbitant prices. They rip off insurance so insurance in turn rips you off. Make no mistake insurance is ripping you off, but there's healthy competition in this area. One insurance company can undercut another if they could... But they can't. Because the root of the problem is the cost of care itself.

Of course the reason why there's a low supply of doctors is a deliberate business maneuver made by the AmA to lower the amount of doctors by lowering the amount of medical schools. In short Doctors are the reason why there's a low supply of doctors.

The problem isn't purely capitalistic. It's sort of a hybrid of capitalism and government thats what causes this problem. When doctors aka the AmA are able to influence law to give them an unfair advantage that's what is responsible for the current state of things.

In fact hybrids are responsible for most of corruption. You have to keep a clear separation between business and government for a healthy free market economy as in business cannot influence government but government can influence business. The incentives are misaligned for both parties. One is profit the other is honor, thus they cannot mix.

Re: Towards accurate differential diagnosis with large language models

#80

Post by corethree »

rscho wrote: post #78
corethree wrote: post #71

Earlier quoted context omitted.

https://www.cdph.ca.gov/Programs/OSPHLD/LFS/CDPH%20Document%... This is a cartel based law. Basically patients can't order tests or even look at results from tests by law. The "made up" reasoning for why this law exists is because the law makers acquiesced into agreeing with doctors that all patients are fucking idiots. That's the bullshit reason. But really it's business reasons. Same reason why the doctor supply is…

I am a doctor, and I encourage you to learn about how diagnostic test performance is modelled, and the consequences linked to overtesting and testing without contextual knowledge. It may change your perspective and lead you towards less extreme opinions.

Are you a US doctor? I urge you to explain why you charge so much and have the poorest performance out of all 1st world countries? It may change your perspective on treating patients like idiots when really the US doctor is the epitome of fraudulent incompetence.

Over testing is a problem. The way to lessen over testing is to not be an incompetent doctor so that patients don't take things into their own hands. Barring that you have no choice but to let the patient test himself. It's the moral thing to do.

I guess here's an interim solution. Even if the disease is extremely unlikely, if the possibility remains valid based on the given symptoms and the patient requests the test you should allow it. The reason is the patient is more intimately familiar with his own symptoms and your just pattern matching based off of a high level probability. If you've never seen a patient with a certain condition you often bias towards the conditions you've seen.

Yes patients can be biased but so can you. And there's no denying that patients can do more detailed research and they have a perspective with a resolution you will never achieve. The good doctor needs to listen to them.

Post reply on HN