Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards accurate differential diagnosis with large language models

arxiv.org

Post reply on HN
Print view View original

61–70 of 99 posts

Re: Towards accurate differential diagnosis with large language models

#61

Post by intended »

corethree wrote: post #56
staunton wrote: post #48

Earlier quoted context omitted.

> statements like this literally set me off. The patient being right and the doctor ignoring the patient and treating the patient like an idiot There's a huge number of people who vastly overestimate their medical knowledge, google some symptoms (I'm sure you know the pitfalls of that but many don't), do self-diagnosis as a hobby with friends over coffee, come to absurd conclusions and then bother their doctors about…

Given the amount paid by the patient to the doctor, the doctor is honor bound to listen to the patient, to consider what he said. This is not a trivial amount of money that the patient is forking over to the doctor. It's understandable from the standpoint of irritation, if the doctor is presented with the same crack pot theories all the time then it's understandable how he would ignore such theories. It's not underst…

If the doctor was free, would any of these things change?

Re: Towards accurate differential diagnosis with large language models

#62

Post by corethree »

intended wrote: post #61
corethree wrote: post #56

Earlier quoted context omitted.

Given the amount paid by the patient to the doctor, the doctor is honor bound to listen to the patient, to consider what he said. This is not a trivial amount of money that the patient is forking over to the doctor. It's understandable from the standpoint of irritation, if the doctor is presented with the same crack pot theories all the time then it's understandable how he would ignore such theories. It's not underst…

If the doctor was free, would any of these things change?

A patient isn't just damaged by bad treatment. He's damaged by medical costs. He's damaged by the very thing that makes doctors rich. And often given in return incompetency. Bled dry by "experts" holding their lives hostage.

https://www.cnn.com/2023/01/31/health/us-health-care-spendin...

It's disgusting.

Oh and check this:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6139931/#:~:tex....

It's not just incompetence. Trustworthiness is also an issue.

Re: Towards accurate differential diagnosis with large language models

#63

Post by intended »

resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

I’m very glad you set the argument this way, since it very neatly sets the stage for the core disagreement to be argued.

> Simply very good information synthesizers

> Simple very good syntactic synthesizers

I argue that theorists tend to adhere to the former, and practitioners the latter.

Consider prototyping: For an expert, it costs nothing (or near nothing) to check the output. You even start better, because you know how to ask the right questions to the tool in the first place.

LLMs dont get the semantics right, they get syntactic correlations right. When a subject is common place (literature), the correlations are good enough to get the reasoning right.

There is an IMPRESSIVE amount that gets done with just this. However, expecting to get reasoning right is the bridge too far.

Wrong expectations of tools, result in bad management projects and failures.

You can test this out right now. IF an LLM is able to do information synthesis, it is trivial to set up parallel prompts to work as teams. Try it out.

I did, I know why it can’t work as a result. The simple question to ask is “whats your error rate, and your hallucination rate?”

LLMs are always ‘hallucinating’.

I do hope, that the difference eventually becomes academic. That there is so much training material, that correlation is equal to reasoning. However, it will not be reasoning/semantic prediction.

Finally - the real world work has properties that are emergent. You can read every spec sheet you like, but when you start assembling components together, they are going to do weird things.

Re: Towards accurate differential diagnosis with large language models

#64

Post by rrsp »

aix1 wrote: post #28
rrsp wrote: post #4

I wonder if they made any effort to check whether the NEJM case studies that this whole study is based are in the PALM-2 training dataset.

Did you see section 6.5 Contamination Analysis?

No I missed it, was looking in the methods section as opposed to the results, thank you!

Re: Towards accurate differential diagnosis with large language models

#65

Post by evgen »

corethree wrote: post #54
acover wrote: post #42

Earlier quoted context omitted.

You can make the right call but still be wrong sometimes. There's a cost to tests of rare diseases.

Then let the person who is wrong foot the cost. The cost isn't just the cost of the test. It's the cost of what occurred because the doctor decided to block a test.

No one is preventing you from going to a private clinic and paying for the test yourself. The doctor is not 'blocking' the test or preventing you from getting it, they are just not authorising it. Sounds like you have a problem with either your insurance provider or your wallet.

Re: Towards accurate differential diagnosis with large language models

#66

Post by RandomLensman »

resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

The real mental work is when your client, patient, etc. struggles to articulate the problems, data is wrong or mislabeled, etc.

Re: Towards accurate differential diagnosis with large language models

#67

Post by amluto »

evgen wrote: post #65
corethree wrote: post #54

Earlier quoted context omitted.

Then let the person who is wrong foot the cost. The cost isn't just the cost of the test. It's the cost of what occurred because the doctor decided to block a test.

No one is preventing you from going to a private clinic and paying for the test yourself. The doctor is not 'blocking' the test or preventing you from getting it, they are just not authorising it. Sounds like you have a problem with either your insurance provider or your wallet.

Or the jurisdiction in which you live in which labs are not permitted to test without a doctor’s order? This is very common.

Re: Towards accurate differential diagnosis with large language models

#69

Post by mda »

bradknowles wrote: post #47
doix wrote: post #26

This doesn't really surprise me given my experience with doctors. If it's a common problem then doctors do fine. As soon as you're not the common case or have something somewhat rare, it becomes a bit of a crap shoot, at least in the UK when you deal primarily with your GP (general practitioner). I have a friend with Crohn's who was feeling low energy. I was a gym bro at the time and convinced him to take a testoster…

I have hypertension, as well as diabetes. I wanted my cardiologist to test me for hyperaldosteronism, because I had read it was not uncommon and could be a factor in hypertension that continues to get worse and worse, without an obvious explanation. He refused, saying that it was the job of my endocrinologist to do that test. So, I talked to my endocrinologist and he said it wasn't a normal test they do, but he was h…

It seems they usually test for hyperaldosteronism if they see the obvious signs like low potassium levels. Maybe they did not observe this at the beginning?

Re: Towards accurate differential diagnosis with large language models

#70

Post by IanCal »

intended wrote: post #63
resters wrote: post #5

If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…

I’m very glad you set the argument this way, since it very neatly sets the stage for the core disagreement to be argued. > Simply very good information synthesizers > Simple very good syntactic synthesizers I argue that theorists tend to adhere to the former, and practitioners the latter. Consider prototyping: For an expert, it costs nothing (or near nothing) to check the output. You even start better, because you kn…

> IF an LLM is able to do information synthesis, it is trivial to set up parallel prompts to work as teams. Try it out.

What do you mean? I've had GPT-4 (3.5 I think too) talking to itself, setup as a planner and someone critiquing a plan, the end result is much better.

> However, expecting to get reasoning right is the bridge too far.

Hmm, how do you position othello-gpt in this? It builds a world model and makes moves based on that, so it's not just correlation of input (x,y,z typically followed by A so respond A).

Post reply on HN