Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used Claude Code to get a second opinion on my MRI

antoine.fi

Post reply on HN
Print view View original

621–630 of 748 posts

Re: I used Claude Code to get a second opinion on my MRI

#621

Post by lucfranken »

yread wrote: post #618
lucfranken wrote: post #581

Earlier quoted context omitted.

That does make sense but the order of it might be: doctor analysis. Then show LLM as double check to doctor. Doctor assesses and may keep en improve the doctors first analysis?

Another problem is that general models' performance just sucks. From an upcoming conf. talk (in pathology) where they ran 2 Medgemma models on 100 slides with known diagnosis: > Results: Full concordance with the reference diagnosis was 8% (27B) and 5% (1.5 4B; McNemar p=0.68), while partial matches were 29% vs 20% respectively (McNemar p=0.053). When correct diagnoses anywhere in the differential were counted, 51% (…

Totally agree on that, if you have to look constantly at something 95% there is no value. The expectation of course is that it will be better. But if not at a certain level useless.

Not sure how that research compares to the claims being made by many that a second opinion via ai in the end led to changes in treatment. Likely people spent quite some time searching and figuring out. That would be a different and n=1 result. Don't have enough knowledge of that research to determine how much result can be gained when the models are managed in a way that produces better results.

And of course how much time/effort/cost that would take. How much is custom and how much is an automated programmable flow.

Re: I used Claude Code to get a second opinion on my MRI

#622

Post by throwaway2037 »

deaux wrote: post #565
themantalope wrote: post #492

Radiologist. I don’t read MR shoulder exams in my day to day practice, but from the few pictures shown , I can’t conclusively disagree with the original report. These models are generally terrible at reading medical images. The amount of public training data on the internet compared to the number of scans a radiologist reads in training is minuscule. There’s obviously a ton of medical images in general but very few,…

Absolutely. It's very unfortunate that this post used the worst example possible of using LLMs for medical purposes. General-purpose LLMs are _fantastic_ at medical diagnosis that do not involve imaging. I am completely convinced that given enough information and time, frontier models already outperform >90% of doctors on initial diagnosis of internal issues and suggesting medical tests to further reject or confirm t…

    > General-purpose LLMs are _fantastic_ at medical diagnosis that do not involve imaging.
Can you share the reasons that you believe this?

    > At the same time, they're worse than useless at anything involving medical imaging.
What is special about medical imaging that makes AI/LLMs specifically bad?

Re: I used Claude Code to get a second opinion on my MRI

#624

Post by bpicolo »

KingMob wrote: post #532
charles_f wrote: post #356

Earlier quoted context omitted.

> It funny to see the community here expects the human body to be treated like a deterministic function In a community largely made of people whose job it is to produce such functions, I'd say it's to be expected

It's funny (and a little depressing), because HN routinely assumes that their world view, and thus, their domain expertise, transfers. There's no shortage of tech people convinced they deeply understand law, medicine, philosophy, etc. despite never having read much on the topics.

The internet at large is full of armchair experts, it's not just a tech thing.

Re: I used Claude Code to get a second opinion on my MRI

#625

Post by jappgar »

voidUpdate wrote: post #619
weird-eye-issue wrote: post #617

Earlier quoted context omitted.

No, it does not work like that, it actually can process the image itself there is not an intermediate image to text step

How does a Large Language Model process images then?

It can only deal in tokens, so you're essentially right that it creates a textual description before describing it back to you. This process is obviously incredibly lossy and details are easily missed

Re: I used Claude Code to get a second opinion on my MRI

#626

Post by weatherlite »

mixologic wrote: post #27
weatherlite wrote: post #13

Earlier quoted context omitted.

Nightmare because they're always right and the A.I second guessing is always wrong, or because they just don't like to be second guessed?

Its a nightmare because it erodes trust. Doctors are not "always right" which is why "always get a second opinion" is codified in culture. But AI's problem is that its completely full of shit, sometimes, and the people most qualified to evaluate whether its full of shit are the doctors, not the patients, but just like OP's original article, patients are left feeling like their second opinion from AI might be more tru…

> But AI's problem is that its completely full of shit, sometimes

It's now quite unusual that it's "Completely full of shit". If it contradicts something your doctor said I don't see why you should feel ashamed to bring it up. Sure it complicates the doctor's work, having ignorant obedient patients must be more comfortable for the doctor, but the end result could be more accurate diagnosis.

Re: I used Claude Code to get a second opinion on my MRI

#627

Post by ferfumarma »

jeswin wrote: post #31

I would not trust AI on images. But I once had ChatGPT tell me that an MRI report was very likely to be incorrect based on the text, and offered a different diagnosis. Since it was semi insisting, I visited another doctor who made me do a retest. Long story short, ChatGPT was correct. Again, this is just one single person's experience. So not worth much.

This sounds fascinating. Can you provide any detail regarding the nature of the diagnosis or problem it identified?

Re: I used Claude Code to get a second opinion on my MRI

#629

Post by jbs789 »

AceJohnny2 wrote: post #117

> There's something incredibly peaceful about being in the hands of an expert you trust. [...] AI can absolutely shatter that feeling in an uncomfortable way [...] but I don't know if I can fully trust AI either. This really is key. We know we can't trust the AI, but at the same time we're also more comfortable asking the AI for clarifications or confronting it. Not having a time-bound appointment or paying by the ho…

Especially in the medical field where the placebo effect / mindset shapes outcomes.

Re: I used Claude Code to get a second opinion on my MRI

#630

Post by 1970-01-01 »

Seems like we've forgotten something already? LLMs are bad with the butterfly effect by their very nature. An extremely slight phrasing change can absolutely trigger a deterministic output decision that is out of touch with reality. Tuning only helps if there is training from a previous input. As far as I know (casual research), there still isn't a great solution to this, only marketing hype.
Post reply on HN