Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used Claude Code to get a second opinion on my MRI

antoine.fi

Post reply on HN
Print view View original

521–530 of 748 posts

Re: I used Claude Code to get a second opinion on my MRI

#521

Post by [deleted] »

Paracompact wrote: post #360
ambicapter wrote: post #227

Earlier quoted context omitted.

> So an ultrasound report can state there are no calcifications while a plain radiograph can report the presence of calcifications without being inconsistent. Obviously very confusing to patients and people unfamiliar with medical jargon This is being overly nice, I think. Anyone who doesn't understand this is an idiot imo. You would have to assume that every type of diagnosis instrument has infinite clarity and is a…

"On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

[deleted]

Re: I used Claude Code to get a second opinion on my MRI

#523

Post by marcta »

sizzle wrote: post #509
scheme271 wrote: post #317

Earlier quoted context omitted.

The problem is how do you know whether the answer is just the most persuasive or actually the most accurate one? It's hard to figure this out without domain knowledge.

Take the output to a Radiologist and verify the veracity of the statements.

Why should a radiologist have to debunk AI slop? They have enough to do already. That's the same mentality that is frustrating open-source repositories with sloppy pull requests, and saying "here, sort this out for me".

Re: I used Claude Code to get a second opinion on my MRI

#524

Post by adamddev1 »

marcus_holmes wrote: post #481
Aurornis wrote: post #167

Earlier quoted context omitted.

I have multiple LLM subscriptions at any given time, plus an array of local models. When I ask a question outside of my domain of expertise I like to ask all of the LLMs I have access to. I also create separate sessions and ask the same question multiple ways. It’s revealing to see how many different and contradictory answers I get, most of which are presented confidently. The last time I ran a medical question throu…

In my day job we tried creating a credit assessor tool using LLM as the credit assessor. It did great, generated a report on the assessed business that was incredibly detailed and plausible. Then I started running tests and getting into the details, and found that if you ran the same report on the same data, it generated completely different, still very plausible, results. I could run the same source data through the…

Linters and tests help of course, but they cannot "fix" the problem since tests cannot prove the absence of bugs.

Re: I used Claude Code to get a second opinion on my MRI

#525

Post by otabdeveloper4 »

dirkt wrote: post #520
Aurornis wrote: post #167

Earlier quoted context omitted.

I have multiple LLM subscriptions at any given time, plus an array of local models. When I ask a question outside of my domain of expertise I like to ask all of the LLMs I have access to. I also create separate sessions and ask the same question multiple ways. It’s revealing to see how many different and contradictory answers I get, most of which are presented confidently. The last time I ran a medical question throu…

What happened to VERIFYING an answer? Does nobody do that anymore? When I ask an LLM, I trace the sources, and see if they make sense. More often than not the sources don't actually say anything about the topic in particular... > It’s also scary how easily you can lead each LLM to the answer you have in mind. Exactly. Which is why "treat an LLM like a human expert who can answer your question" doesn't work. It's more…

> When I ask an LLM, I trace the sources, and see if they make sense.

Professional tip: you can cut out the LLM middleman here and save a lot of time and money.

Re: I used Claude Code to get a second opinion on my MRI

#526

Post by make3 »

themantalope wrote: post #492

Radiologist. I don’t read MR shoulder exams in my day to day practice, but from the few pictures shown , I can’t conclusively disagree with the original report. These models are generally terrible at reading medical images. The amount of public training data on the internet compared to the number of scans a radiologist reads in training is minuscule. There’s obviously a ton of medical images in general but very few,…

[flagged]

Re: I used Claude Code to get a second opinion on my MRI

#527

Post by marcus_holmes »

adamddev1 wrote: post #524
marcus_holmes wrote: post #481

Earlier quoted context omitted.

In my day job we tried creating a credit assessor tool using LLM as the credit assessor. It did great, generated a report on the assessed business that was incredibly detailed and plausible. Then I started running tests and getting into the details, and found that if you ran the same report on the same data, it generated completely different, still very plausible, results. I could run the same source data through the…

Linters and tests help of course, but they cannot "fix" the problem since tests cannot prove the absence of bugs.

agree, and I think we'll see more use of formal methods with LLMs for this reason

Re: I used Claude Code to get a second opinion on my MRI

#528

Post by mncharity »

igortg wrote: post #369
kgeist wrote: post #341

A few years ago (before the AI craze), I was misdiagnosed with tuberculosis. I had a chronic cough, and an outsourced radiologist at a clinic found signs of tuberculosis. The findings were sent to the city's tuberculosis hospital, as required by the country's law. The doctors there took the radiologist's conclusion at face value and required me to stay at their hospital for at least 8 months under a strict, prison-li…

I had a similar experience. My son had pneumonia and was still filling pain after 10 days of antibiotics. Took an X-Ray to three different doctors, and only one got the right diagnosis (pleural effusion). It's really something we should have a central place with top notch professionals looking at it, instead having each doctor to find by themselves.

I once worked on a medical hackathon concept for computer-assisted population screening for cervical cancer in a developing nation. Community health workers take photos. The AI would look at the images, and make a call of "clearly negative" vs "clearly positive" vs "needs (scarce) expert review". But taking good photos is hard, so it's also "photos insufficient" and "worker needs additional mentorship on taking photos". Only by computes reducing all three costs - expert workload, exam success, and quality-control/training - might successful deployment be financially and logistically plausible for that nation.

Re: I used Claude Code to get a second opinion on my MRI

#529

Post by ryukoposting »

AceJohnny2 wrote: post #117

> There's something incredibly peaceful about being in the hands of an expert you trust. [...] AI can absolutely shatter that feeling in an uncomfortable way [...] but I don't know if I can fully trust AI either. This really is key. We know we can't trust the AI, but at the same time we're also more comfortable asking the AI for clarifications or confronting it. Not having a time-bound appointment or paying by the ho…

> I got 3 completely unrelated recommendations, including one that I knew was invalid! I felt worse off than when I started!

I almost had a very similar experience with my beater Lexus. It took 2 independent shops and 3 dealers to finally figure out what was causing the ABS to go off randomly at low speeds. Turns out there's some obscure Toyota-specific tool from the late '90s that picked up a proprietary diagnostic code, and the third dealer was the only one that still had that particular piece of equipment.

...and of course, the thing that's broken has been out of production for 20 years and remanufactured ones cost more than the car is worth. I ended up just unplugging the ABS control module.

Point being: once I knew what was wrong, all the seemingly contradictory information from the other 4 shops suddenly fit together. It's just such a weird thing to go wrong that no reasonable tech would ever have considered it.

Re: I used Claude Code to get a second opinion on my MRI

#530

Post by ytoawwhra92 »

nkrisc wrote: post #437
MattyMc wrote: post #425

Earlier quoted context omitted.

> Anyone who doesn't understand this is an idiot imo I don’t think that’s true. Avoiding this mistake requires knowing that an ultrasound may not detect calcification. For a patient reading their own report, I don’t think that’s intuitive. I would expect most people to read “no calcifications” and assume that their joint has no calcifications.

Most people should have learned at a young age that absence of evidence is not evidence of absence. My 8 year old understands this. After all, you can rarely ever prove something does not exist, only that it is unlikely to exist. If a report states that X was not found, it does not mean X did not exist, it means it was not found. What may be lost on the layperson is the nuance and understanding of how thorough or not…

"Calcifications not found" is a different statement from "no calcifications".

Even then, the context that "ultrasound isn't a great way to assess for calcification" is important when reading either statement. Laypeople don't necessarily have that context.

Post reply on HN