Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used Claude Code to get a second opinion on my MRI

antoine.fi

Post reply on HN
Print view View original

291–300 of 748 posts

Re: I used Claude Code to get a second opinion on my MRI

#291

Post by xivzgrev »

rafterydj wrote: post #15

I feel like I'm going nuts. There are other commenters saying this is a good practice they've also done for other injuries. You are saying you are an actual radiologist and immediately clock the problems with its advice. I have seen this pattern over and over again. Anytime someone is an actual expert at anything, AI output appears insufficient or incomplete or outright misleading. It is only when you do not know wha…

Well that's part of the problem. AI is not accountable - if you take its advice and hurt yourself, who is responsible?

A real doctor is accountable.

They might both "know" a lot of things but implicitly the party who is accountable is going to be more trustworthy.

And I don't see that going away until AI companies must be licensed for application x and can lose their license / be sued if engaging in malpractice.

Re: I used Claude Code to get a second opinion on my MRI

#292

Post by aspenmartin »

Analemma_ wrote: post #177
brokencode wrote: post #162

Earlier quoted context omitted.

I seriously doubt that data set poisoning will be a real limiter in model performance. For one, if your website/book is poisoned, who is going to trust it for anything at all, much less for training models? For two, all the major AI labs hire or contract for subject matter experts to create curated data sets, evaluate model performance, etc. Unless they hire malicious experts, this will provide a growing, high qualit…

I think you underestimate just how much money is being poured into LLM SEO at the moment. It's real quiet because they don't want to draw attention and countermeasures from the frontier labs, but this is getting huge investment, and they will have a monomaniac focus on juicing product results whereas the attention of the labs necessarily has to be spread out.

Data curation is important and expensive and frontier labs can afford to do it right. Natural data isn't the limitation, we are already literally out of tokens. It doesn't matter how much you poison things it's not going to stop the progress train.

Re: I used Claude Code to get a second opinion on my MRI

#293

Post by jurgenaut23 »

chmod775 wrote: post #248
brokencode wrote: post #162

Earlier quoted context omitted.

I seriously doubt that data set poisoning will be a real limiter in model performance. For one, if your website/book is poisoned, who is going to trust it for anything at all, much less for training models? For two, all the major AI labs hire or contract for subject matter experts to create curated data sets, evaluate model performance, etc. Unless they hire malicious experts, this will provide a growing, high qualit…

There's a post every other month where some dude who put nonsense information online celebrates because it actually ended up in some frontier models weights. If it's easy enough that some randos can do it for fun, what do you think happens when there's commercial interest behind it? Obviously companies are going try nudging AI towards recommending whatever they're selling. It's a logical extension of SEO - and that's…

Do you have examples of such celebrations?

Re: I used Claude Code to get a second opinion on my MRI

#294

Post by aspenmartin »

qnleigh wrote: post #137
rafterydj wrote: post #15

I feel like I'm going nuts. There are other commenters saying this is a good practice they've also done for other injuries. You are saying you are an actual radiologist and immediately clock the problems with its advice. I have seen this pattern over and over again. Anytime someone is an actual expert at anything, AI output appears insufficient or incomplete or outright misleading. It is only when you do not know wha…

Totally agree. I'm a scientist, and like most scientists I have some specialized skills that most of my colleages don't. AI has empowered them to learn and build things that they might have otherwise needed me for. But there have been quite a few cases where it led them very far down a wrong path. This has started happening way more often in the last few months.* We've known since the beginning that AIs confidently s…

Right but hallucination rates have been consistently decreasing every model iteration. It's about error rates. As also a fellow scientist, I also will mess something up. Humans have an error rate. Once that error rate is low enough, it doesn't matter that it's > 0, it matters that it's low enough to be trustworthy and useful. Coding agents of 2024-25 had error rates too large; you couldn't meaningfully vibe code anything and needed a ton of oversight. It's still true but FAR less so, and this is after like a year of iteration.

Re: I used Claude Code to get a second opinion on my MRI

#295

Post by piterrro »

It funny to see the community here expects the human body to be treated like a deterministic function: for input X expect output Y - and that transfers to diagnosis - people expect to receive the same diagnosis from different specialists for the same issue.

Given human body complexity, the diagnosis is a compound output of the experience, knowledge gained throughout the career and diagnosis methods/equipment, the title (like Dr) is a certification imposed by the state so its "safe" to let people practice since they passed "the bar" - but that doesn't imply everyone will be treating the same.

Some specialists update their knowledge monthly, some yearly and some don't do it at all, there are so many variables in play here (geo, politics, even weather haha).

Having said that, choosing the specialist is really important, getting opinions about their practice and their speciality, you can only maximize your chance of getting the right diagnosis, but don't expect to get it right just because somebody is called a Dr.

Re: I used Claude Code to get a second opinion on my MRI

#296

Post by lukeinator42 »

cube00 wrote: post #85
hennell wrote: post #53

Personally my favourite feature of the new ai world is not when I use it directly but it's when one of my managers uses it to try to fix a problem, then issue to me their findings and I have to defend my process to someone who understands neither my process, their suggested solution nor often the problem they're solving in the first place.

It gets worse when they challenge your solutions by feeding it back into the LLM and sending the response on to you, arguing with an LLM is exhausting, arguing by proxy with a human parroting its responses is excruciating. On the plus side when they do this they can't flood your calendar with those "quick chat" meetings because they know they won't be able to hold a conversation on the issue beyond the first minute.

This happened to me on a paper I submitted recently where it was clear the reviewers used AI. Revising a paper based on LLM review is also exhausting, haha.

Re: I used Claude Code to get a second opinion on my MRI

#297

Post by ethbr1 »

kjellsbells wrote: post #259
something98 wrote: post #154

Earlier quoted context omitted.

This is a very misleading statement; most of those physicians are using LLMs to transcribe notes from visits and/or for billing purposes (e.g., proper billing codes).

The problems isnt LLMs per se, it is the shift to trusting the output of the machine coupled with a decline in verifying that the output is reasonable. It's basically what your teachers warned you about with wikipedia in eight grade except applied to all areas of life, including medicine. Dictation is already high-stakes and LLMs do not automatically reduce that risk. Here is an example. My provider sent me this note…

> Again the problem is not dictation, or LLMs. The problem is humans ignoring their responsibility to check the output of a machine.

100%. Also, management.

I wish someone would go ahead and coin an AI version of Amdahl's law that states the work speedup from AI is dependent on amount of unverified AI output used.

Iow, if you 1:1 verified everything, there would be no time savings.

Ergo, you get management saying (1) we demand time savings due to AI & (2) we demand you fully check anything you use AI for.

End result? People skip (2) to hit (1).

Then management burns anyone at the stake whenever inevitable mistakes happen.

Re: I used Claude Code to get a second opinion on my MRI

#298

Post by aspenmartin »

operatingthetan wrote: post #286
aetherson wrote: post #279

Earlier quoted context omitted.

You're confusing the training method with the internal process. If I had you repeatedly attempt to learn how to make believable completions of partial documents about a given topic, you would eventually learn things about that topic and could use your knowledge to create more believable completions of documents about that topic.

LLMs do not learn. You put it out to pasture and create a new one. "Memory" in a session is essentially a context window party trick.

They do learn in context, and very sample efficiently. Continual learning is active area of research and we sort of already have something resembling it with persistent context. So yes they do learn.

Re: I used Claude Code to get a second opinion on my MRI

#299

Post by operatingthetan »

aspenmartin wrote: post #298
operatingthetan wrote: post #286

Earlier quoted context omitted.

LLMs do not learn. You put it out to pasture and create a new one. "Memory" in a session is essentially a context window party trick.

They do learn in context, and very sample efficiently. Continual learning is active area of research and we sort of already have something resembling it with persistent context. So yes they do learn.

I consider that to be the illusion of learning. You are not wrong, I think they may actually learn in the future though. But not today.

Re: I used Claude Code to get a second opinion on my MRI

#300

Post by singpolyma3 »

operatingthetan wrote: post #220
singpolyma3 wrote: post #210

Earlier quoted context omitted.

That presumes that we have a definition of "thinking" or that we know that anything is "thinking" when in fact neither is true. The problem is real but I don't think positing a philosophical root is helpful

The claim that we are assigning human-like agency to a machine with none is simple and factual.

The idea that humans have agency is supernatural thinking imo
Post reply on HN