Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used Claude Code to get a second opinion on my MRI

antoine.fi

Post reply on HN
Print view View original

281–290 of 748 posts

Re: I used Claude Code to get a second opinion on my MRI

#281

Post by operatingthetan »

ForceBru wrote: post #252
operatingthetan wrote: post #220

Earlier quoted context omitted.

The claim that we are assigning human-like agency to a machine with none is simple and factual.

What's "thinking"? What's "agency"? What's "human-like agency"? If "agency" is making decisions and performing corresponding actions in the real world, then LLMs most definitely LOOK LIKE they're making decisions (what's the next token? which tool to use? what's to say, in general? what idea to convey?) and performing actions (tool use). Can we tell whether they are ACTUALLY making decisions? Well, are the people aro…

I think you're mildly obfuscating the issues at hand by diving too deeply into philosophical questions.

It's quite simple, the agency that the LLM appears to have is actually your own. Without a prompt an LLM does nothing. It has no thoughts between prompts about you or your problems.

Re: I used Claude Code to get a second opinion on my MRI

#282

Post by natebc »

shimman wrote: post #272
janmatejka wrote: post #205

Earlier quoted context omitted.

Because some people develop actual psychosis. They go down some rabbit hole with an LLM until the LLM makes them believe they invented new kind of physics that makes them go harassing experts who obviously try to ignore them because its all nonsense.

Reminds me of that clip of Travis Kalanick, sexual deviant and harasser of women, talking about "discovering new physics."

The Uber guy? Yeah that was a painful watch.

Re: I used Claude Code to get a second opinion on my MRI

#284

Post by natebc »

tayo42 wrote: post #277
Analemma_ wrote: post #177

Earlier quoted context omitted.

I think you underestimate just how much money is being poured into LLM SEO at the moment. It's real quiet because they don't want to draw attention and countermeasures from the frontier labs, but this is getting huge investment, and they will have a monomaniac focus on juicing product results whereas the attention of the labs necessarily has to be spread out.

Who's doing llm seo right now? How does that work when you only gets feedback every few months when a new model is out?

I'm pretty sure the Optimization part is just ... not present at all.

This is how we get LLM summaries presenting something mentioned once by some nutjob in a reddit thread as bona fide FACT

Re: I used Claude Code to get a second opinion on my MRI

#285

Post by perching_aix »

EA-3167 wrote: post #72
whatever1 wrote: post #49

Earlier quoted context omitted.

So if you want to do a surgery but you don’t see any surgeons around you ask a grocery butcher to have his way?

People, especially in medical crises, are desperate for answers that they often can't get because their clinicians don't know. The illusion of an all-knowing guru who sounds like their doctor and tells them ANYTHING is extremely alluring. If you're waiting to hear back from a doctor about test results (which these days probably showed up on your online account the moment they were completed) can be agonizing. Ok for…

The companies are 100% aware, yes, and so they did make quite a few changes over the years.

Current trend is that the models will try to explicitly steer you towards "asking better questions from your medical provider", rather than providing diagnoses. They do also evaluate whether something can actually be established rather than just listen and nod along. And so the "you must have very little faith in the intelligence of their members" goes right back against these failure mode ideas.

Now of course, given a sufficiently desperate person, they can probably torture anything they want to hear out of these models. But so can they out of actual people, so that's kind of a high bar. When you get to the point where people are willfully misreading a given piece of text, bets tend to be rather off.

Re: I used Claude Code to get a second opinion on my MRI

#286

Post by operatingthetan »

aetherson wrote: post #279
operatingthetan wrote: post #193

Earlier quoted context omitted.

>This is the root of AI psychosis. There’s a lot of unpack here, and I won’t go too deep because you can’t really have a discussion with affected folks because their fundamental basis is not evidence, it’s belief. Treating it as if it is an intelligence is the problem. The problem is that AI psychosis is fundamentally the belief that an LLM is "thinking" at all. Outputs are just believable word vomit which resembles…

You're confusing the training method with the internal process. If I had you repeatedly attempt to learn how to make believable completions of partial documents about a given topic, you would eventually learn things about that topic and could use your knowledge to create more believable completions of documents about that topic.

LLMs do not learn. You put it out to pasture and create a new one. "Memory" in a session is essentially a context window party trick.

Re: I used Claude Code to get a second opinion on my MRI

#287

Post by Esophagus4 »

Aurornis wrote: post #167
AceJohnny2 wrote: post #117

> There's something incredibly peaceful about being in the hands of an expert you trust. [...] AI can absolutely shatter that feeling in an uncomfortable way [...] but I don't know if I can fully trust AI either. This really is key. We know we can't trust the AI, but at the same time we're also more comfortable asking the AI for clarifications or confronting it. Not having a time-bound appointment or paying by the ho…

I have multiple LLM subscriptions at any given time, plus an array of local models. When I ask a question outside of my domain of expertise I like to ask all of the LLMs I have access to. I also create separate sessions and ask the same question multiple ways. It’s revealing to see how many different and contradictory answers I get, most of which are presented confidently. The last time I ran a medical question throu…

Have you ever let the LLMs “discuss” with each other to see if that would give better answers?

You might end up with the answer from the most persuasive LLM, but you might also end up with better results.

Wonder if there is a paper out there on this.

Re: I used Claude Code to get a second opinion on my MRI

#288

Post by operatingthetan »

corndoge wrote: post #255
operatingthetan wrote: post #193

Earlier quoted context omitted.

>This is the root of AI psychosis. There’s a lot of unpack here, and I won’t go too deep because you can’t really have a discussion with affected folks because their fundamental basis is not evidence, it’s belief. Treating it as if it is an intelligence is the problem. The problem is that AI psychosis is fundamentally the belief that an LLM is "thinking" at all. Outputs are just believable word vomit which resembles…

Often times the words produced do have legitimate factual information though. It's less psychosis and more a confluence of well known human tendencies - salience bias, automation bias, etc.

The big problem is often times they don't as well. That's why we can't rely on them.

Re: I used Claude Code to get a second opinion on my MRI

#289

Post by DougN7 »

tayo42 wrote: post #277
Analemma_ wrote: post #177

Earlier quoted context omitted.

I think you underestimate just how much money is being poured into LLM SEO at the moment. It's real quiet because they don't want to draw attention and countermeasures from the frontier labs, but this is getting huge investment, and they will have a monomaniac focus on juicing product results whereas the attention of the labs necessarily has to be spread out.

Who's doing llm seo right now? How does that work when you only gets feedback every few months when a new model is out?

Look at G2.com - they found their website is highly references by AIs and they are leaning into it hard.

Re: I used Claude Code to get a second opinion on my MRI

#290

Post by aspenmartin »

chmod775 wrote: post #248
brokencode wrote: post #162

Earlier quoted context omitted.

I seriously doubt that data set poisoning will be a real limiter in model performance. For one, if your website/book is poisoned, who is going to trust it for anything at all, much less for training models? For two, all the major AI labs hire or contract for subject matter experts to create curated data sets, evaluate model performance, etc. Unless they hire malicious experts, this will provide a growing, high qualit…

There's a post every other month where some dude who put nonsense information online celebrates because it actually ended up in some frontier models weights. If it's easy enough that some randos can do it for fun, what do you think happens when there's commercial interest behind it? Obviously companies are going try nudging AI towards recommending whatever they're selling. It's a logical extension of SEO - and that's…

It's not really a problem. We're out of natural tokens anyway. The future is synthetic verifiable traces (already the way we train coding agents).
Post reply on HN