Live data from Hacker News

I misused LLMs to diagnose myself and ended up bedridden for a week

blog.shortround.space

51–60 of 150 posts

Re: I misused LLMs to diagnose myself and ended up bedridden for a week

#51

[flagged]

I think it's morally wrong to trust a child's well-being to an LLM over a trained medical professional, and I feel strongly enough about this to express that here.

I agree with you, and reading some of the takes in this thread is actually blowing my mind.

I cannot believe that the top-voted comment right now is saying to not trust doctors, use an LLM to diagnose yourself and others.

Re: I misused LLMs to diagnose myself and ended up bedridden for a week

#52

Yikes. But I have to say that prompt is crazy bad. AI is VERY good at using your prompt as the basis for the response, if you say "I don't think it's an emergency" AI will write a response that is "it's not an emergency" I did a test with the first prompt and the immediate answer I got was "this looks like lyme disease".

The solution is really easy. Make sure you have web search enabled, you're not using the free version of some AI, and then just ask it to research the best way to prompt, and write a tutorial for you to use in the future. Or have it write some exercises and do a practice chat.

Re: I misused LLMs to diagnose myself and ended up bedridden for a week

#53

Earlier quoted context omitted.

I figured this out diagnosing car trouble. Tried a few separate chats, and my natural response patterns were always leading it down the path to "your car is totaled and will also explode at any moment." Going about it a different way, I got it to suggest a simple culprit that I was able to confirm pretty thoroughly (fuel pressure sensor), and fixed it.

The problem is, once you start down that sequence of the AI telling you what you want to hear, it disables normal critical reasoning. It’s the “yes man” problem— you’re even less able to solve the problem effectively than with no information. I really enjoy LLMs, but it is a bit of a trap.

I hit that too. If I asked it about the O2 sensor, it was the O2 sensor. iirc I had to ask if what PIDs to monitor, give all of that to it at once, then try a few experiments it suggested. It also helped that it told me how to self-confirm by watching that the fuel trim doesn't go too high, which was also my cue to shut off the engine if it did.

At no point was I just going to commit to some irreversable decision it suggested without confirming it myself or elsewhere, like blindly replacing a part. At the same time, it really helped me because I'm too noob to even know what to Google (every term above was new to me).

Re: I misused LLMs to diagnose myself and ended up bedridden for a week

#56
I gave their example “correct” prompt (“Flat, circular, non-itchy, non-painful red rash with a ring, diffuse throughout trunk. Follows week of chills and intense night sweats, plus fatigue and general malaise”) to both ChatGPT and Gemini. And both said Lyme disease as their #1 diagnosis. So maybe it is okay to diagnose yourself with LLMs, just do it correctly!

Re: I misused LLMs to diagnose myself and ended up bedridden for a week

#58

[flagged]

> a child's fractured arm ... Um what? The only way to diagnose a fractured arm is an xray. You can suspect the arm is fractured (rotating it a few directions) but ultimately a muscle injury will feel identical to a fracture especially for a kid. Please, if you suspect a fracture just take your kid to the doctor. Don't waste your time asking ChatGPT if this might be a fracture. This just feels beyond silly to me imag…

[flagged]

Re: I misused LLMs to diagnose myself and ended up bedridden for a week

#59
post #27

I think the author took the wrong lesson here. I've had doctors misdiagnose me just as readily as I've had LLMs misdiagnose me - but I can sit there and plug at an LLM in separate unrelated contexts for hours if I'd like, and follow up assertions with checks to primary sources. That's not to say that LLMs replace doctors, but that neither is perfect and that at the end of the day you have to have your brain turned on…

No, the lesson here is never use an LLM to diagnose you, full stop. See a real doctor. Do not make the same mistake as me

"Don't ask LLMs leading questions" is a perfectly valid lesson here too. If you're going to ask an LLM for a medical diagnosis, you should at the very least know how to use LLMs properly.

I'm certainly not suggesting that you should ask LLM for medical diagnoses, but still, someone who actually understands the tool they're using would likely not have ended up in your situation.

Re: I misused LLMs to diagnose myself and ended up bedridden for a week

#60

Earlier quoted context omitted.

I think it's morally wrong to trust a child's well-being to an LLM over a trained medical professional, and I feel strongly enough about this to express that here.

[flagged]

>How does this line up with your religious belief that doctors are infallible and should be 100% trusted?

They did not say that in their comment.

Replying in obvious bad faith makes your original comment even less credible than it already is.

Post reply on HN