Live data from Hacker News

ChatGPT terms disallow its use in providing legal and medical advice to others

ctvnews.ca

251–260 of 439 posts

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#251

Earlier quoted context omitted.

I’ve found it just as accurate and a better experience than using telehealth or generalist doctors

You can’t possibly have enough data to support that statement.

there was no mention of sample size, though, so the statement might be true for the commenter but not widely applicable, to your point

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#252
post #226

Earlier quoted context omitted.

I assure you the medical profession is not generally open to non-medical professionals using AI for medical purposes.

The medical profession is not open about any kind of self diagnosis. I've learned through experience that telling a doctor "I have X and I would like to be treated with Y" is not a good idea. They want to be the ones who came up with the diagnosis. They need to be the smartest person in the room. In fact I've had doctors go in a completely different direction just to discredit my diagnosis. Of course in the end I was…

If I was an airline pilot, I'm not going to listen to a passenger telling me which route I should be taking.

This is not about ego or trying to be the smartest person in the room, it's about actually being the most qualified person in the room. When you've done medical school, passed the boards, done your residency and have your own private practice, only then would I expect a doctor to care what you think a correct diagnosis is.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#253

That’s a fair limitation. Legal and medical advice can directly impact someone’s life or safety, so AI tools must stay within ethical and regulatory boundaries. It’s better for AI to guide people toward professional help than pretend to replace it.

You had me right up until ‘AI tools must stay within ethical and regulatory boundaries’. I guarantee you any AI LLM company which cares about ethics is destined to fail, because none of their peers do.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#254

I've used ChatGPT to help understand medical records too. It's definitely faster than searching everything on my own, but whether the information is reliable still depends on personal judgment or asking a real doctor. More people are treating it like a doctor or lawyer now, and the more it's used that way, the higher the chance something goes wrong. OpenAI is clearly drawing a line here. You're free to ask questions,…

You should not be feeding your medical records into ChatGPT.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#255

Earlier quoted context omitted.

I’ve found it just as accurate and a better experience than using telehealth or generalist doctors

If you're not a doctor, how do you know it's accurate? This is the huge problem with using LLMs for this kind of thing. How do you verify that it is better? What is the ground truth you are testing it against? If you wanted to verify that ChatGPT could do math, you'd ask it 100 math problems and then (importantly) verify its answers with a calculator. How do you verify that ChatGPT can interpret medical information w…

Nobody would use these services for anything important if they _actually understood_ these glorified markov chains are just as likely to confidently assert something false and lie about it when pressed as they are to produce accurate information.

These AI companies have sold a bill of goods but the right people are making money off it so they’ll never be held responsible in a scenarios the one you described.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#256

Earlier quoted context omitted.

I’ve found it just as accurate and a better experience than using telehealth or generalist doctors

If you're not a doctor, how do you know it's accurate? This is the huge problem with using LLMs for this kind of thing. How do you verify that it is better? What is the ground truth you are testing it against? If you wanted to verify that ChatGPT could do math, you'd ask it 100 math problems and then (importantly) verify its answers with a calculator. How do you verify that ChatGPT can interpret medical information w…

I know it’s hard to accept, but it’s got to be weighed against the real-world alternative:

You get about 5-8 minutes of face time with an actual doctor, but have to wait up to weeks to actually get in to see a doctor, except maybe at an urgent care or ER.

Or people used to just play around on WebMD which was even worse since it wasn’t in any way tailored to what the patient’s stated situation is.

There’s the rest of the Internet too. You can also blame AI for this part, but today the Internet in general is even more awash in slop that is just AI-generated static BS. Like it or not, the garbage is there and it will be most of what people find on Google if they couldn’t use a real ChatGPT or similar this way.

Against this backdrop, I’d rather people are asking the flagship models specific questions and getting specific answers that are halfway decent.

Obviously the stuff you glean from the AI sessions needs to be taken to a doctor for validation and treatment, but I think coming into your 5-minute appointment having already had all your dumbest and least-informed ideas and theories shot down by ChatGPT is a big improvement and helps you maximize your time. It’s true the people shouldn’t recklessly attempt to self-treat based on GPT, but the unwise people doing that were just self-treating based off WebMD hunches before.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#257
post #73

Things like this really favor models offered from countries that have fewer legal restrictions. I just don't think it's realistic to expect people not to have access to these capabilities. It would be reasonable to add a disclaimer. But as things stand I think it's fair to consider talking to ChatGPT to be the same as talking to a random person on the street, meaning normal free-speech protections would apply.

What capabilities? The article says the study found it was entirely correct 31% of the time.

This is what really scares me about people using AI. It will confidently hallucinate studies and quotes that have absolutely no basis in reality, and even in your own field you're not going to know whether what it's saying is real or not without following up on absolutely every assertion. But people are happy to completely buy its diagnoses of rare medical conditions based on what, exactly?

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#258
post #239

I read this not as "you can't ask ChatGPT about {medical or legal topic}" but rather "you can't build something on ChatGPT that provides {medical or legal} advice or support to someone else." For example, Epic couldn't embed ChatGPT into their application to have it read your forms for you. You can still ask it - but Epic can't build it. That said, I haven't found the specific terms and conditions that are mentioned…

And this seems actually wildly reasonable. It’s actually pretty scammy to take people’s money (or whatever your business model is) for legal or medical advice just to give them whatever AI shits out. If I wanted ChatGPT’s opinion (as I actually often do) I’d just ask ChatGPT for free or nearly-free. Anyone repackaging ChatGPT’s output as fit for a specialized purpose is scamming their customers.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#259

That’s a fair limitation. Legal and medical advice can directly impact someone’s life or safety, so AI tools must stay within ethical and regulatory boundaries. It’s better for AI to guide people toward professional help than pretend to replace it.

Add financial advice to it, too. Really, any advice. Why the fuck are people asking a probabilistic plagiarizing machine for advice on anything ? This is total insanity!

People aren’t more reliable either.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#260
post #73

Things like this really favor models offered from countries that have fewer legal restrictions. I just don't think it's realistic to expect people not to have access to these capabilities. It would be reasonable to add a disclaimer. But as things stand I think it's fair to consider talking to ChatGPT to be the same as talking to a random person on the street, meaning normal free-speech protections would apply.

What capabilities? The article says the study found it was entirely correct 31% of the time.

Does it say how often doctors are correct as a baseline?
Post reply on HN