Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parents say ChatGPT got their son killed with bad advice on party drugs

theverge.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#21

Post by Wowfunhappy »

OneDeuxTriSeiGo wrote: post #18
Wowfunhappy wrote: post #16

Earlier quoted context omitted.

But ChatGPT doesn’t claim to have a medical license! You can give people whatever terrible medical advice you want—and people absolutely do—you just can’t claim to be a doctor!

> You can give people whatever terrible medical advice you want, you just can’t claim to be a doctor! Fun fact this is still practicing medicine without a license. You are just less likely to have someone come after you for it. If you present yourself in such a way that you could be misconstrued as a medical expert, then if you are practicing medicine, even if you never explicitly claim to be a medical expert you are…

I don’t think it’s reasonable to interpret the output of ChatGPT as medical advice. Maybe once ChatGPT Health launches, but not now.

But, that’s not a hill I want to die on. If your position is that ChatGPT needs to have disclaimer text somewhere in the UI saying “ChatGPT is not a doctor and cannot provide medical advice”, I don’t disagree.

I just don’t think it would make a difference, because as I said, I don’t think anyone reasonably thinks that ChatGPT is a licensed doctor. They just choose to believe ChatGPT anyway, which is their choice in a free society.

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#22

Post by bluefirebrand »

tibbydudeza wrote: post #3
Wowfunhappy wrote: post #2

If someone published a book advising people to take drugs, would people be filing lawsuits? No—we would agree that people are allowed to write whatever they want, even if what they say is terrible, right? I really think these criticisms are misguided. I realize an LLM is not a person—but it does still represent speech, and certainly, any guardrails put in place would themselves be human-authored speech. There are all…

Agreed - people should learn the ChatGPT does not give good advice, but the question is did OpenAI advertise ChatGPT as a good and reliable source of information on health ???.

OpenAI has advertised ChatGPT as a good and reliable source of information on everything

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#23

Post by vablings »

kbelder wrote: post #13

I feel like the son should take the blame. There's never been any shortage of bad advice being passed around. He made the credulous decision to take a mix of party drugs and drink, and I can't believe he had never been told that's a stupid idea. It's sad and I'm not heartless, but sometimes kids make bad decisions. It's not always somebody else's fault.

I agree, there are a few simple hard and fast rules you can follow to be a safe drug user and never mixing drugs is paramount. That is one thing I will always explain to my children that mixing drugs is another layer of gambling on top of you already being dose-unaware and purity-unaware.

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#24

Post by tencentshill »

What if someone relied on random number generator software for dosage information? What if the number generator added affirmative, pleasing text to every number it generated? Will be interesting to see where the line is drawn in these cases. Especially risky for openAI since they now market it to "support, not replace, medical care". https://openai.com/index/introducing-chatgpt-health/

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#25

Post by UncleMeat »

Wowfunhappy wrote: post #2

If someone published a book advising people to take drugs, would people be filing lawsuits? No—we would agree that people are allowed to write whatever they want, even if what they say is terrible, right? I really think these criticisms are misguided. I realize an LLM is not a person—but it does still represent speech, and certainly, any guardrails put in place would themselves be human-authored speech. There are all…

The issue was not "you should take drugs."

The thing that killed this person was being advised to take xanax while having a lot of kratom and alcohol in their system. And yeah, if you published a book telling people that xanax is a great treatment for alcohol induced nausea and people died following this advice you should go to prison.

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#26

Post by heavyset_go »

kbelder wrote: post #13

I feel like the son should take the blame. There's never been any shortage of bad advice being passed around. He made the credulous decision to take a mix of party drugs and drink, and I can't believe he had never been told that's a stupid idea. It's sad and I'm not heartless, but sometimes kids make bad decisions. It's not always somebody else's fault.

If a real person gave them this advice, like a doctor or pharmacist, there would be standing for a lawsuit, might even be criminal.

Looking past "drugs bad mkay", the same ChatGPT that gave this advice is just as capable of giving the same, or worse, advice to someone wondering if they can take an allergy medication like Benedryl with their MAOI antidepressant.

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#27

Post by spoiler »

heavyset_go wrote: post #26
kbelder wrote: post #13

I feel like the son should take the blame. There's never been any shortage of bad advice being passed around. He made the credulous decision to take a mix of party drugs and drink, and I can't believe he had never been told that's a stupid idea. It's sad and I'm not heartless, but sometimes kids make bad decisions. It's not always somebody else's fault.

If a real person gave them this advice, like a doctor or pharmacist, there would be standing for a lawsuit, might even be criminal. Looking past "drugs bad mkay", the same ChatGPT that gave this advice is just as capable of giving the same, or worse, advice to someone wondering if they can take an allergy medication like Benedryl with their MAOI antidepressant.

Yes, but if chokemegently420 on some random sub Reddit gave them that advice, nobody would be the wiser. It's not like ChatGPT is a certified clinician

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#28

Post by heavyset_go »

spoiler wrote: post #27
heavyset_go wrote: post #26

Earlier quoted context omitted.

If a real person gave them this advice, like a doctor or pharmacist, there would be standing for a lawsuit, might even be criminal. Looking past "drugs bad mkay", the same ChatGPT that gave this advice is just as capable of giving the same, or worse, advice to someone wondering if they can take an allergy medication like Benedryl with their MAOI antidepressant.

Yes, but if chokemegently420 on some random sub Reddit gave them that advice, nobody would be the wiser. It's not like ChatGPT is a certified clinician

Why would they believe that when AI is smarter than any human and is going to replace doctors and themselves?

If it isn't going to replace doctors, why is ChatGPT giving medical advice at all, especially deadly medical advice?

Re: Parents say ChatGPT got their son killed with bad advice on party drugs

#30

Post by justinclift »

anon291 wrote: post #20
oompydoompy74 wrote: post #14

Earlier quoted context omitted.

Sure as shit looks like it to me https://openai.com/index/introducing-chatgpt-health/

to the contrary... this is a specific product whith is on a waitlist. Normal ChatGPT is not for health advice.

The front and center most-of-page video on that literally has this in big arse font displayed at page load time:

> Every day, millions of people ask ChatGPT for support with their health

Second paragraph of the main page text then says:

> Health is already one of the most common ways people use ChatGPT, with hundreds of millions of people asking health and wellness questions each week

Clearly ChatGPT is positioned for providing health advice in their main, non-specialised product, not just in their dedicated "Health" product.

Post reply on HN