21–30 of 262 posts
I was just reading about a suicide tied to AI chatbot 'therapy' uses. This stuff is a nightmare scenario for the vulnerable.
Here is what Illinois says: https://idfpr.illinois.gov/content/dam/soi/en/web/idfpr/news... I get the impression that it is now illegal in Illinois to claim that an AI chatbot can take the place of a licensed therapist or counselor. That doesn't mean people can't do what they want with AI. It only means that counseling services can't offer AI as a cheaper replacement for a real person. Am I wrong? This sounds good to…
In another comment I wondered whether a general chatbot producing text that was later determined in a courtroom to be "therapy" would be a violation. I can read the bill that way, but IANAL.
I was just reading about a suicide tied to AI chatbot 'therapy' uses. This stuff is a nightmare scenario for the vulnerable.
https://hai.stanford.edu/news/exploring-the-dangers-of-ai-in...
Therapists are more valuable that advice from a random friend (for therapy at least) because they can act when triage is necessary (e.g. send in the men in white coats, or refer to something that's not just CBT) and mostly because they're really good at cutting through the bullshit without having the patient walk out.
AIs are notoriously bad at cutting through bullshit. You can always 'jailbreak' an AI, or convince it of bad ideas. It's entirely counterproductive to enable their crazy (sorry, 'maladaptive') behaviour but that's what a lot of AIs will do.
Even if someone makes a good AI, there's always a bad AI in the next tab, and people will just open up a new tab to find an AI gives them the bad advice they want, because if they wanted to listen to good advice they probably wouldn't need to see a therapist. If doctor shopping is as fast and free as opening a new tab, most mental health patients will find a bad doctor rather than listen to a good one.
Good. It's difficult to imagine a worse use case for LLMs.
Just using an LLM as is for therapy, maybe with an extra prompt, is a terrible idea. On the other hand, I could image some more narrow uses where an LLM could help. For example, in Cognitive Behavioral Therapy, there are different methods that are pretty prescriptive, like identifying cognitive distortions in negative thoughts. It's not too hard to imagine an app where you enter a negative thought on your own and exe…
It would still need a therapist to set you on the right track for independent work, and has huge disadvantages compared to the current state-of-the-art, a paper worksheet that you fill out with a pen.
Here is what Illinois says: https://idfpr.illinois.gov/content/dam/soi/en/web/idfpr/news... I get the impression that it is now illegal in Illinois to claim that an AI chatbot can take the place of a licensed therapist or counselor. That doesn't mean people can't do what they want with AI. It only means that counseling services can't offer AI as a cheaper replacement for a real person. Am I wrong? This sounds good to…
Correct. It is more provider-oriented proscription ("You can't say your chatbot is a therapist.") It is not a limitation on usage. You can still, for now, slavishly fall in love with your AI and treat it as your best friend and therapist. There is a specific section that relates to how a licensed professional can use AI: Section 15. Permitted use of artificial intelligence. (a) As used in this Section, "permitted use…
It's not obvious to me as a non-lawyer whether a chat history could be decided to be "therapy" in a courtroom. If so, this could count as a violation. Probably lots of law around this stuff for lawyers and doctors cornered into giving advice at parties already that might apply (e.g., maybe a disclaimer is enough to workaround the prohibition)?
I was just reading about a suicide tied to AI chatbot 'therapy' uses. This stuff is a nightmare scenario for the vulnerable.
Good. It's difficult to imagine a worse use case for LLMs.
Earlier quoted context omitted.
Correct. It is more provider-oriented proscription ("You can't say your chatbot is a therapist.") It is not a limitation on usage. You can still, for now, slavishly fall in love with your AI and treat it as your best friend and therapist. There is a specific section that relates to how a licensed professional can use AI: Section 15. Permitted use of artificial intelligence. (a) As used in this Section, "permitted use…
Yes, but also "An... entity may not provide... therapy... to the public unless the therapy... services are conducted by... a licensed professional". It's not obvious to me as a non-lawyer whether a chat history could be decided to be "therapy" in a courtroom. If so, this could count as a violation. Probably lots of law around this stuff for lawyers and doctors cornered into giving advice at parties already that might…