371–380 of 439 posts
this is a disaster doomer's in control, again
Literally nothing to do with "doomers" X-risk concerns. See if you can find "medical advice" ever mentioned as a problem: https://www.lesswrong.com/posts/kgb58RL88YChkkBNf/the-proble...
Earlier quoted context omitted.
I keep seeing this problem more and more with humans. What should we call it? Maybe Hallucinations? Where there is an accurate true thing and then it just gets altered by these guys who call themselves journalists and reporters and the like until it is just ... completely unrecognizable? I'm pretty sure it's a fundamental issue with the architecture.
Also these guys who call themselves doctors. I have narcolepsy and the first 10 or so doctors I went to hallucinated the wrong diagnosis.
Hmm. So OpenAI doesn't care about other people's terms, copyrights or any sort of IP; but they get to have their own terms.
Earlier quoted context omitted.
...or they want to be the first to do it, as others don't have it. OpenAI is the leading company, if they provide an instance you can use for legal advice, with relative certification etc., it'd be better to trust them rather than another random player. They create the software, the certification and the need for it.
Suppose I am the CTO at a big legal firm, and I have been charged with finding an AI service to integrate with. What could an OpenAI sales person tell me that justifies the value of this certification? Lets suppose they've already told me that their company is a "leader in the AI space", to which I responded that the guy at Claude told me the same thing on Monday, with which I was equally unimpressed.
Or: "your tokens will cost 1/3, because we have crafted a particular gpt instance that is superoptimised for your use case only, look we just created one for you before the meeting".
Money is the biggest motivator. A company that is throwing $$$ left and right needs EE yesterday.
A lot like AWS credits. "Nice I can use up to 200'000$ for free".........
I read this not as "you can't ask ChatGPT about {medical or legal topic}" but rather "you can't build something on ChatGPT that provides {medical or legal} advice or support to someone else." For example, Epic couldn't embed ChatGPT into their application to have it read your forms for you. You can still ask it - but Epic can't build it. That said, I haven't found the specific terms and conditions that are mentioned…
I'm confused how you would "build something on ChatGPT" in the first place. Does ChatGPT have an API? Of course OpenAI's GPT-5 and family are available as an API, but this is the first time I'm hearing about the ability to build on top of ChatGPT (the consumer product). I'm guessing this is a mistake by the journalist, who didn't realize that you can use GPT-5 without using ChatGPT? It seems that they have a unified…
Earlier quoted context omitted.
> Next will be: you're allowed to use it, if your company/product is "HippocraticGPT"-certified, a bit like SOC or PCI compliance. That's exactly how it should be. That's why one needs a license to practice medicine. I work with a company in this space. All the AI tools are for the professional to use - the patient never interacts with them. The inferred output (I prefer the word prediction) is only ever seen by a me…
What is the value of the certification on the AI output? If OpenAI says they'll get guarantee the output and be accountable for errors, then thats of value, and would justify the certification. Is that the bar?
I don't believe they will go this far, but somewhere in the middle. And I think they might still be trying to figure it out.
Last year ChatGPT helped save my life from having a stroke. LLMs are incredibly beneficial in providing medical information and advice today.
> LLMs are incredibly beneficial ... today. LLMs sometimes can be incredibly beneficial ... today LLMs sometimes can be incredibly harmful ... today Non-deterministic things aren't just one thing, they're whatever they happen to be in that particular moment.
Most things that are generally helpful and beneficial are not 100% helpful and beneficial 100% of the time.
I used GPT-4 as a second opinion on my medical tests and doctor's advice, and it suggested an alternate diagnosis and treatment plan that turned out to be correct. That was incredibly helpful and beneficial.
You're replying to a person who had a similar and even more helpful and beneficial experience because they're alive today.
Pedantically pointing out that a beneficial and helpful thing isn't 100% beneficial and helpful 100% of the time doesn't add anything useful to the conversation since everyone here already knows it's not 100%.
I have seen more than once people using ChatGPT as a psychologist and psychiatrist and self diagnosing mental illnesses. It would be hilarious if it was not so disasterous. People with real questions get roped in by an enthusiastically confirming sycophantic model that will never question or push back, just confirm and lead on. Tbh, and I usually do not like this way of thought, but these are lawsuits waiting to happ…
> I have seen more than once people using ChatGPT as a psychologist and psychiatrist and self diagnosing mental illnesses. I'm working with a company that writes AI tools for mental health professionals. The output of the models _never_ goes to the patient. I've heard quite a few mouthfuls of how harmful LLM advice is for mental health patients - with specific examples. I'm talking on the order of suicide attempts (a…
Basically there needs to be an LLM that can push back.
I have seen more than once people using ChatGPT as a psychologist and psychiatrist and self diagnosing mental illnesses. It would be hilarious if it was not so disasterous. People with real questions get roped in by an enthusiastically confirming sycophantic model that will never question or push back, just confirm and lead on. Tbh, and I usually do not like this way of thought, but these are lawsuits waiting to happ…
It is awful, but at the same time this isn't new. People have for a long time used Google searches to self diagnose their issues. ChatGPT just makes that process ever easier now. From my viewpoint it speaks more to a problem of the healthcare system.
Earlier quoted context omitted.
It is awful, but at the same time this isn't new. People have for a long time used Google searches to self diagnose their issues. ChatGPT just makes that process ever easier now. From my viewpoint it speaks more to a problem of the healthcare system.
I agree with everything you said but chatGPT does have an insidious thing where it confirms your biases. It kind of senses what you want and actually runs with it. For truly unbiased responses you literally have to hide your intention from chatGPT. But even so chatGPT can many times still sense your intent and give you a biased answer.