Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT terms disallow its use in providing legal and medical advice to others

ctvnews.ca

Post reply on HN
Print view View original

361–370 of 439 posts

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#361

Post by liampulles »

mk89 wrote: post #304
liampulles wrote: post #296

Earlier quoted context omitted.

I think this would apply if they sought out government regulation that applies to all AI players, not just their own company.

...or they want to be the first to do it, as others don't have it. OpenAI is the leading company, if they provide an instance you can use for legal advice, with relative certification etc., it'd be better to trust them rather than another random player. They create the software, the certification and the need for it.

Suppose I am the CTO at a big legal firm, and I have been charged with finding an AI service to integrate with. What could an OpenAI sales person tell me that justifies the value of this certification?

Lets suppose they've already told me that their company is a "leader in the AI space", to which I responded that the guy at Claude told me the same thing on Monday, with which I was equally unimpressed.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#362

Post by liampulles »

dotancohen wrote: post #357
mk89 wrote: post #295

I was wondering when they would start with the regulations. Next will be: you're allowed to use it, if your company/product is "HippocraticGPT"-certified, a bit like SOC or PCI compliance. This way they can craft an instance of GPT for your specific purposes (law, medicine, etc) and you know it's "safe" to use. This way they sell EE licenses, which is where the big $$$ are.

> Next will be: you're allowed to use it, if your company/product is "HippocraticGPT"-certified, a bit like SOC or PCI compliance. That's exactly how it should be. That's why one needs a license to practice medicine. I work with a company in this space. All the AI tools are for the professional to use - the patient never interacts with them. The inferred output (I prefer the word prediction) is only ever seen by a me…

What is the value of the certification on the AI output? If OpenAI says they'll get guarantee the output and be accountable for errors, then thats of value, and would justify the certification. Is that the bar?

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#363

Post by jeromegv »

evv wrote: post #360
shagie wrote: post #239

I read this not as "you can't ask ChatGPT about {medical or legal topic}" but rather "you can't build something on ChatGPT that provides {medical or legal} advice or support to someone else." For example, Epic couldn't embed ChatGPT into their application to have it read your forms for you. You can still ask it - but Epic can't build it. That said, I haven't found the specific terms and conditions that are mentioned…

I'm confused how you would "build something on ChatGPT" in the first place. Does ChatGPT have an API? Of course OpenAI's GPT-5 and family are available as an API, but this is the first time I'm hearing about the ability to build on top of ChatGPT (the consumer product). I'm guessing this is a mistake by the journalist, who didn't realize that you can use GPT-5 without using ChatGPT? It seems that they have a unified…

Don't assume a general news journalist on a deadline to drop a quick non-important article would understand fully between the API access of OpenAI, and ChatGPT itself.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#364

Post by terminalshort »

sarchertech wrote: post #244
bushbaba wrote: post #234

Earlier quoted context omitted.

I’ve found it just as accurate and a better experience than using telehealth or generalist doctors

You can’t possibly have enough data to support that statement.

Do you have enough data to support the opposite conclusion? If not, your call for data is just a pathetic attempt to sound rational while making an empty appeal to authority.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#365

Post by stonemetal12 »

miki123211 wrote: post #326
xp84 wrote: post #258

Earlier quoted context omitted.

And this seems actually wildly reasonable. It’s actually pretty scammy to take people’s money (or whatever your business model is) for legal or medical advice just to give them whatever AI shits out. If I wanted ChatGPT’s opinion (as I actually often do) I’d just ask ChatGPT for free or nearly-free. Anyone repackaging ChatGPT’s output as fit for a specialized purpose is scamming their customers.

What if Chat GPT is used as a "power tool" for actual legal / medical professionals? E.G. Here's the set of documents uploaded by the customer. Here are my most important highlights (notice that document #68 is an official letter from last year informing the person that form 68d doesn't need to be filled in their specific situation). Here are my takeaways. Here's my recommendation. Apporve or change?

Ok, but it didn't mention doc 74 that is actually the most important to the matter at hand so now your patient\client is dead or in jail.

It is only a power tool if it works reliably and predictably. If you have to do all the work anyway then the tool is adding to your burden not saving time and effort.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#366

Post by bluGill »

johnny_canuck wrote: post #312
PeterStuer wrote: post #273

I have seen more than once people using ChatGPT as a psychologist and psychiatrist and self diagnosing mental illnesses. It would be hilarious if it was not so disasterous. People with real questions get roped in by an enthusiastically confirming sycophantic model that will never question or push back, just confirm and lead on. Tbh, and I usually do not like this way of thought, but these are lawsuits waiting to happ…

It is awful, but at the same time this isn't new. People have for a long time used Google searches to self diagnose their issues. ChatGPT just makes that process ever easier now. From my viewpoint it speaks more to a problem of the healthcare system.

Even before google people got books to self-diagnose problems.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#367

Post by simsla »

shagie wrote: post #239

I read this not as "you can't ask ChatGPT about {medical or legal topic}" but rather "you can't build something on ChatGPT that provides {medical or legal} advice or support to someone else." For example, Epic couldn't embed ChatGPT into their application to have it read your forms for you. You can still ask it - but Epic can't build it. That said, I haven't found the specific terms and conditions that are mentioned…

I think it's simpler than that, and they're just trying to avoid liability.

With all of their claims about how GPT can pass the legal/medical bar etc. I wouldn't be surprised if they're eventually held accountable for some of the advice their model gives out.

This way, they're covered. Similarly, Epic could still build that feature, but they'd have to add a disclaimer like "AI generated, this does not constitute legal advice, consult a professional if needed".

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#368

Post by siva7 »

evv wrote: post #360
shagie wrote: post #239

I read this not as "you can't ask ChatGPT about {medical or legal topic}" but rather "you can't build something on ChatGPT that provides {medical or legal} advice or support to someone else." For example, Epic couldn't embed ChatGPT into their application to have it read your forms for you. You can still ask it - but Epic can't build it. That said, I haven't found the specific terms and conditions that are mentioned…

I'm confused how you would "build something on ChatGPT" in the first place. Does ChatGPT have an API? Of course OpenAI's GPT-5 and family are available as an API, but this is the first time I'm hearing about the ability to build on top of ChatGPT (the consumer product). I'm guessing this is a mistake by the journalist, who didn't realize that you can use GPT-5 without using ChatGPT? It seems that they have a unified…

wait... so you can't use openai models for educational content? so they can kick out any competi.. ehm company they don't like as this list is quite... extensive?

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#369

Post by shagie »

siva7 wrote: post #368
evv wrote: post #360

Earlier quoted context omitted.

I'm confused how you would "build something on ChatGPT" in the first place. Does ChatGPT have an API? Of course OpenAI's GPT-5 and family are available as an API, but this is the first time I'm hearing about the ability to build on top of ChatGPT (the consumer product). I'm guessing this is a mistake by the journalist, who didn't realize that you can use GPT-5 without using ChatGPT? It seems that they have a unified…

wait... so you can't use openai models for educational content? so they can kick out any competi.. ehm company they don't like as this list is quite... extensive?

I believe the key word in that passage is "decisions".

You can't use ChatGPT (or other OpenAI offerings) to grade essays, decide who is the least risky tenant, assign risk for insurance, filter resumes, approve a loan, determine sentencing...

Those are things that require human agency for "a person decided this" rather than "we fed it into the program and took the answer."

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#370

Post by elcritch »

shagie wrote: post #369
siva7 wrote: post #368

Earlier quoted context omitted.

wait... so you can't use openai models for educational content? so they can kick out any competi.. ehm company they don't like as this list is quite... extensive?

I believe the key word in that passage is "decisions". You can't use ChatGPT (or other OpenAI offerings) to grade essays, decide who is the least risky tenant, assign risk for insurance, filter resumes, approve a loan, determine sentencing... Those are things that require human agency for "a person decided this" rather than "we fed it into the program and took the answer."

Which of course means humans will use it for those decisions until there’s some lawsuits and maybe some laws.
Post reply on HN