Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT terms disallow its use in providing legal and medical advice to others

ctvnews.ca

Post reply on HN
Print view View original

331–340 of 439 posts

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#331

Post by jjulius »

adangert wrote: post #278

Last year ChatGPT helped save my life from having a stroke. LLMs are incredibly beneficial in providing medical information and advice today.

>LLMs are incredibly beneficial...

No, they can be. To state that they are, as an absolute, based on your sample size of one, especially with regard to other instances where ChatGPT has failed the user with serious physical results, is fallacious.

I am glad that you are OK, but as another user suggested, it's nowhere near as consistently accurate as it needs to be in order to be anywhere near an adequate substitute for a call to a GP or 911.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#332

Post by saberience »

zekica wrote: post #314
KeplerBoy wrote: post #289

Earlier quoted context omitted.

Non-deterministic doesn't mean random or unpredictable. That's like saying the weather forecast is useless because it's not deterministic or always 100% accurate.

Non-deterministic means random - that's the definition of the word. The weather forecast is also random - in fact, weather forecast is (if you simplify it too much) an average of several predictive (generative) models.

ChatGPT isn’t random though.

If you ask it what a star is, it’s never going to tell you it’s a giant piece of cheese floating in the the sky.

If you don’t believe me, try it, write a for loop which asks ChatGPT, what is a star (astronomy) exactly? Ask it 1000 times and then tell me how random it is versus how consistent it is.

The idea that non deterministic === random is totally deluded. It just means you cannot predict the exact tokens which will be produced but it doesn’t mean it’s random like a random number generator and it could be any thing.

If you ask what is Michael Jackson the entertainer famous for it’s going to tell you he’s famous for music and dancing. 1000/1000 times, is that random?

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#333

Post by sarchertech »

jjtheblunt wrote: post #251
sarchertech wrote: post #244

Earlier quoted context omitted.

You can’t possibly have enough data to support that statement.

there was no mention of sample size, though, so the statement might be true for the commenter but not widely applicable, to your point

I’m saying that the commenter doesn’t have a large enough sample size to draw conclusions even for themselves.

Just based on the amount of time ChatGPT has been out and the maximum number of different diagnosable and confirmable ailments they could have had in that time.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#334

Post by sarchertech »

matwood wrote: post #269
ryandrake wrote: post #248

Earlier quoted context omitted.

If you're not a doctor, how do you know it's accurate? This is the huge problem with using LLMs for this kind of thing. How do you verify that it is better? What is the ground truth you are testing it against? If you wanted to verify that ChatGPT could do math, you'd ask it 100 math problems and then (importantly) verify its answers with a calculator. How do you verify that ChatGPT can interpret medical information w…

How does anyone know if what the doctor says is accurate? Obviously people should put the most relative weight in their doctor's opinion, but there's a reason people always say to get a second opinion. Unfortunately because of how the US healthcare system works today people have to become their own doctors and advocates. LLMs are great at surfacing the unknown unknowns, and I think can help people better prepare for…

> How does anyone know if what the doctor says is accurate?

They don’t. That’s nearly the entire justification for licensure.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#335

Post by KeplerBoy »

zekica wrote: post #314
KeplerBoy wrote: post #289

Earlier quoted context omitted.

Non-deterministic doesn't mean random or unpredictable. That's like saying the weather forecast is useless because it's not deterministic or always 100% accurate.

Non-deterministic means random - that's the definition of the word. The weather forecast is also random - in fact, weather forecast is (if you simplify it too much) an average of several predictive (generative) models.

> Non-deterministic means random - that's the definition of the word.

That's not really the defintion. Non-determinism just means the outcome is not a pure function of the inputs. A PRNG doesn't become truly random just because we don't know the state and seed when calling the function and the same holds for LLMs. The non-determinism in LLMs comes from accepted race conditions in the GPU floating point math and the PRNG in the sampler.

That's besides the point, but we could have perfectly deterministic LLMs.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#336

Post by sarchertech »

xp84 wrote: post #256
ryandrake wrote: post #248

Earlier quoted context omitted.

If you're not a doctor, how do you know it's accurate? This is the huge problem with using LLMs for this kind of thing. How do you verify that it is better? What is the ground truth you are testing it against? If you wanted to verify that ChatGPT could do math, you'd ask it 100 math problems and then (importantly) verify its answers with a calculator. How do you verify that ChatGPT can interpret medical information w…

I know it’s hard to accept, but it’s got to be weighed against the real-world alternative: You get about 5-8 minutes of face time with an actual doctor, but have to wait up to weeks to actually get in to see a doctor, except maybe at an urgent care or ER. Or people used to just play around on WebMD which was even worse since it wasn’t in any way tailored to what the patient’s stated situation is. There’s the rest of…

> but I think coming into your 5-minute appointment having already had all your dumbest and least-informed ideas and theories shot down by ChatGPT is a big improvement and helps you maximize your time.

That’s balanced out by the time wasted by all the people who’ve had their worst fears and paranoias confirmed by ChatGPT.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#337

Post by sarchertech »

matwood wrote: post #268
lp0_on_fire wrote: post #255

Earlier quoted context omitted.

Nobody would use these services for anything important if they _actually understood_ these glorified markov chains are just as likely to confidently assert something false and lie about it when pressed as they are to produce accurate information. These AI companies have sold a bill of goods but the right people are making money off it so they’ll never be held responsible in a scenarios the one you described.

What is the most likely cause of this set of facts is how diagnostics works. LLMs are tailor made for this type of use case.

That’s not what they’re trained on though and not what they’re doing.

When you ask a general LLM like ChatGPT “I have a headache what is the cause” it’s going to give you the most statistically likely response in it’s training data. That might coincide with the most likely cause, but it’s also very possible it doesn’t. Especially for problems with widespread misconceptions.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#338

Post by lazide »

pksebben wrote: post #217
lazide wrote: post #214

Earlier quoted context omitted.

LLMs are trained on material doing all these things though.

true, true. Turtles all the way down and such.

Even more so with the ouruborus issue caused by LLMs trying to train on an increasingly LLM generated internet.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#339

Post by zombot »

That's almost too reasonable to be credible. They are probably just covering their ass until they can do it with impunity. With the rapidly progressing corruption of the legal system it's merely a question of time until companies doing it for profit can disseminate medical and legal bullshit all they want. Politicians are already doing it.

Re: ChatGPT terms disallow its use in providing legal and medical advice to others

#340

Post by altcognito »

xp84 wrote: post #258
shagie wrote: post #239

I read this not as "you can't ask ChatGPT about {medical or legal topic}" but rather "you can't build something on ChatGPT that provides {medical or legal} advice or support to someone else." For example, Epic couldn't embed ChatGPT into their application to have it read your forms for you. You can still ask it - but Epic can't build it. That said, I haven't found the specific terms and conditions that are mentioned…

And this seems actually wildly reasonable. It’s actually pretty scammy to take people’s money (or whatever your business model is) for legal or medical advice just to give them whatever AI shits out. If I wanted ChatGPT’s opinion (as I actually often do) I’d just ask ChatGPT for free or nearly-free. Anyone repackaging ChatGPT’s output as fit for a specialized purpose is scamming their customers.

> Anyone repackaging ChatGPT’s output as fit for a specialized purpose is scamming their customers.

This is genuinely hilarious given what LLMs are.

Post reply on HN