Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT provides false information about people, and OpenAI can't correct it

noyb.eu

Post reply on HN
Print view View original

51–60 of 92 posts

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#51

Post by krapp »

TheRoque wrote: post #33
trustno2 wrote: post #26

Earlier quoted context omitted.

Nobody put what I think about AI better than this. Yes, there are good cases where "realistically sounding bullshit" is useful! I remember in the early days, before everyone grew kind of tired (well, at least me) of GPT, people used "hehe write me PC manual in style of slam poetry" or whatever, and that is fun. However, people use it to get facts! Just a few days ago, I have read somewhere that hospitals are planning…

People use it to get facts, and trust it more than Google. I have some tech illiterate boss that asked me to do stuff some way because "ChatGPT said so", instead of trusting me, an experience professional. It wasn't like this with a Google search, so why now ? Natural language has a big impact on how the product is perceived

We've seen numerous stories at this point about lawyers trusting AI to generate case documents that turned out to have false cites - AI generated scientific papers are being published. Doctors are using AI. Law enforcement is using AI. Everyone is using it and a lot of people are using it with the assumption that it's intelligent and factual. That it works like the computer from Star Trek. People on this very forum who should know better have said they trust AI more than they trust themselves and more than other people.

AI probably has a niche where it's useful, but because it smells like a magic money machine that will allow managers to replace employees and create value from essentially nothing, modern capitalism dictates we must optimize our entire economy around it, no holds barred, damn the torpedos and full speed ahead because "money." I just hope the fever breaks before people start getting killed.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#53

Post by latexr »

ShamelessC wrote: post #34

Didn’t the media cover this already over and over since ChatGPT’s inception? Not that it isn’t important but to imply it is a new revelation seems sensationalist. > While inaccurate information may be tolerable when a student uses ChatGPT to help him with their homework, it is unacceptable when it comes to information about individuals. I don’t even understand this part. Why would inaccurate information be at all acc…

> Didn’t the media cover this already over and over since ChatGPT’s inception?

This is not “the media”. noyb isn’t reporting on what other people did, they are informing us of what they just did.

> Not that it isn’t important but to imply it is a new revelation seems sensationalist.

The article isn’t about the failings of ChatGPT, it’s about a specific legal complaint being made.

> Why would inaccurate information be at all acceptable for homework?

It’s acceptable in a legal sense, you get a bad grade and that’s that. But falsehoods about a particular individual on a popular resource could be quite damaging and are against EU law.

> This article is bad.

Rather, you completely misunderstood it.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#54

Post by BlackFly »

Squeeze2664 wrote: post #6

This seems so backwards to me. As a user of LLMs, it's clear to me that tokens generated by ChatGPT and similar are not to be interpreted as a legally-valid statement by the company making the LLMs, unless explicitly stated by said company. I certainly don't engage with them that way. I believe this to be fairly obvious to anyone who has used such tools, so I just see this as an opportunity to sue for a quick buck.

Are the statements by the neighbourhood drunk in the corner pub commonly interpreted as legally valid statements by the individual making them? Nobody takes the drunk seriously, yet the neighbourhood drunk shouldn't be treated more seriously with respect to slander than a large corporation with respect to libel.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#55

Post by michaelt »

LASR wrote: post #10

People have a fundamental misunderstanding of what LLMs are. I’ve had to explain this so many times even to engineers. People keep using it as Google. It is not a mechanism to retrieve facts. It’s rather a reasoning mechanism, that when used like a search engine generates text that looks like output from a retrieval of facts. The article talks about OpenAI being unwilling to correct errors. But they just can’t. There…

> The article talks about OpenAI being unwilling to correct errors. But they just can’t.

There are actually several algorithms intended to allow fact editing in LLMs: https://github.com/zjunlp/EasyEdit?tab=readme-ov-file#curren...

They don't work perfectly (e.g. "Tim Cook is CEO of Apple" and "The CEO of Apple is Tim Cook" for some reason have to be edited separately) but they can deal with the most egregious cases. And, you know, maybe OpenAI can improve them further, given they're always going on about 'safety' as a reason their competitors should be regulated out of existence.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#56

Post by krapp »

eclectic29 wrote: post #52

Glad that people are realizing the ChatGPT/AI hype that is driving all companies nuts by creating unnecessary peer pressure to integrate some kind of GenAI in their products even if it doesn’t make sense.

They've always been around, they've just been dismissed as "Luddites" who fear an inevitable future. But of course, the Luddites are always proven correct in hindsight.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#57

Post by r_c_a_d_t_s »

palad1n wrote: post #32

I wonder if my insight on these is wrong, but from what I understand, the GPTs and such are basically all hallucinations, and it's just that most of these are also correlated to reality, what actually exists.

It's whatever is in the training data, plus some hacks. For example, if I ask ChatGPT when my birthday is it says "I don't have personal information on individuals"; but if I ask when King Charles III birthday is it tells me a date... which matches what's in Wikipedia... so it might be right. If there are lots of instances in the training data where the date is wrong, then it will just repeat that wrong date back to you as fact.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#58

Post by Lerc »

palad1n wrote: post #32

I wonder if my insight on these is wrong, but from what I understand, the GPTs and such are basically all hallucinations, and it's just that most of these are also correlated to reality, what actually exists.

That is a matter of perspective. Generally hallucinations are considered to be the output that is not correlated to reality. You are correct that the same method is used for all output. If that were not the case then removing hallucinations would be trivial.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#59

Post by coremoff »

palad1n wrote: post #32

I wonder if my insight on these is wrong, but from what I understand, the GPTs and such are basically all hallucinations, and it's just that most of these are also correlated to reality, what actually exists.

Expressing it that way removes a useful word from our vocab; when people say "halicinations" in this context it specifically means that the generated response does not match reality. If you say that it's just a coincidence when it's correct, and that it's hallicinations all the way down, how do you communicate succinctly to a human that AI responses are not necessarily bound in reality? (I mean in the specific case, not the general case).

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#60

Post by latexr »

Squeeze2664 wrote: post #6

This seems so backwards to me. As a user of LLMs, it's clear to me that tokens generated by ChatGPT and similar are not to be interpreted as a legally-valid statement by the company making the LLMs, unless explicitly stated by said company. I certainly don't engage with them that way. I believe this to be fairly obvious to anyone who has used such tools, so I just see this as an opportunity to sue for a quick buck.

> I believe this to be fairly obvious to anyone who has used such tools

It is not. Not even people in tech understand this, let alone non-technical people. These tools are being marketed as a way to get factual information. Don’t let your knowledge of the technology blind you to the fact that people outside your circle don’t know what you do.

Post reply on HN