Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT provides false information about people, and OpenAI can't correct it

noyb.eu

Post reply on HN
Print view View original

71–80 of 92 posts

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#71

Post by palad1n »

coremoff wrote: post #59
palad1n wrote: post #32

I wonder if my insight on these is wrong, but from what I understand, the GPTs and such are basically all hallucinations, and it's just that most of these are also correlated to reality, what actually exists.

Expressing it that way removes a useful word from our vocab; when people say "halicinations" in this context it specifically means that the generated response does not match reality. If you say that it's just a coincidence when it's correct, and that it's hallicinations all the way down, how do you communicate succinctly to a human that AI responses are not necessarily bound in reality? (I mean in the specific case,…

Well, I might be taking a pessimistic view on it, but if you expressed it this way to the average user, they might pay more attention to what's actually there when an answer returns to them. I think they have in mind that the text is (generally) grounded in reality like how a human mind grounds things, and this is definitely a hurdle in understanding how these LLM machines work.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#72

Post by Kwpolska »

r_c_a_d_t_s wrote: post #57
palad1n wrote: post #32

I wonder if my insight on these is wrong, but from what I understand, the GPTs and such are basically all hallucinations, and it's just that most of these are also correlated to reality, what actually exists.

It's whatever is in the training data, plus some hacks. For example, if I ask ChatGPT when my birthday is it says "I don't have personal information on individuals"; but if I ask when King Charles III birthday is it tells me a date... which matches what's in Wikipedia... so it might be right. If there are lots of instances in the training data where the date is wrong, then it will just repeat that wrong date back to…

It might be able to produce true data about very famous individuals, and it might refuse to provide information about unknown individuals. But if you ask about a YouTuber, or a smaller star/celebrity, it is more likely to produce a false statement. I asked about the birthday of Tom Scott (the YouTuber) three times and got three different dates (and none supported by a Google search).

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#73

Post by Vuizur »

bux93 wrote: post #37
subroutine wrote: post #24

You can make LLMs say pretty much whatever you want with the right prompts. This is a complex issue, and if EU citizens want access to LLMs the GDPR is going to need a different set of rules for LLMs than for websites and search engines.

Meh. GDPR sets limits on 'processing' personally identifiable information. In the context of an LLM, its outputs may contain PII if its inputs do. Those inputs are training input and prompts. So long(!) as the training input doesn't have PII, the output will only have it if the prompts do. Same as if you save a file on onedrive, if you save PII there, you're the data controller, and Microsoft is a processor on your b…

It is impossible to remove personal data ("any information which are related to an identified or identifiable natural person") from the LLM training data.

As far as I understand it ChatGPT and all other similar systems are blatantly violating GDPR, they would have to for example publish their related training data to conform.

I guess the EU authorities don't do anything for now because they don't want to admit that their funny law basically bans all state-of-the-art AI.

(Ok, Openai also broke the law in almost all countries by downloading shadow libraries, but here they at least have more plausible deniability.)

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#74

Post by DonHopkins »

oldpersonintx wrote: post #3
esher wrote: post #2

Popcorn time: NOYB (Max Schrems) filed a complaint against OpenAI with the Austrian DPA: ChatGPT is not GDPR compliant.

Sounds like a win for the US...get competing economies to block the technology for trivial reasons, then by time the bugs are worked out they will be so far behind that their only choice will be to choose US-based solutions

You're suffering from ChatGPT Derangement Syndrome.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#75

Post by blitzar »

krapp wrote: post #51
TheRoque wrote: post #33

Earlier quoted context omitted.

People use it to get facts, and trust it more than Google. I have some tech illiterate boss that asked me to do stuff some way because "ChatGPT said so", instead of trusting me, an experience professional. It wasn't like this with a Google search, so why now ? Natural language has a big impact on how the product is perceived

We've seen numerous stories at this point about lawyers trusting AI to generate case documents that turned out to have false cites - AI generated scientific papers are being published. Doctors are using AI. Law enforcement is using AI. Everyone is using it and a lot of people are using it with the assumption that it's intelligent and factual. That it works like the computer from Star Trek. People on this very forum w…

It turns out in a lot of (low skilled) knowledge work the nonsense that AI spits out is superior to humans.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#76

Post by r_c_a_d_t_s »

Kwpolska wrote: post #72
r_c_a_d_t_s wrote: post #57

Earlier quoted context omitted.

It's whatever is in the training data, plus some hacks. For example, if I ask ChatGPT when my birthday is it says "I don't have personal information on individuals"; but if I ask when King Charles III birthday is it tells me a date... which matches what's in Wikipedia... so it might be right. If there are lots of instances in the training data where the date is wrong, then it will just repeat that wrong date back to…

It might be able to produce true data about very famous individuals, and it might refuse to provide information about unknown individuals. But if you ask about a YouTuber, or a smaller star/celebrity, it is more likely to produce a false statement. I asked about the birthday of Tom Scott (the YouTuber) three times and got three different dates (and none supported by a Google search).

Yes, I agree. That's what I was trying to say. It also "doubles down" when called out: I asked about the mathematician Richard Taylor and it responded with a paragraph that called him Sir Richard Lawrence Taylor, so I asked when he was knighted (he wasn't) and it said "The 2014 birthday honours list" which is wrong... probably because there was a Dr Richard Thomas Taylor awarded an MBE (not a knighthood) in that list.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#77

Post by rsynnott »

palad1n wrote: post #32

I wonder if my insight on these is wrong, but from what I understand, the GPTs and such are basically all hallucinations, and it's just that most of these are also correlated to reality, what actually exists.

The 'hallucination' term is an unfortunate bit of anthropomorphisation. It's a machine for generating plausible text; sometimes the text is true, accidentally. This bears no particular resemblance to any conventional meaning of 'hallucination'.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#78

Post by rsynnott »

oldpersonintx wrote: post #3
esher wrote: post #2

Popcorn time: NOYB (Max Schrems) filed a complaint against OpenAI with the Austrian DPA: ChatGPT is not GDPR compliant.

Sounds like a win for the US...get competing economies to block the technology for trivial reasons, then by time the bugs are worked out they will be so far behind that their only choice will be to choose US-based solutions

Yes; the US will lead the world in plausible-looking bullshit generation.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#79

Post by subroutine »

Mordisquitos wrote: post #38
subroutine wrote: post #24

You can make LLMs say pretty much whatever you want with the right prompts. This is a complex issue, and if EU citizens want access to LLMs the GDPR is going to need a different set of rules for LLMs than for websites and search engines.

If LLM providers want access to the EU market they will need to find a way to comply with GDPR, and if OpenAI cannot find a way to do it then a different LLM provider will.

> the GDPR requires information about individuals is accurate

Given that you can make LLMs say pretty much whatever you want using the right prompts, this seems impossible. LLMs are not a search engine, and based on conversational context might say Emmanuel Macron is the president of France or a baby giraffe.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#80

Post by Kwpolska »

r_c_a_d_t_s wrote: post #76
Kwpolska wrote: post #72

Earlier quoted context omitted.

It might be able to produce true data about very famous individuals, and it might refuse to provide information about unknown individuals. But if you ask about a YouTuber, or a smaller star/celebrity, it is more likely to produce a false statement. I asked about the birthday of Tom Scott (the YouTuber) three times and got three different dates (and none supported by a Google search).

Yes, I agree. That's what I was trying to say. It also "doubles down" when called out: I asked about the mathematician Richard Taylor and it responded with a paragraph that called him Sir Richard Lawrence Taylor, so I asked when he was knighted (he wasn't) and it said "The 2014 birthday honours list" which is wrong... probably because there was a Dr Richard Thomas Taylor awarded an MBE (not a knighthood) in that list…

ChatGPT will confidently fabricate data out of thin air, it is less likely to admit it does not know something or is not sure. And that’s the primary problem with it: the statement is believable, but incorrect.
Post reply on HN