Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT provides false information about people, and OpenAI can't correct it

noyb.eu

Post reply on HN
Print view View original

31–40 of 92 posts

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#31

Post by Mordisquitos »

LASR wrote: post #10

People have a fundamental misunderstanding of what LLMs are. I’ve had to explain this so many times even to engineers. People keep using it as Google. It is not a mechanism to retrieve facts. It’s rather a reasoning mechanism, that when used like a search engine generates text that looks like output from a retrieval of facts. The article talks about OpenAI being unwilling to correct errors. But they just can’t. There…

The organic neural network inside my skull isn't a mechanism to retrieve facts either, nor are specific outputs that it may provide discernible from the weights. And yet, if it responds with a factual error to a given prompt, it can be trivially corrected so as to provide the right answer for future prompts.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#33

Post by TheRoque »

trustno2 wrote: post #26
chx wrote: post #7

Every single time... https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, f…

Nobody put what I think about AI better than this. Yes, there are good cases where "realistically sounding bullshit" is useful! I remember in the early days, before everyone grew kind of tired (well, at least me) of GPT, people used "hehe write me PC manual in style of slam poetry" or whatever, and that is fun. However, people use it to get facts! Just a few days ago, I have read somewhere that hospitals are planning…

People use it to get facts, and trust it more than Google. I have some tech illiterate boss that asked me to do stuff some way because "ChatGPT said so", instead of trusting me, an experience professional. It wasn't like this with a Google search, so why now ? Natural language has a big impact on how the product is perceived

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#34

Post by ShamelessC »

Didn’t the media cover this already over and over since ChatGPT’s inception? Not that it isn’t important but to imply it is a new revelation seems sensationalist.

> While inaccurate information may be tolerable when a student uses ChatGPT to help him with their homework, it is unacceptable when it comes to information about individuals.

I don’t even understand this part. Why would inaccurate information be at all acceptable for homework? You need accurate information there just as much as with people. In fact there’s probably a good deal of overlap.

This article is bad.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#35

Post by Mordisquitos »

mrtksn wrote: post #4

IMHO this is not a fixable problem, it's not even a new problem because false information existed since ever. With the Web it become globally available, with social media it become globally spreadable and with "AI" it has become of endless supply in form and variety. Humans need to develop humane defense mechanisms for the new reality, tech cannot be stopped and cannot be hermetically insulated against mistakes or ba…

Why is it "not a fixable problem"? So many problems introduced by new technologies have been claimed to be "non fixable" by the businesses promoting them until, lo and behold, the law has forced them to do so.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#36

Post by tlrobinson »

yunwal wrote: post #22
lnxg33k1 wrote: post #17

Earlier quoted context omitted.

I don't know, it seems recently that we've detached what things are from what the people behind them say things are, so bing AI is an LLM, phind is an LLM, and they market themselves as search engine, like to see in details you say >> It is not a mechanism to retrieve facts Then bing AI powered by chatgpt shows on its site >> [Hello, this is Bing! I’m the new AI-powered chat mode of Microsoft Bing that] can help you…

Bing is not just an LLM. It’s RAG, the LLM is just a layer on top of a typical search engine. The LLM is not “getting” any facts, it’s just synthesizing them in a human-readable format.

LLMs are also often used in the search component of RAG, by generating embeddings that are then indexed and searched.

(I don’t know if that’s how Bing AI works)

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#37

Post by bux93 »

subroutine wrote: post #24

You can make LLMs say pretty much whatever you want with the right prompts. This is a complex issue, and if EU citizens want access to LLMs the GDPR is going to need a different set of rules for LLMs than for websites and search engines.

Meh. GDPR sets limits on 'processing' personally identifiable information. In the context of an LLM, its outputs may contain PII if its inputs do. Those inputs are training input and prompts. So long(!) as the training input doesn't have PII, the output will only have it if the prompts do. Same as if you save a file on onedrive, if you save PII there, you're the data controller, and Microsoft is a processor on your behalf.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#38

Post by Mordisquitos »

subroutine wrote: post #24

You can make LLMs say pretty much whatever you want with the right prompts. This is a complex issue, and if EU citizens want access to LLMs the GDPR is going to need a different set of rules for LLMs than for websites and search engines.

If LLM providers want access to the EU market they will need to find a way to comply with GDPR, and if OpenAI cannot find a way to do it then a different LLM provider will.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#39

Post by sharpshadow »

We here in Germany, as so other countries, have a issue with Wikipedia and their actions against a certain clientele.

If you get branded a conspiracy theorist your Wikipedia entry will denounce you as such and the admins and moderators will lock the article and you can do nothing about it.

Obviously ChatGPT will take information about your persona from Wikipedia if available and will update this information accordingly to the changes in Wikipedia.

And you can’t do anything about it.

Re: ChatGPT provides false information about people, and OpenAI can't correct it

#40

Post by intelVISA »

Mordisquitos wrote: post #31
LASR wrote: post #10

People have a fundamental misunderstanding of what LLMs are. I’ve had to explain this so many times even to engineers. People keep using it as Google. It is not a mechanism to retrieve facts. It’s rather a reasoning mechanism, that when used like a search engine generates text that looks like output from a retrieval of facts. The article talks about OpenAI being unwilling to correct errors. But they just can’t. There…

The organic neural network inside my skull isn't a mechanism to retrieve facts either, nor are specific outputs that it may provide discernible from the weights. And yet, if it responds with a factual error to a given prompt, it can be trivially corrected so as to provide the right answer for future prompts.

Bold of you to assume 'Open' AI is accountable to anything other than printing more money f-for h-humanity of course.
Post reply on HN