[dead]
We need to tell people ChatGPT will lie to them, not debate linguistics
341–350 of 485 posts
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#342Earlier quoted context omitted.
This whole conversation is speculative, obviously. The AI doomers are all speculating without evidence as well. The tendency GPT and other LLMs to hallucinate is clearly documented. Is this not evidence? I think it's fair to predict that if we can't solve or mitigate this problem, it's going to put a significant cap on this kind of AI's usefulness and become a blocker to reaching AGI.
Gpt-4 hallucinates meaningfully less than gpt-3. There is more evidence in favor of “more improvements to come” than “ai winter approaches”. A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data. Based on what I’ve read and roughly speculating, it looks there is easily enough existing data for gpt-5 and probably a few more versions after. I’m not sure…
> There is more evidence in favor of “more improvements to come” than “ai winter approaches”.
Not to mention that we haven't really seen what GPT-4 can do to the world yet. Most people are still treating it as a cool toy, when it can probably replace 20% of standard office workers across all professions with the capabilities it has today.
I mean, ChatGPT is out-diagnosing medical professionals and out-advising experienced lawyers in some cases. How do you think it would fare against the average (essentially untrained) customer support representative?
The major effects are yet to come, and there doesn't even have to be a GPT-5 for that to happen – the world just needs to fully realize what GPT-4 already is.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#343A few days ago I asked ChatGPT to help me find an argument to perform "some filtering operation" with a world class command line tool. It confidently aswered.
./tool arg1 param1 arg2 param2 --filtering-operation param3
I went for it and it didn't work... I did google a bit and `--filtering-operation` didn't exist.
I then told chat GPT that `--filtering-operation` wasn't real. It apologized and offered me an option that made sense if I made some extra work.
Googling and scanning the crappy documentation would have probably gotten me a faster result.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#344Earlier quoted context omitted.
Gpt-4 hallucinates meaningfully less than gpt-3. There is more evidence in favor of “more improvements to come” than “ai winter approaches”. A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data. Based on what I’ve read and roughly speculating, it looks there is easily enough existing data for gpt-5 and probably a few more versions after. I’m not sure…
I think good annotation is much harder than obtaining data. But now Reddit, Twitter and Quora will realise what kind goldmine their data is they might close easy access to it.
It’s too easy to scrape them, but even that is already done by many intermediary companies all too happy to resell that data.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#345The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…
ChatGPT scales better than "demagogues", so it does have more than a "little practical difference".
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#346> This is a serious bug. Isn't describing this as a 'bug' rather than a misuse of a powerful text generation tool, playing into the framing that it's a truth telling robot brain? I saw a quote that said "it's a what text would likely come next machine", if it makes up a url pointing to a fake article with a plausible title by a person who works in that area, that's not a bug. That's it doing what it does, generating…
I think it's an entire category of bugs. The thing where you paste in a URL and it says "here is a summary of the content of that page: ..." is very definitely a bug. It's a user experience bug - the system should not confuse people by indicating it can do something that it cannot. The thing where you ask for a biography of a living person and it throws in 80% real facts and 20% wild hallucinations - like saying they…
A better way to put it: It's a bug of the "GPT-4 + Chatbot interface intended to be useful" combination, rather than a bug of GPT-4.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#347Earlier quoted context omitted.
That's interesting, what language is that? Also, is it possible to give a prompt to make ChatGPT switch to feminine gender?
Many gendered languages use male gender as the default when it's ambiguous or unknown, so that would be expected behavior. And yes, it's absolutely possible to make it switch to whatever gender you want, and generally to pretend to be whatever you want it to be. The identity comes from the conversation context (including all the hidden messages), not from the LLM.
It's the default when speaking about groups of people or people whose identity isn't known. But if someone speaks about themselves then they have to choose the right one.
This is also something that language learners have to pay attention to - to not sound weird.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#348The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…
> The part that's concerning about ChatGPT is that a computer program that is "confidently wrong"
I think we are just seeing Dunning-Krüger in the machine: It isn't smart enough to know it doesn't know. It likely isn't very far though.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#349Earlier quoted context omitted.
Lying implies a hefty dose of anthropomorphism. Lying means a lot of things, all of which have to do with intelligence. By telling it lies you actually make it seem more intelligent. > human intelligence is not merely pattern matching Citation needed
I agree that lying also has a connotation of intent and therefore intelligence, and so is at least technically inappropriate. Yet I agree with the researchers that the benefits of using "lying" instead of the more technically accurate "confabulation" outweigh the negatives. In communicating with new users who may over-rely on it, getting across highly technically correct metaphors is less important than delivering th…
What’s wrong with: “it is often wrong”?
I - or GPT - can think of various variations that do not have links with “lying”. I think it is a bad idea.
Let people use it and see for themselves. It has unique failure modes, not unlike every other tool we have. You will only learn by interacting with it. Some people are in the habit of “protecting the people” but I am not quite sure that is necessary.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#350Earlier quoted context omitted.
This whole conversation is speculative, obviously. The AI doomers are all speculating without evidence as well. The tendency GPT and other LLMs to hallucinate is clearly documented. Is this not evidence? I think it's fair to predict that if we can't solve or mitigate this problem, it's going to put a significant cap on this kind of AI's usefulness and become a blocker to reaching AGI.
I believe the relevant assertion is > Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence To me that is an open question. OpenAI has not revealed a lot of the relevant stats behind GPT-4. I also haven't seen anything about their future LLM research (though I haven't looked very hard), but I think the above assertion remains to be seen.
It seems like people are focusing on training with more data and working the data. A human does not need to learn all the contents of the internet to learn skepticism and logic. A smart Ai should probably not either, that would point me to a different approach and those are not predictable. I have no doubt we can make gpt better and more useful. I do have some doubts we can make it trustworthy enough for most of its taunted potential.