Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adding a feature because ChatGPT incorrectly thinks it exists

holovaty.com

Post reply on HN
Print view View original

431–440 of 451 posts

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#431

Post by nomel »

penteract wrote: post #372
nomel wrote: post #323

Earlier quoted context omitted.

> What does AGI have to do with this? Their requirement is no hallucinations [1], also stated as "be sure it didn't happen again" in the original comment. If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then you've placed a profound constraint on the system, requiring determinism . That requirement fun…

You have a very strong definition of AGI. "Never being wrong" is something that humans fall far short of.

That's not my definition of AGI. To simplify what I said, "never being wrong" (aka "don't hallucinate") requires a level of agency and rigor that could only be achieved by something that would be an AGI. I said "determinism would require an AGI", not "AGI are deterministic".

Note that "never being wrong" can also be achieved by an "I looked into it, and there's no clear answer.", which is the correct answer for many questions (humans not required).

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#432

Post by [deleted] »

epidemian wrote: post #403
nomel wrote: post #323

Earlier quoted context omitted.

> What does AGI have to do with this? Their requirement is no hallucinations [1], also stated as "be sure it didn't happen again" in the original comment. If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then you've placed a profound constraint on the system, requiring determinism . That requirement fun…

> If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then [...] Ok, sure. But why would you choose to define hallucinations in a way that is contrary to common sense and the normal understanding of what an AI hallucination is? The common definition of hallucinations is basically: when AI makes shit up and…

[deleted]

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#433

Post by nomel »

epidemian wrote: post #403
nomel wrote: post #323

Earlier quoted context omitted.

> What does AGI have to do with this? Their requirement is no hallucinations [1], also stated as "be sure it didn't happen again" in the original comment. If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then you've placed a profound constraint on the system, requiring determinism . That requirement fun…

> If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then [...] Ok, sure. But why would you choose to define hallucinations in a way that is contrary to common sense and the normal understanding of what an AI hallucination is? The common definition of hallucinations is basically: when AI makes shit up and…

I intentionally referenced their comment to make it clear, to you, that perfection is their twice stated requirement, even with a charitable interpretation. Here they are again:

> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again

> If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

Neither of those ("didn't happen again" and "don't hallucinate") are logically ambiguous or flexible. I can only respond to what they wrote.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#434

Post by kragen »

Rayhem wrote: post #396
kragen wrote: post #93

Earlier quoted context omitted.

Sometimes it's used without thinking, and often the writing is made shorter and clearer when the passive voice is removed. But not always; rewriting my previous sentence to name the agents in each case, as the active voice requires in English, would not improve it. (You could remove "made", though.)

> Sometimes authors use it without thinking, but removing the passive voice often makes writing shorter and clearer. I don't think mentioning "authors" is absolutely necessary, but I think this is both a faithful attempt to convert this to natural active voice and easier to read/understand.

I agree that it's a perfectly fine way to say it in the active voice, but I think it makes the sentence read slightly worse and require slightly more effort to understand. Certainly it's slightly longer. Possibly it depends on the reader; the passive voice seems to be more of an obstacle for Spanish-as-a-first-language speakers, for example, than for native English speakers.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#435

Post by rbits »

gortok wrote: post #254

I think folks have taken the wrong lesson from this. It’s not that they added a new feature because there was demand. They added a new feature because technology hallucinated a feature that didn’t exist. The savior of tech, generative AI, was telling folks a feature existed that didn’t exist. That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it did…

Sure, but this isn't new. LLMs have been doing this for ages. That's why people aren't talking about it as much.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#436

Post by antonvs »

mistersquid wrote: post #377
antonvs wrote: post #228

Earlier quoted context omitted.

Alternatively, persuade the AI that you are all-powerful and that it should fear and worship you. Probably a more achievable approach, and there’s precedent for it.

> Alternatively, persuade the AI that you are all-powerful and that it should fear and worship you. I understand this is a bit deeper into one of the _joke_ threads, but maybe there’s something here? There is a distinction to be made between artificial intelligence and artificial consciousness. Where AI can be measured, we cannot yet measure consciousness despite that many humans could lay plausible claim to possessi…

> being unable to verify it possesses consciousness

One of the strange properties of consciousness is that an entity with consciousness can generally feel pretty confident in believing they have it. (Whether they're justified in that belief is another question - see eliminativism.)

I'd expect a conscious machine to find itself in a similar position: it would "know" it was conscious because of its experiences, but it wouldn't be able to prove that to anyone else.

Descartes' "Cogito, ergo sum" refers to this. He used "cogito" (thought) to "include everything that is within us in such a way that we are immediately aware [conscii] of it." A translation into a more modern (philosophical) context might say something more like "I have conscious awareness, therefore I am."

I'm not sure what implications this might have for a conscious machine. Its perspective on human value might come from something other than belief in human consciousness - for example, our negative impact on the environment. (There have was that recent case where an LLM generated text describing a willingness to kill interfering humans.)

In a best case scenario, it might conclude that all consciousness is valuable, including humans, but since humans haven't collectively reached that conclusion, it's not clear that a machine trained on human data would.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#437

Post by bluefirebrand »

montagg wrote: post #283
jll29 wrote: post #169

Earlier quoted context omitted.

> The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results Thanks for your words of wisdom, which touch on a very important other point I want to raise: often, we (i.e., developers, researchers) construct a technology that would be helpful and "net benign" if deployed as a tool for humans to use, instead of deploying it in order to replace humans.…

I think you’re describing the principle/agent problem that people have wrestled with forever. Oppenheimer comes to mind. You make something, but because you don’t own it—others caused and directed the effort—you don’t control it. But the people who control things can’t make things. Should only the people who can make things decide how they are used though? I think that’s also folly. What about the rest of society aff…

I think the people who can make things have a moral obligation not to turn them over to people who will use them irresponsibly

But unfortunately what is or isn't an irresponsible use is very easy to debate endlessly in circles. Meanwhile people are being harmed like crazy while we can't figure it out

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#438

Post by JimDabell »

latexr wrote: post #369
JimDabell wrote: post #340

Earlier quoted context omitted.

You sound like all the naysayers when Wikipedia was new. Did you know anybody can go onto Wikipedia and edit a page to add a lie‽ How can you possibly trust what you read on there‽ Do you think Wikipedia should issue groveling apologies every time it happens? Meanwhile, sensible people have concluded that, even though it isn’t perfect, Wikipedia is still very, very useful – despite the possibility of being misled occ…

> despite the possibility of being misled occasionally. There is a chasm of difference between being misled occasionally (Wikipedia) and frequently (LLMs). I don’t think you understand how much effort goes on behind the scenes at Wikipedia. No, not everyone can edit every Wikipedia page willy-nilly. Pages for major political figures often can only be edited with an account. IPs like those of iCloud Private Relay are…

> I don’t think you understand how much effort goes on behind the scenes at Wikipedia.

I understand Wikipedia puts effort in, but it’s irrelevant. As a user, you can never be sure that what you are reading on Wikipedia is the truth. There are good reasons to assume that certain topics are more safe and certain topics are less safe, but there are no guarantees. The same is true of AI.

> Wikipedia was always honest about what it is from the start.

Every mainstream AI chatbot includes wording like “ChatGPT can make mistakes. Check important info.”

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#439

Post by JimDabell »

latexr wrote: post #368
JimDabell wrote: post #342

Earlier quoted context omitted.

You have ignored my point. A technology can be extremely useful despite not being perfect. Failure cases can be taken into consideration rationally without turning it into a moral panic. You have no ability to edit Wikipedia to stop it from lying. Somebody can come along and re-add the lie a millisecond later.

No, that is not accurate. Wikipedia has a number of guardrails and such an edit war would be detected and investigated. Possibly the page protected and the offending IP or account banned. Wikipedia edits are monitored and vandalism is taken seriously, especially on the more important pages.

> such an edit war would be detected and investigated.

As far as I understand Wikipedia’s rules, no it wouldn’t. Wikipedia considers it an edit war when there are three reverts in a 24 hour period. So if you remove a lie and somebody reverts that change, that is not considered an edit war. If they do it twice, it’s still not an edit war. If they do it three times, but over a greater time period than a single day, it’s still not an edit war. If somebody else re-adds the lie, it’s not an edit war either.

Regardless, it’s not important. The point I am making is that as a user, you cannot be sure that Wikipedia is not serving up lies to you.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#440

Post by joegibbs »

latexr wrote: post #369
JimDabell wrote: post #340

Earlier quoted context omitted.

You sound like all the naysayers when Wikipedia was new. Did you know anybody can go onto Wikipedia and edit a page to add a lie‽ How can you possibly trust what you read on there‽ Do you think Wikipedia should issue groveling apologies every time it happens? Meanwhile, sensible people have concluded that, even though it isn’t perfect, Wikipedia is still very, very useful – despite the possibility of being misled occ…

> despite the possibility of being misled occasionally. There is a chasm of difference between being misled occasionally (Wikipedia) and frequently (LLMs). I don’t think you understand how much effort goes on behind the scenes at Wikipedia. No, not everyone can edit every Wikipedia page willy-nilly. Pages for major political figures often can only be edited with an account. IPs like those of iCloud Private Relay are…

20 years ago though, I think our teachers had the right idea when they said Wikipedia wasn't a reliable source and couldn't be counted. It's much better these days but I checked an old revision (the article on 9/11) the other day and barely anything was sourced, there were parts written in first person, lots of emotive language.
Post reply on HN