Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft's AI shopping announcement contains hallucinations in the demo

perfectrec.com

Post reply on HN
Print view View original

101–108 of 108 posts

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#101

Post by JieJie »

tremon wrote: post #69
imchillyb wrote: post #10

A hallucination is an unexpected emergence. The 'making up' facts, because it cannot determine a fact from fiction, is entirely expected behavior. There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. The bullshit comes from there being no model of trust these AIs subscribe to. I'd love-love-love to see these AI producers be held to some responsibility…

because it cannot determine a fact from fiction This is way too narrow. Even if it were able to determine fact from fiction, a neural network would still be able to hallucinate as long as it has no ontology: if it doesn't "know" the boundary between objects it has no way of knowing the atomicity of its facts, so it will inevitably combine even known "facts" into falsehoods. To illustrate, the following fact-based syl…

"Lies are attempts to hide the truth by willfully denying facts. Fiction, on the other hand, is an attempt to reveal the truth by ignoring facts. — John Green

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#102

Post by JieJie »

BaculumMeumEst wrote: post #54
meindnoch wrote: post #42

Earlier quoted context omitted.

https://en.m.wikipedia.org/wiki/Confabulation

That seems more technically accurate but less likely to catch on since it's a much less common word. I think my parents are a lot more likely to understand from context a news story that mentions that "lawyers relied on an AI that hallucinated court cases" than "lawyers relied on an AI that confabulated court cases".

I feel like "fabrication" is just sitting right there begging to be used.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#103

Post by JieJie »

tremon wrote: post #72
irrational wrote: post #31

Earlier quoted context omitted.

Call them confabulations. "Confabulation refers to the production or creation of false or erroneous memories without the intent to deceive, sometimes called 'honest lying'" "Confabulation is the creation of false memories in the absence of intentions of deception. Individuals who confabulate have no recognition that the information being relayed to others is fabricated. Confabulating individuals are not intentionally…

The words confabulation and lie have the same problem when applied to the current state of "AI": they embody are certain level of intent; in the former case, the implication is that there was no intent to deceive, while a lie is the opposite. Still, they both imply intent, and for as far as I know nobody has been able to conclusively demonstrate intent on the part of a chatbot. Hallucination doesn't require intent.

I like "fabrication" because it correctly implies the data is being pulled from somewhere and assembled. It does not put a judgment on the initial data, nor on the assembled product.

It can take on a positive or negative meaning, depending on the context.

"ChatGPT fabricated an answer that was technically correct, but misleading," or "ChatGPT was able to fabricate an innovative solution that had eluded us."

Edit: And of course, everyone's favorite, "ChatGPT found guilty of fabricating case citations."

https://www.techspot.com/news/98860-chatgpt-found-guilty-fab...

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#104

Post by hmcq6 »

circuit10 wrote: post #100
hmcq6 wrote: post #99

Earlier quoted context omitted.

You're misunderstanding the thought experiment then. By definition the person inside the Chinese Room doesn't understand Chinese. > but that doesn't mean the state of the program doesn't contain any understanding or intelligence Programs don't contain understanding or intelligence, they contain instructions. > We're not asking whether the computer chip itself understands things but whether the something contained in…

> “You're misunderstanding the thought experiment then.” So if I don’t agree with it, I’m misunderstanding it? It even says in the Wikipedia article for it: > "The overwhelming majority", notes BBS editor Stevan Harnad, "still think that the Chinese Room Argument is dead wrong". So don’t try to pretend it’s some absolute truth, it’s just a flawed argument > Programs don't contain understanding or intelligence, they c…

> So if I don’t agree with it, I’m misunderstanding it?

Now you're misunderstanding me. I'm not saying you're not allowed to have a different opinion on the full thought experiment. You're assuming intelligence in the setup of the thought experiment and that is objectively not how it is meant to be interpreted.

> Why can intelligence and understanding not come from a sufficiently complex set of instructions?

Again I didn't say it can't just that convolutional neural networks as they currently exist are not that complex. It's a fancy Markov chain.

> I guess you will always win if you redefine all the words to mean what you want.

You say directly after making up your own definition of intelligence. I'm not interested in discussing your definition of intelligence or the definition of intelligence, I'm talking about this specific application of technology and if it meets a common definition of intelligence. Please point to a dictionary definition if you wanna continue this back and forth

> Learning? Of course they are, I don’t see how you can argue that they aren’t

Because learning has a definition. Theres a reason AI researchers call it "Training" and not "Teaching"

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#105

Post by 12_throw_away »

circuit10 wrote: post #40
12_throw_away wrote: post #28

Earlier quoted context omitted.

> There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. Exactly. These are models that predict text sequences. These sequences often semantically express falsehoods, but the model's not "lying", it's not "hallucinating", and it's definitely not malfunctioning. It's doing exactly what it was designed to do. There definitely are "lies" and "hallucinations…

It clearly has some level of intelligence, though it’s pretty far from human level. The hallucinations don’t make it less intelligent because it’s not “trying” to avoid them, as you seem to know already

> It clearly has some level of intelligence

Absolutely not, this is not remotely "clear", and it's a very strange thing to assert.

> The hallucinations don’t make it less intelligent because it’s not “trying” to avoid them, as you seem to know already

What? No. What does "as you seem to know already" mean in this context?

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#106

Post by circuit10 »

12_throw_away wrote: post #105
circuit10 wrote: post #40

Earlier quoted context omitted.

It clearly has some level of intelligence, though it’s pretty far from human level. The hallucinations don’t make it less intelligent because it’s not “trying” to avoid them, as you seem to know already

> It clearly has some level of intelligence Absolutely not, this is not remotely "clear", and it's a very strange thing to assert. > The hallucinations don’t make it less intelligent because it’s not “trying” to avoid them, as you seem to know already What? No. What does "as you seem to know already" mean in this context?

> Absolutely not, this is not remotely "clear", and it's a very strange thing to assert.

I guess it depends how you define intelligence but I guess I would say intelligence is the ability to find the best action to take to achieve a certain goal, and AI can do that reasonably well

> What does "as you seem to know already" mean in this context?

It means that based on the comment I was replying to the person seems to already understand what I just said

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#107

Post by circuit10 »

hmcq6 wrote: post #104
circuit10 wrote: post #100

Earlier quoted context omitted.

> “You're misunderstanding the thought experiment then.” So if I don’t agree with it, I’m misunderstanding it? It even says in the Wikipedia article for it: > "The overwhelming majority", notes BBS editor Stevan Harnad, "still think that the Chinese Room Argument is dead wrong". So don’t try to pretend it’s some absolute truth, it’s just a flawed argument > Programs don't contain understanding or intelligence, they c…

> So if I don’t agree with it, I’m misunderstanding it? Now you're misunderstanding me. I'm not saying you're not allowed to have a different opinion on the full thought experiment. You're assuming intelligence in the setup of the thought experiment and that is objectively not how it is meant to be interpreted. > Why can intelligence and understanding not come from a sufficiently complex set of instructions? Again I…

> You say directly after making up your own definition of intelligence.

I guess but I think the one I’m using is more common and useful. The Google dictionary says “the ability to acquire and apply knowledge and skills” which is closer to mine (having knowledge and the ability to apply it) than yours (some abstract idea of consciousness that can’t be measured)

> Theres a reason AI researchers call it "Training" and not "Teaching"

They also call it machine learning

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#108

Post by hmcq6 »

circuit10 wrote: post #107
hmcq6 wrote: post #104

Earlier quoted context omitted.

> So if I don’t agree with it, I’m misunderstanding it? Now you're misunderstanding me. I'm not saying you're not allowed to have a different opinion on the full thought experiment. You're assuming intelligence in the setup of the thought experiment and that is objectively not how it is meant to be interpreted. > Why can intelligence and understanding not come from a sufficiently complex set of instructions? Again I…

> You say directly after making up your own definition of intelligence. I guess but I think the one I’m using is more common and useful. The Google dictionary says “the ability to acquire and apply knowledge and skills” which is closer to mine (having knowledge and the ability to apply it) than yours (some abstract idea of consciousness that can’t be measured) > Theres a reason AI researchers call it "Training" and n…

> the ability to acquire and apply knowledge and skills

Ok but what is knowledge? You need to follow that rabbit hole. Knowledge isn't just data. You'll find that knowledge is frequently defined with some tie in to experience and the definition of experience is tied to consciousness.

> They also call it machine learning

They have called the field Artificial Intelligence (or ML) since 1956 but that doesn't mean they had an example of an instance of artificial intelligence. It's just the name of the field. I've never heard of a researcher referring to the act of training as "machine learning" though, just the field.

Post reply on HN