Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft's AI shopping announcement contains hallucinations in the demo

perfectrec.com

Post reply on HN
Print view View original

91–100 of 108 posts

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#91

Post by geoduck14 »

rvz wrote: post #27
cryptozeus wrote: post #12

Is it just me or does everyone trust AI opinions less and less ? Every time I ask it to find top 5 of something, I go and double check myself and almost always find it to be wrong. For example try searching for top 5 restaurants around me in bard. Some of them dont even exist lol and some are just random if you cross verify with actual popularity from yelp etc.

Well it doesn’t surprise me since I have been saying this for a while that these LLMs hallucinate nonsense to the point where you end up triple checking whatever it outputs. LLMs thrive in applications that involve creativity and non-serious applications mostly around fantasy or creative writing. Anyone using them seriously outside of summarization for high risk use cases is going to be very disappointed.

I recommend LLM users leverage the RAG technique

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#92

Post by sorokod »

cubefox wrote: post #84
sorokod wrote: post #82

Earlier quoted context omitted.

I get different answers every time to "what is the third element in the periodic table" from llama2. I'll hold off actually using them for now.

That model is not state of the art. Even GPT-3.5 can answer this question.

On GPT 3.5

Q: "What is the seventy fourth element of the periodic table?"

A: "The seventy-fourth element of the periodic table is Rhenium..."

But this is really shooting fish in a barrel. Given the way LLMs work why would you expect them to provide factually correct text completion?

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#93

Post by DanHulton »

cubefox wrote: post #66
imchillyb wrote: post #10

A hallucination is an unexpected emergence. The 'making up' facts, because it cannot determine a fact from fiction, is entirely expected behavior. There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. The bullshit comes from there being no model of trust these AIs subscribe to. I'd love-love-love to see these AI producers be held to some responsibility…

Speaking with GPT-4, it is hard to deny the conjecture that its weights encode an internal world model somewhere. If so, the difficulty is not that the model has no conception of truth and falsity, it is rather to motivate the model to tell the truth. Or more precisely, to let the model be honest, to only tell things it believes to be true, things which are part of its world model. Unfortunately, we can't just tell t…

It is, like you said, conjecture. The best we can say is that it _usually_ provides responses that are _consistent_ with responses coming from an intelligence with an internal world model. That doesn't mean that's the only way to get those responses, nor does it mean that this is necessarily what's happening in this case.

So saying things like "the model has come to the conclusion that" or "smarter than", or "learns to be deceptive", I think that's premature at best. I'm not yet convinced that there's sufficient evidence to show appreciable internal state and logical processes. There's so, so many examples where what looks like legit understanding breaks down with the slightest tweak to the prompt, and it goes from looking like a savant to someone high on just a tremendous amount of LSD.

If there was an internal world model that just wasn't correct, I would expect to see its incorrect answers be at least logically consistent, but instead it looks way, way more like the trick just doesn't work for this case.

So to get back to the original point, this is MS trying to leverage this trick to do a task that requires actual logical reasoning, factual evaluation, and internal world state, and we're just not there. (I hesitate to use the word "yet", because there's still a lot of not-yet-conclusive discussion around whether current LLM techniques will ever get us "there." Colour me tentatively pessimistic in the meantime. =) )

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#94

Post by irrational »

tremon wrote: post #72
irrational wrote: post #31

Earlier quoted context omitted.

Call them confabulations. "Confabulation refers to the production or creation of false or erroneous memories without the intent to deceive, sometimes called 'honest lying'" "Confabulation is the creation of false memories in the absence of intentions of deception. Individuals who confabulate have no recognition that the information being relayed to others is fabricated. Confabulating individuals are not intentionally…

The words confabulation and lie have the same problem when applied to the current state of "AI": they embody are certain level of intent; in the former case, the implication is that there was no intent to deceive, while a lie is the opposite. Still, they both imply intent, and for as far as I know nobody has been able to conclusively demonstrate intent on the part of a chatbot. Hallucination doesn't require intent.

Hallucinations are defined as “Perception of visual, auditory, tactile, olfactory, or gustatory experiences without an external stimulus and with a compelling sense of their reality, usually resulting from a mental disorder or as a response to a drug.”

That doesn’t sound like what AI/LLMs are doing, at all. There is no mental disorder or drugs causing then to output what we would consider to be false information. The machine is not perceiving anything without an external stimulus. Everything they generate is from the stimulus we have given it.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#95

Post by TheRealPomax »

overnight5349 wrote: post #75
TheRealPomax wrote: post #22

Earlier quoted context omitted.

"Making shit up in order to fulfill some requirement" is the definition of lying, so whether it's a human or an AI, just making shit up in order to generate prompted output is flat out lying. Not "hallucinating". And the best part is that until LLM get valitidy checks baked in, even the things they get right are lies if presented with authority, because the LLM doesn't know whether it's true or not. In fact, the LLM…

Your argument is that any untruth is a lie. Do you consider fiction authors to be liars?

No, my argument is that if you don't KNOW what truth is, everything you say is a lie.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#96

Post by TheRealPomax »

irrational wrote: post #30
TheRealPomax wrote: post #22

Earlier quoted context omitted.

"Making shit up in order to fulfill some requirement" is the definition of lying, so whether it's a human or an AI, just making shit up in order to generate prompted output is flat out lying. Not "hallucinating". And the best part is that until LLM get valitidy checks baked in, even the things they get right are lies if presented with authority, because the LLM doesn't know whether it's true or not. In fact, the LLM…

> "Making shit up in order to fulfill some requirement" is the definition of lying I'd argue that there is an element of intent or agency involved. When a human makes things up intentionally or by choice, that is lying. When they do it unintentionally, that is not lying. It is usually called confabulation (or, honest lying - where the actor does not know they are not telling the truth). I don't think AIs/LLMs have ag…

The AI intentionally makes things up because that's literally the whole point of the LLM concept, both abstractly and concretely. WE MAKE IT LIE by conditioning it to lie. The "AI" part is not separate from the humans who made it, we are part of the system, and we made a computer that constantly and continuously lies in order to generate seemingly credible responses.

We made a computer that lies, all the time, about everything.

"...Why?"

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#97

Post by circuit10 »

hmcq6 wrote: post #76
circuit10 wrote: post #40

Earlier quoted context omitted.

It clearly has some level of intelligence, though it’s pretty far from human level. The hallucinations don’t make it less intelligent because it’s not “trying” to avoid them, as you seem to know already

> It clearly has some level of intelligence https://plato.stanford.edu/entries/chinese-room/#LargPhilIss...

To me the Chinese Room thought experiment seems like it's meant to show that AIs can be intelligent, not the opposite?

"Searle could receive Chinese characters through a slot in the door, process them according to the program's instructions, and produce Chinese characters as output, without understanding any of the content of the Chinese writing."

Sure, but that doesn't mean the state of the program doesn't contain any understanding or intelligence, it's just that the human doesn't have a high-level view that can be used to decode that internal state. We're not asking whether the computer chip itself understands things but whether the something contained in the program running on it does. The human could also run a physics simulation as in https://xkcd.com/505/ and recreate a human brain which would be no different to a physical brain in terms of behavior and so there would be no reason not to call it intelligent

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#98

Post by SirMaster »

ethanbond wrote: post #87
SirMaster wrote: post #83

Earlier quoted context omitted.

The problem I have with this is it seems too generic. Shouldn't we try to categorize the types of errors at least somewhat?

Sure, and it's probably wise not to pick error categorizations that impute consciousness onto a thing that's 1) almost certainly not conscious, 2) behaves quite similarly to a conscious thing, and 3) might become conscious at some point in the near or distant future Hallucinations are definitionally features of conscious experience. Pick a different word or make one up!

I guess to me hallucination doesn't necessarily imply consciousness.

"an experience involving the apparent perception of something not present"

I guess it depends on exactly how you define "experience" and "perception".

But yeah there are better words than hallucination that are even more generic and do work better.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#99

Post by hmcq6 »

circuit10 wrote: post #97
hmcq6 wrote: post #76

Earlier quoted context omitted.

> It clearly has some level of intelligence https://plato.stanford.edu/entries/chinese-room/#LargPhilIss...

To me the Chinese Room thought experiment seems like it's meant to show that AIs can be intelligent, not the opposite? "Searle could receive Chinese characters through a slot in the door, process them according to the program's instructions, and produce Chinese characters as output, without understanding any of the content of the Chinese writing." Sure, but that doesn't mean the state of the program doesn't contain a…

You're misunderstanding the thought experiment then. By definition the person inside the Chinese Room doesn't understand Chinese.

> but that doesn't mean the state of the program doesn't contain any understanding or intelligence

Programs don't contain understanding or intelligence, they contain instructions.

> We're not asking whether the computer chip itself understands things but whether the something contained in the program running on it does.

I feel like your saying "I'm not accusing the blender of being intelligent, I'm saying the recipe for this margarita is self aware." It doesn't matter if its hardware or software, neither is capable of understanding because understanding is a conscious experience and neither a blender nor a recipe are sentient.

> The human could also run a physics simulation

Cool XKCD but I'm not arguing about wether AI is possible. Just pointing out that convolutional neural networks are not self aware or intelligent or actually learning (at least not yet).

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#100

Post by circuit10 »

hmcq6 wrote: post #99
circuit10 wrote: post #97

Earlier quoted context omitted.

To me the Chinese Room thought experiment seems like it's meant to show that AIs can be intelligent, not the opposite? "Searle could receive Chinese characters through a slot in the door, process them according to the program's instructions, and produce Chinese characters as output, without understanding any of the content of the Chinese writing." Sure, but that doesn't mean the state of the program doesn't contain a…

You're misunderstanding the thought experiment then. By definition the person inside the Chinese Room doesn't understand Chinese. > but that doesn't mean the state of the program doesn't contain any understanding or intelligence Programs don't contain understanding or intelligence, they contain instructions. > We're not asking whether the computer chip itself understands things but whether the something contained in…

> “You're misunderstanding the thought experiment then.”

So if I don’t agree with it, I’m misunderstanding it? It even says in the Wikipedia article for it:

> "The overwhelming majority", notes BBS editor Stevan Harnad, "still think that the Chinese Room Argument is dead wrong".

So don’t try to pretend it’s some absolute truth, it’s just a flawed argument

> Programs don't contain understanding or intelligence, they contain instructions.

Why can intelligence and understanding not come from a sufficiently complex set of instructions?

> understanding is a conscious experience and neither a blender nor a recipe are sentient.

That’s an odd definition of understanding. By my definition understanding is having information about something and the ability to process it such that you can effectively predict its behaviour and possibly take actions to change its state to fit a goal. I guess you will always win if you redefine all the words to mean what you want. Your definition is useless because it’s unfalsifiable because you can’t measure whether something is “sentient”

> Just pointing out that convolutional neural networks are not self aware or intelligent or actually learning

Self aware? Probably no

Intelligent? To some extent, yes

Learning? Of course they are, I don’t see how you can argue that they aren’t

Post reply on HN