Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft's AI shopping announcement contains hallucinations in the demo

perfectrec.com

Post reply on HN
Print view View original

31–40 of 108 posts

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#31

Post by irrational »

barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

Call them confabulations.

"Confabulation refers to the production or creation of false or erroneous memories without the intent to deceive, sometimes called 'honest lying'"

"Confabulation is the creation of false memories in the absence of intentions of deception. Individuals who confabulate have no recognition that the information being relayed to others is fabricated. Confabulating individuals are not intentionally being deceptive and sincerely believe the information they are communicating to be genuine and accurate."

https://clinmedjournals.org/articles/ijnn/international-jour...

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#32

Post by batch12 »

barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

Maybe we could just call it babbling.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#33

Post by nowooski »

2bitencryption wrote: post #25
siva7 wrote: post #21

Opinion pieces like shopping recommendations are quite hard for current LLMs. Either it is a hard fact - or pure creative work - that's where AI shines. Anything between and things get tricky

This is one of those areas where the poor quality of the data influences the output, I think. There are so many garbage, lazily written product reviews, by websites that only exist to get people to click affiliate links. These sites only have one goal, which is to get you to click an affiliate link and make a purchase. So it is not in their best interest to say "You shouldn't buy this." Rather, they make a list of "t…

For sure. The garbage in, garbage out problem is quite real for ecommerce applications.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#34

Post by CSSer »

cubefox wrote: post #26
cryptozeus wrote: post #12

Is it just me or does everyone trust AI opinions less and less ? Every time I ask it to find top 5 of something, I go and double check myself and almost always find it to be wrong. For example try searching for top 5 restaurants around me in bard. Some of them dont even exist lol and some are just random if you cross verify with actual popularity from yelp etc.

Using language models for location or time based things is not recommended, as this usually requires non-textual data. Better to use them for general knowledge questions, programming help, translation, or writing. Asking them to do any complex calculations (especially when they also require non-text raw data, like inflation in a given time period) is also futile.

> general knowledge questions, programming help, translation, or writing.

They get all of these wrong too. It's like some AI-specific variant of the Gell-man amnesia effect. It's usually right in the first sentence, but if you really know the answer, it's often either very debatable or completely wrong by the halfway mark of the paragraph. Meanwhile, the associated brand authority is problematic.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#35

Post by BaculumMeumEst »

barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

> If we're going to anthropomorphize AIs, let's just call it bullshitting and lies.

why? "bullshitting and lies" suggests that the AI is intentionally being deceptive. "hallucinations" conveys the idea that the information is incorrect, but the AI perceives it to be correct, which is more in line with what is actually happening.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#36

Post by vineyardmike »

barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

If I make a claim based on prior knowledge and statistics I’ve learned over time, it’s not lying if it’s wrong. Lying has intent. Plenty of people say incorrect facts that they think are correct.

In second grade, my cousin talked a lot about flax farmers in South America, after learning about them in class. Turns out the lesson was on quinoa farmers, and he forgot the original produce and “hallucinated” the statistics about flax farmers instead. Technically the term is confabulation. Was he lying? No because he wasn’t trying to tell us fake facts.

LLMs have no intention of being wrong. Their “hallucinations” or whatever are just whatever makes sense from their statistical models. They’re really just confabulations.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#37

Post by majormajor »

vineyardmike wrote: post #36
barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

If I make a claim based on prior knowledge and statistics I’ve learned over time, it’s not lying if it’s wrong. Lying has intent. Plenty of people say incorrect facts that they think are correct. In second grade, my cousin talked a lot about flax farmers in South America, after learning about them in class. Turns out the lesson was on quinoa farmers, and he forgot the original produce and “hallucinated” the statistic…

"Bullshitting" seems like a good term for accurate or inaccurate responses.

Let's extend "LLMs have no intention of being wrong" to "LLMs have no inherent sense of being correct" - sometimes their predictions happen to be correct, sometimes they don't. But they're all hallucinations generated from the same process.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#38

Post by cubefox »

2bitencryption wrote: post #25
siva7 wrote: post #21

Opinion pieces like shopping recommendations are quite hard for current LLMs. Either it is a hard fact - or pure creative work - that's where AI shines. Anything between and things get tricky

This is one of those areas where the poor quality of the data influences the output, I think. There are so many garbage, lazily written product reviews, by websites that only exist to get people to click affiliate links. These sites only have one goal, which is to get you to click an affiliate link and make a purchase. So it is not in their best interest to say "You shouldn't buy this." Rather, they make a list of "t…

There is an argument to be made for automatically downweighting (be it training epochs or pagerank rating) anything with affiliate links. But I guess it would be trivial to hide them behind a redirect.

That being said, I recently asked the Bing chatbot about the difference between two similar sounding printer models, and it gave a good explanation which I previously couldn't quickly find via Google. In case of Bing it is sometimes not completely clear to which degree its answer depends on the Web search, if it performed one, and to which degree it is just answering from its background knowledge (which could be prone to hallucination, but is less "gullible", so to speak). It provides sources, but not everything it says is necessarily present in the source. I'm actually surprised how quickly Bing is able to search (load and read) multiple websites, given that the loading times are not always trivial. It turns out they are much faster at reading than at typing. Indeed, each forward pass reads the entire context window, so once for every generated token!

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#40

Post by circuit10 »

12_throw_away wrote: post #28
imchillyb wrote: post #10

A hallucination is an unexpected emergence. The 'making up' facts, because it cannot determine a fact from fiction, is entirely expected behavior. There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. The bullshit comes from there being no model of trust these AIs subscribe to. I'd love-love-love to see these AI producers be held to some responsibility…

> There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. Exactly. These are models that predict text sequences. These sequences often semantically express falsehoods, but the model's not "lying", it's not "hallucinating", and it's definitely not malfunctioning. It's doing exactly what it was designed to do. There definitely are "lies" and "hallucinations…

It clearly has some level of intelligence, though it’s pretty far from human level. The hallucinations don’t make it less intelligent because it’s not “trying” to avoid them, as you seem to know already
Post reply on HN