Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft's AI shopping announcement contains hallucinations in the demo

perfectrec.com

Post reply on HN
Print view View original

71–80 of 108 posts

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#71

Post by unqueued »

joker_minmax wrote: post #64
barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

Hinton called them "confabulations" according to this: https://www.technologyreview.com/2023/05/02/1072528/geoffrey...

I think "confabulation" is a way more accurate term, I wish it had stuck instead of "hallucination".

A hallucination is a problem with input. Confabulation is false output.

Confabulation is when a person mistakenly recalls details and tries to "fill in the blanks", without realizing what they are saying is untrue.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#72

Post by tremon »

irrational wrote: post #31
barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

Call them confabulations. "Confabulation refers to the production or creation of false or erroneous memories without the intent to deceive, sometimes called 'honest lying'" "Confabulation is the creation of false memories in the absence of intentions of deception. Individuals who confabulate have no recognition that the information being relayed to others is fabricated. Confabulating individuals are not intentionally…

The words confabulation and lie have the same problem when applied to the current state of "AI": they embody are certain level of intent; in the former case, the implication is that there was no intent to deceive, while a lie is the opposite. Still, they both imply intent, and for as far as I know nobody has been able to conclusively demonstrate intent on the part of a chatbot.

Hallucination doesn't require intent.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#73

Post by godelski »

Cagrosso wrote: post #47
godelski wrote: post #44

Earlier quoted context omitted.

I'm not so concerned with that as I am with the fact that this isn't one. Article says > they tend to make up fake information – errors called “hallucinations.” Hallucinations are a certain kind of error. But what appears to have happened here is a _direct_ manipulation from Microsoft. Which is a risky play by them. It doesn't take much to erode trust. People tend to trust LLMs because they tend to get things right.…

> Microsoft manipulated the output to prefer their own products and boy is that a risky strategy. Makes me wonder how they plan to monetize these chatbots and if they won’t just fizzle out like voice assistants. I don’t see how there won’t be concerns over asking a chatbot for the best pizza in town and receiving an answer like “Customers love the new Meat Lover’s Pizza from Pizza Hut! Brought to you by Pizza Hut… (l…

Chatbots just seems like the marketing tool if we're being honest. I can't see any way to monetize them without destroying them. Even just having them could threaten their own existence (or make bloggers higher qualities if Google fucking decides to fix SEO...). LLMs on the other hand have plenty of use cases, though I think people are still way over hyped on them and aren't interested in how the boring stuff has good utility.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#74

Post by cubefox »

CSSer wrote: post #34
cubefox wrote: post #26

Earlier quoted context omitted.

Using language models for location or time based things is not recommended, as this usually requires non-textual data. Better to use them for general knowledge questions, programming help, translation, or writing. Asking them to do any complex calculations (especially when they also require non-text raw data, like inflation in a given time period) is also futile.

> general knowledge questions, programming help, translation, or writing. They get all of these wrong too. It's like some AI-specific variant of the Gell-man amnesia effect. It's usually right in the first sentence, but if you really know the answer, it's often either very debatable or completely wrong by the halfway mark of the paragraph. Meanwhile, the associated brand authority is problematic.

GPT-4 outperforms average students in exams of several fields. There are a lot of benchmarks, and GPT-4 does mostly do very well as long as the field relies enough on declarative knowledge.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#75

Post by overnight5349 »

TheRealPomax wrote: post #22
wtallis wrote: post #5

Earlier quoted context omitted.

Bullshitting and lies is what the humans selling the AI-powered services are doing. Hallucination, delusion and confabulation are what the AIs are doing (and some of the humans, too).

"Making shit up in order to fulfill some requirement" is the definition of lying, so whether it's a human or an AI, just making shit up in order to generate prompted output is flat out lying. Not "hallucinating". And the best part is that until LLM get valitidy checks baked in, even the things they get right are lies if presented with authority, because the LLM doesn't know whether it's true or not. In fact, the LLM…

Your argument is that any untruth is a lie.

Do you consider fiction authors to be liars?

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#76

Post by hmcq6 »

circuit10 wrote: post #40
12_throw_away wrote: post #28

Earlier quoted context omitted.

> There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. Exactly. These are models that predict text sequences. These sequences often semantically express falsehoods, but the model's not "lying", it's not "hallucinating", and it's definitely not malfunctioning. It's doing exactly what it was designed to do. There definitely are "lies" and "hallucinations…

It clearly has some level of intelligence, though it’s pretty far from human level. The hallucinations don’t make it less intelligent because it’s not “trying” to avoid them, as you seem to know already

> It clearly has some level of intelligence

https://plato.stanford.edu/entries/chinese-room/#LargPhilIss...

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#78

Post by apomekhanes »

tiffanyg wrote: post #61
barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

Ha! Yup, one of my friends who has been working with "transformer models" for years now told me "oh yeah, it's a bullshitter" when I tried my hand and got some truly bizarre, digressive, "addled", etc. output. OTOH, it reminded me very much of my own mind (reinforced by ADHD, in my case). This suggests to me, at least, that "the problem" isn't these models, per se. It's more like: these are probably only one module /…

Not sure why you're being downvoted.

Are there other opinions about the cortex or module idea? Is there a fundamental problem with that idea I'm missing?

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#79

Post by thewataccount »

jarofghosts wrote: post #58
sporadicallyjoe wrote: post #56

Is anyone shipping AI products that DO NOT contain hallucinations? I thought that was pretty much a given.

Hallucinating is roughly how they work, we just label it as such when it's something obviously weird

This is something I'm not sure people understand.

LLM's only make a "best guess" for each next token. That's it. When it's wrong we call it a "hallucination" but really the entire thing was a "hallucination" to begin with.

This is also analogous to humans - who also "hallucinate" incorrect answers, usually "hallucinate" incorrect answers less when they "Think through this step by step before giving your answer", etc.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#80

Post by thewataccount »

sporadicallyjoe wrote: post #56

Is anyone shipping AI products that DO NOT contain hallucinations? I thought that was pretty much a given.

Well there isn't a human that never "hallucinates" in meaning we use for LLMs aka gives "incorrect answers" confidently.

Human's brains use lots of heuristics - we don't "think step by step" through everything - instead we rapidly construct an answer for almost everything.

What we say is "hallucinations" for AI in humans is "misspeaking, misremembering anything, off by 1 math/counting, missidentifying someone, using the wrong variable/method when programming, etc."

Post reply on HN