Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft's AI shopping announcement contains hallucinations in the demo

perfectrec.com

Post reply on HN
Print view View original

61–70 of 108 posts

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#61

Post by tiffanyg »

barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

Ha! Yup, one of my friends who has been working with "transformer models" for years now told me "oh yeah, it's a bullshitter" when I tried my hand and got some truly bizarre, digressive, "addled", etc. output.

OTOH, it reminded me very much of my own mind (reinforced by ADHD, in my case).

This suggests to me, at least, that "the problem" isn't these models, per se. It's more like: these are probably only one module / layer in a system more similar to our brains. Just as scientists have identified distinct regions (more) involved in, say, language production, or (direct) visual perception, or etc., I'd suggest we've only just built the first substantially more practical / realistic hack / simulation (much like 3D game engines almost always use hacks - e.g., not even using the simple "Newtonian optics" model fully [i.e., "ray tracing"]) of a sort of language cortex. I'd further guess that it's going to take some maturation of a number of methods, technologies, etc. to realistically add more "cortices", but, I do think it's quite likely to happen in approx. the "decades" range...

Highly highly speculative - rather naively based on the way other technologies have developed and with a little basis in work I've done more directly in neurobio etc. No deep(er) reason / analysis, but, just my current very tentative hypothesis.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#62

Post by phyzome »

There is information, here, in the observations that all these "AI" demos contain blatant inaccuracies, with apparently no fact-checking having taken place. It's clear that these companies (Microsoft, Google, OpenAI) do not care about accuracy, correctness, or the truth. It is not part of their business model.

There is no respect for your time, your safety, your reputation. Your role as a customer is to be conned into using the products for long enough that a return on investment can be made; the companies will pivot to a new product as soon as the untrustworthiness of the old one becomes common knowledge.

Short-term thinking. Desperation.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#63

Post by visarga »

majormajor wrote: post #37
vineyardmike wrote: post #36

Earlier quoted context omitted.

If I make a claim based on prior knowledge and statistics I’ve learned over time, it’s not lying if it’s wrong. Lying has intent. Plenty of people say incorrect facts that they think are correct. In second grade, my cousin talked a lot about flax farmers in South America, after learning about them in class. Turns out the lesson was on quinoa farmers, and he forgot the original produce and “hallucinated” the statistic…

"Bullshitting" seems like a good term for accurate or inaccurate responses. Let's extend "LLMs have no intention of being wrong" to "LLMs have no inherent sense of being correct" - sometimes their predictions happen to be correct, sometimes they don't. But they're all hallucinations generated from the same process.

Bullshitting always requires a hidden intention to manipulate.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#64

Post by joker_minmax »

barbariangrunge wrote: post #2

Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term

Hinton called them "confabulations" according to this:

https://www.technologyreview.com/2023/05/02/1072528/geoffrey...

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#66

Post by cubefox »

imchillyb wrote: post #10

A hallucination is an unexpected emergence. The 'making up' facts, because it cannot determine a fact from fiction, is entirely expected behavior. There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. The bullshit comes from there being no model of trust these AIs subscribe to. I'd love-love-love to see these AI producers be held to some responsibility…

Speaking with GPT-4, it is hard to deny the conjecture that its weights encode an internal world model somewhere.

If so, the difficulty is not that the model has no conception of truth and falsity, it is rather to motivate the model to tell the truth. Or more precisely, to let the model be honest, to only tell things it believes to be true, things which are part of its world model.

Unfortunately, we can't just tell the model to be honest, since we can't distinguish between responses the model does or does not believe to be true. With RLHF fine-tuning, we can train the model to tend to give answers the human raters believe to be true. But we want the model to tell what it believes to be true, not what it believes that we believe is true!

For example, human raters may overwhelmingly rate response X as false, but the model, having read the entire Internet, may have come to the conclusion that X is true. So RLHF would train it to lie about X, to answer not-X instead of X.

This problem could turn out to be fatal when a model becomes significantly smarter than humans, because this means it would less often believe according to human biases and misconceptions, so it would learn to be deceptive and to tell us only what we want to believe. This could have frightening consequences if this leads it to conceal any of its possible misalignments with human values from us.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#67

Post by uoaei »

visarga wrote: post #63
majormajor wrote: post #37

Earlier quoted context omitted.

"Bullshitting" seems like a good term for accurate or inaccurate responses. Let's extend "LLMs have no intention of being wrong" to "LLMs have no inherent sense of being correct" - sometimes their predictions happen to be correct, sometimes they don't. But they're all hallucinations generated from the same process.

Bullshitting always requires a hidden intention to manipulate.

Mislead is probably a more accurate term. Bullshit exists outside of the correct/incorrect binary and serves only to build narrative.

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#69

Post by tremon »

imchillyb wrote: post #10

A hallucination is an unexpected emergence. The 'making up' facts, because it cannot determine a fact from fiction, is entirely expected behavior. There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. The bullshit comes from there being no model of trust these AIs subscribe to. I'd love-love-love to see these AI producers be held to some responsibility…

because it cannot determine a fact from fiction

This is way too narrow. Even if it were able to determine fact from fiction, a neural network would still be able to hallucinate as long as it has no ontology: if it doesn't "know" the boundary between objects it has no way of knowing the atomicity of its facts, so it will inevitably combine even known "facts" into falsehoods.

To illustrate, the following fact-based syllogism would sound perfectly valid in the absence of a working ontology:

  A: That green flask costs $10
  B: This flask is green
  => This flask costs $10

Re: Microsoft's AI shopping announcement contains hallucinations in the demo

#70

Post by majormajor »

visarga wrote: post #63
majormajor wrote: post #37

Earlier quoted context omitted.

"Bullshitting" seems like a good term for accurate or inaccurate responses. Let's extend "LLMs have no intention of being wrong" to "LLMs have no inherent sense of being correct" - sometimes their predictions happen to be correct, sometimes they don't. But they're all hallucinations generated from the same process.

Bullshitting always requires a hidden intention to manipulate.

Nah, it can be just talking without much rigor or verification of fact or anything, often with a loose boundary between opinion and fact - the "to talk in an exaggerated or foolish manner" definition.

As in "my buddies and I were bullshitting about movies the other day."

ChatGPT definitely talks with an exaggerated manner confidence-wise.

Post reply on HN