Opinion pieces like shopping recommendations are quite hard for current LLMs. Either it is a hard fact - or pure creative work - that's where AI shines. Anything between and things get tricky
Microsoft's AI shopping announcement contains hallucinations in the demo
21–30 of 108 posts
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#22Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term
Bullshitting and lies is what the humans selling the AI-powered services are doing. Hallucination, delusion and confabulation are what the AIs are doing (and some of the humans, too).
"Making shit up in order to fulfill some requirement" is the definition of lying, so whether it's a human or an AI, just making shit up in order to generate prompted output is flat out lying. Not "hallucinating". And the best part is that until LLM get valitidy checks baked in, even the things they get right are lies if presented with authority, because the LLM doesn't know whether it's true or not. In fact, the LLM doesn't know, full stop. It's still just a very well crafted autocomplete, and literally nothing more. So if we're going to anthropomorphise, call them what they'd be when humans do the same:
lies, and damned lies.
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#23Earlier quoted context omitted.
In psychology, we've got a term which is almost 100% matching: confabulation. The only part which isn't correct is association with brain damage. https://en.wikipedia.org/wiki/Confabulation In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. It is generally associated with certain types of brain damage (especially an…
I think it's a good term for this but it also side steps the issue that this isn't a actually intelligence coming up with it. It's just machine noise
It doesn’t matter if it’s noise, it only needs to be useful. Confabulations are not only not useful, they’re actively harmful.
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#24Earlier quoted context omitted.
To be fair, if we are going to anthropomorphize it, bullshit and lies implies some sort of negative intent that I’m not sure the models have. Bullshit is probably the closest, as people will bullshit for all sorts of reasons, but hallucinations is at least intent-neutral, which I think is the point.
A person can 100% believe in the lies they've been told, but that person is not hallucinating. Take for example climate change deniers; apart from the corporations and the politicians that abuse scepticism to maintain their power and wealth, many of the most fervent deniers truly believe the nonsense they're saying. Perhaps a more neutral term like "falsehoods" is applicable here.
I think calling it lying requires intent, I’d just say those people are wrong rather than lying
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#25Opinion pieces like shopping recommendations are quite hard for current LLMs. Either it is a hard fact - or pure creative work - that's where AI shines. Anything between and things get tricky
This is one of those areas where the poor quality of the data influences the output, I think.
There are so many garbage, lazily written product reviews, by websites that only exist to get people to click affiliate links. These sites only have one goal, which is to get you to click an affiliate link and make a purchase. So it is not in their best interest to say "You shouldn't buy this."
Rather, they make a list of "top X Foobars", they start with a really expensive one, then they follow with a more reasonably-priced one, and give it a very positive review. It leads to clicks and purchases.
Given this, it's not surprising to me that even the best LLMs carry pieces of this with them. Ask it to predict text describing some tech product on a sales page, and of course parts of that low-quality data will bleed through.
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#26Is it just me or does everyone trust AI opinions less and less ? Every time I ask it to find top 5 of something, I go and double check myself and almost always find it to be wrong. For example try searching for top 5 restaurants around me in bard. Some of them dont even exist lol and some are just random if you cross verify with actual popularity from yelp etc.
Using language models for location or time based things is not recommended, as this usually requires non-textual data. Better to use them for general knowledge questions, programming help, translation, or writing. Asking them to do any complex calculations (especially when they also require non-text raw data, like inflation in a given time period) is also futile.
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#27Is it just me or does everyone trust AI opinions less and less ? Every time I ask it to find top 5 of something, I go and double check myself and almost always find it to be wrong. For example try searching for top 5 restaurants around me in bard. Some of them dont even exist lol and some are just random if you cross verify with actual popularity from yelp etc.
Well it doesn’t surprise me since I have been saying this for a while that these LLMs hallucinate nonsense to the point where you end up triple checking whatever it outputs.
LLMs thrive in applications that involve creativity and non-serious applications mostly around fantasy or creative writing. Anyone using them seriously outside of summarization for high risk use cases is going to be very disappointed.
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#28A hallucination is an unexpected emergence. The 'making up' facts, because it cannot determine a fact from fiction, is entirely expected behavior. There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes. The bullshit comes from there being no model of trust these AIs subscribe to. I'd love-love-love to see these AI producers be held to some responsibility…
> There is no 'hallucination' as the behavior is anticipated, expected, and entirely within normal operations processes.
Exactly. These are models that predict text sequences. These sequences often semantically express falsehoods, but the model's not "lying", it's not "hallucinating", and it's definitely not malfunctioning. It's doing exactly what it was designed to do.
There definitely are "lies" and "hallucinations" here though ... but they're coming from the hype-cycle-hucksters trying to convince us that this whole process somehow resembles "intelligence".
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#29Stop calling them hallucinations. If we're going to anthropomorphize AIs, let's just call it bullshitting and lies. If we're not going to anthropomorphize AIs, then we need a different term
Given the euphemism "bug" substituting for "programming error" you'd be tempted to allow something similar for LLMs, but these are not errors, the output is by design. There is no motive for truth, just the most likely output, even if the likeliness is low.
> There is no motive for truth
This also ignores the larger question that has been a known issue for at least 2,000 years: "Quid est veritas?"
Re: Microsoft's AI shopping announcement contains hallucinations in the demo
#30Earlier quoted context omitted.
Bullshitting and lies is what the humans selling the AI-powered services are doing. Hallucination, delusion and confabulation are what the AIs are doing (and some of the humans, too).
"Making shit up in order to fulfill some requirement" is the definition of lying, so whether it's a human or an AI, just making shit up in order to generate prompted output is flat out lying. Not "hallucinating". And the best part is that until LLM get valitidy checks baked in, even the things they get right are lies if presented with authority, because the LLM doesn't know whether it's true or not. In fact, the LLM…
> "Making shit up in order to fulfill some requirement" is the definition of lying
I'd argue that there is an element of intent or agency involved. When a human makes things up intentionally or by choice, that is lying. When they do it unintentionally, that is not lying. It is usually called confabulation (or, honest lying - where the actor does not know they are not telling the truth). I don't think AIs/LLMs have agency or the ability to make things up intentionally. They are just doing what they are programmed to do and everything they produce looks the same to them. It is all true as far as the LLM is concerned. They might be confabulating, but I don't think they are lying.