Why do people act like LLMs only hallucinate some of the time?
It’s not hallucinations here, multiple of the ridiculous results can be directly traced to redit posts where people are joking or saying absurd things
So...every Reddit post?
Google scrambles to manually remove weird AI answers in search
291–300 of 387 posts
Re: Google scrambles to manually remove weird AI answers in search
#292Earlier quoted context omitted.
The right answer is no rocks. Some mentally ill person could type that in and get "eat 1000 rocks" and then die from eating rocks, and that would be Google's fault. It's not funny. I have no doubt right now there are at least 50 youtube videos being made testing different glue's effectiveness holding cheese on a pizza. And some of those idiots are going to taste-test it, too. And then people will try it at home, some…
> The right answer is no rocks. Sand is considered a "rock". If you live in e.g. the USA or the EU you've definitely inadvertently eaten rocks from food produce that's regulated and considered perfectly safe to eat. It's impossible to completely eliminate such trace contaminants from produce. Pedantic? Yes, but you also can't expect a machine to confidently give you absolutes is response to questions that don't even…
This is not a serious reply.
Re: Google scrambles to manually remove weird AI answers in search
#293Earlier quoted context omitted.
So what? You have no way to know for sure if the human you ask the same question, does either. The question that started this thread was related to verifiability. And i still think it is a spurious complaint, given that we have exactly the same limitations when dealing with any human agent.
> You have no way to know for sure if the human you ask the same question, does either. The human might lie, but they generally don't. An LLM is always confabulating when it explains how it reached a conclusion, because that information was discarded as soon as it picked a word. The limitations are not in the same ballpark.
The human may also be... wrong. Saying things that "feel right", except this one time they're factually wrong.
A human can explain their reasoning step by step, if the original reasoning was a System 2, formal, step-by-step process in the first place; otherwise, they're just making shit up after the fact, which feels right, but may or may not be correct (see also the previous paragraph).
Note that it's very rare anyone has an interaction with a human that uses this mode of reasoning - it's unnecessary except in special circumstances, usually math-heavy.
Re: Google scrambles to manually remove weird AI answers in search
#294Earlier quoted context omitted.
I don't know about bikewrench but AskHistorians is a useful source of knowledge because it is strongly moderated and curated. It's not just a bunch of random assholes spouting off on topics. Top level replies are unceremoniously removed if they lack sourcing or make unsourced/unsubstantiated claims. Top level posters also try to self-correct by clearly indicating when they're making claims of fact that are disputed o…
In general, I have trouble trusting environments that can be described as "strongly moderated and curated". I find that environments that rely on censorship tend to foster dogma, rather than knowledge and real understanding of the topics at hand. They give an illusion of quality and trustworthiness. It's something we see happen at this site to some extent, for example. I'd rather see ideas and information being freel…
Your comment is orthogonal to the quality of the AskHistorians subreddit. AskHistorians' moderation tends towards curating posts following the rules rather than content. There's often competing narratives on questions where there's academic dispute of facts.
Regardless of whether you think that's the right approach to moderation, top level posts are sourced and can at least be examined. It's a marked improvement over the unsourced musings of random Redditors.
Re: Google scrambles to manually remove weird AI answers in search
#295Earlier quoted context omitted.
The right answer is no rocks. Some mentally ill person could type that in and get "eat 1000 rocks" and then die from eating rocks, and that would be Google's fault. It's not funny. I have no doubt right now there are at least 50 youtube videos being made testing different glue's effectiveness holding cheese on a pizza. And some of those idiots are going to taste-test it, too. And then people will try it at home, some…
Google is not responsible, and should never be responsible, for protecting mentally ill people from themselves. It would be at a severe detriment to the rest of us if they took on that responsibility. Society should set the bar to “a reasonable person”, otherwise you’re doomed, with no possible alternative to a nanny state.
It's not only mentally ill people that are at risk, but anyone that doesn't know it's not a good idea to put "non-toxic" glue in pizza cheese. That includes a lot of not-mentally-ill but just plain dumb people. Google didn't need to tell people that glue+pizza is a reasonable thing to do, or even just a thing. It sure did frame it like it was a legitimate response. And Google didn't even have to reply with this or anything else, they could have just supplied the links to other sites where it had been suggested, but no - they have to make a show of force with their premature foray into AI, and have it tell real people all kinds of false, and possibly dangerous things. That's an unforced error by Google that they could end up being prosecuted for.
Re: Google scrambles to manually remove weird AI answers in search
#296Re: Google scrambles to manually remove weird AI answers in search
#297Earlier quoted context omitted.
This is confidently stated and incorrect.
Do you have anything to add?
Sure. Your claim has some truth, but is far too strong.
The articles you cited upthread do not support the notion that models consistently activate differently when generating true facts vs false facts.
It is true that models can capture some notion of reliability based on patterns in their training data. For a concrete example, it is entirely plausible that a model can capture the sense that data trained from Reddit is less truthy than data trained from Wikipedia, or that training data with poor grammar and vocabulary is less reliable than more sophisticated inputs.
But this process is not a guarantee, and does not change the fact that LLMs have no mechanism to track the provenance of information. It's probably a fruitful direction of research for reducing the probability of emitting false facts, but there will always be an infinite number of marginal cases for which the activations for true facts are indistinguishable from those for false facts.
Models simply do not track the provenance which is required to make this distinction in every case.
Re: Google scrambles to manually remove weird AI answers in search
#298Earlier quoted context omitted.
Upon a second reading, this is an excellent point. For the sake of clarity, let's remove LLMs from the equation and posit the existence of Encyclopedia Eric. Ask Eric any question, and he will happily research it and come back to you with the answer. But he can sometimes be sloppy in his research, and he gives the correct answer only X percent of the time. Furthermore, Encyclopedia Eric steadfastly refuses to cite hi…
I am glad that you think it an excellent point! I think that this might be a nice way of getting round my objection, but there is one worry, which is that X is relative to a distribution on the questions we ask when we aren’t dealing with Encyclopedia Eric but with an LLM. I don’t actually use LLMs very much myself, partly out of arrogance and Luddite tendencies. But I suspect that the value of X for some sorts of qu…
> X is relative to a distribution on the questions we ask when we aren’t dealing with Encyclopedia Eric but with an LLM.
Assuming I understand what you mean here correctly, this should be the case for both LLMs and Encyclopedia Eric - there are topics Eric knows by heart (or thinks they know); there are specific phrases seared into his mind through sheer exposure during his life prior to becoming a living Encyclopedia. There are words he's used to, and exact synonyms he barely recognizes. All that means your chance of getting correct answer to your query depends, in complex and unknown to you way, on how you state it.
Re: Google scrambles to manually remove weird AI answers in search
#299They should start with just removing reddit from the data set.
Now the AI changed from an immature teenager to a blogspam article
Re: Google scrambles to manually remove weird AI answers in search
#300Earlier quoted context omitted.
Upon a second reading, this is an excellent point. For the sake of clarity, let's remove LLMs from the equation and posit the existence of Encyclopedia Eric. Ask Eric any question, and he will happily research it and come back to you with the answer. But he can sometimes be sloppy in his research, and he gives the correct answer only X percent of the time. Furthermore, Encyclopedia Eric steadfastly refuses to cite hi…
I am glad that you think it an excellent point! I think that this might be a nice way of getting round my objection, but there is one worry, which is that X is relative to a distribution on the questions we ask when we aren’t dealing with Encyclopedia Eric but with an LLM. I don’t actually use LLMs very much myself, partly out of arrogance and Luddite tendencies. But I suspect that the value of X for some sorts of qu…
A relevant point to this is the notion of "System-1" vs "System-2" thinking. Somewhat dubious when applied to actual human psychology but I think a valid metaphor for how LLMs work: they are only capable of System-1 thinking; a single forward pass through the weights of intuition
In my actual life, I don't trust my own System-1 thoughts: for anything important, I'm always going to engage System-2. And LLMs don't have a System-2.
(I also agree that when dealing with LLMs the value of X is not a single value but a highly complex space depending on the nature of the question and the training data of the model. In my mind it does not change the epistemological equation, it just means that even the value of X itself is harder to "know", so this ambiguity can only ever make LLMs a less viable source of knowledge.)