Live data from Hacker News

Google scrambles to manually remove weird AI answers in search

theverge.com

241–250 of 387 posts

Re: Google scrambles to manually remove weird AI answers in search

#241
post #37

I'm actually shocked that a company that has spent 25 years on finetuning search results for any random question people ask in the searchbox does not have a good, clean, dataset to train an LLM on. Maybe this is the time to get out the old Encyclopedia Britannica CD and use that for training input.

I am also surprised that training data are not much more curated. Encyclopedias, textbooks, reputable journals, newspapers and magazines make sense. But to throw in social media? Reddit? Seems insane.

The fact is that I think that there is not much written word, to actually train a sensible model on. A lot of books don't have OCRed scans, or a digital version. Humans can extrapolate knowledge from a relatively succinct book and some guidance. But I don't know how a model can add the common sense part (that we already have) that books relies on to transmit knowledge and ideas.

Re: Google scrambles to manually remove weird AI answers in search

#242

As I mentioned previously, I've seen Bing's LLM stall for about a minute when asked something iffy but uncommon. I wonder if Bing is outsourcing questionable LLM results to humans. Anyone else seeing this?

It could be that, but it also could be a cascade of non-LLM checks and retries to GPT with additional prompting.

Re: Google scrambles to manually remove weird AI answers in search

#243
post #185

Earlier quoted context omitted.

No. Whether a person should eat a certain number of small rocks each day is not a matter of opinion, it's not a deep philosophical problem and it's not a question whose truth is not resolveable. You should not be eating rocks.

You choose such an edge case question - how about this sort of thing: Which is the best political party? Are the side effects to X medical treatment? I bet there are even cases when eating rocks is ok! PS It has been written about: https://www.atharjaber.com/works/writings/the-art-of-eating-... > Lithophagia is a subset of geophagia and is a habit of eating pebbles or rocks. In the setting of famine and poverty, cons…

Are you really going to start eating rocks just to convince yourself that Google's AI isn't shit and objective truth is not real?

Re: Google scrambles to manually remove weird AI answers in search

#244

Earlier quoted context omitted.

Or to put it another way, I think Google should have a way of saying "yes, we know this result is wrong, but we're leaving it in because it's funny." There is a demand for funny results. Someone asking “how many rocks should I eat” is looking for entertainment, so you might as well give it to them.

The right answer is no rocks. Some mentally ill person could type that in and get "eat 1000 rocks" and then die from eating rocks, and that would be Google's fault. It's not funny. I have no doubt right now there are at least 50 youtube videos being made testing different glue's effectiveness holding cheese on a pizza. And some of those idiots are going to taste-test it, too. And then people will try it at home, some…

Google is not responsible, and should never be responsible, for protecting mentally ill people from themselves. It would be at a severe detriment to the rest of us if they took on that responsibility. Society should set the bar to “a reasonable person”, otherwise you’re doomed, with no possible alternative to a nanny state.

Re: Google scrambles to manually remove weird AI answers in search

#245

Earlier quoted context omitted.

To be clear he is saying that the LLM is not capable of justified true belief, not commenting on people who believe LLM output. I don’t think your comment is relevant here.

That reading of the comment did occur to me, but I think neither dictionaries nor LLMs are capable of belief, and the comment was about the status of beliefs derived from them.

Okay we are speaking past each other, and you are still misunderstanding the subtlety of the comment:

A dictionary or a reputable Wikipedia entry or whatever is ultimately full of human-edited text where, presuming good faith, the text is written according to that human's rational understanding, and humans are capable of justified true belief. This is not the case at all with an LLM; the text is entirely generated by an entity which is not capable of having justified true beliefs in the same way that humans and rats have justified true beliefs. That is why text from an LLM is more suspect than text from a dictionary.

Re: Google scrambles to manually remove weird AI answers in search

#246
post #218

Earlier quoted context omitted.

Nothing of the sort. I'm trying to understand why anyone cares about formal verifiability in this context, since it's not something we rely on when asking humans to answer questions for us. We evaluate any answer we get without such mathematical proofs, and instead simply judge the answer we're given on its fit and usefulness. Anyone who doubts the usefulness of even these nascent LLMs is fooling themselves. The proo…

> since it's not something we rely on when asking humans to answer questions for us Because we interact with computers (which includes LLMs) differently than we do with humans and we hold them to higher standards Ironically, Google played a large part in this, delivering high quality results to us with ease for many years. At one point Google was the standard for finding high quality information

Shrug. Seems like clutching pearls to me. People seem to have an emotional reaction and obsess on the aspects that differentiate human cognition from LLMs. But that is a lot of wasted energy.

To the extent that anyone avoids employing these technologies, they will be at a disadvantage to those who do; because these tools just work. Already. Today.

There isn't even room for debate on that issue. Again, the proof is in the pudding. These systems are already successfully, usefully, and correctly answering millions of questions a day. They have failure modes where they produce substandard or even flat out incorrect answers too. They're far from perfect, but they're still incredible tools, even without waiting for the improvements that are sure to come.

Re: Google scrambles to manually remove weird AI answers in search

#247

Earlier quoted context omitted.

Except that 90% of Reddit isn't garbage. It's really useful. Problem is Google can't tell what is garbage or not. No LLM can.

> Except that 90% of Reddit isn't garbage. It's really useful. Citation needed. I've been a Reddit user since its inception and honestly except for niche hobby subreddits, Reddit is mostly low effort garbage, bots and rehashed content. I'd wager that mainstream subreddits are 99% garbage for training an LLM for anything other than shitposting.

Even in the niche hobby subreddits there can be a really high garbage factor. There's plenty of well meaning posters that are just wrong. They're not trying to mislead or lying they're just unaware they're wrong.

Re: Google scrambles to manually remove weird AI answers in search

#248
post #105

Earlier quoted context omitted.

"Search" isn't Google's product. Google hasn't been a search company for 20 years. "Ads" is Google's product. And the only way they'll go bankrupt is if 1) companies realize that advertising is pointless (I'm not holding my breath), or 2) some other company takes over from Google, which seems unlikely without government intervention (I'm not holding my breath). Google is a shit company, but they'll still be around 20…

Still need visitors to see the ads.

Google runs ads for a significant percentage of the web (or the markets for ads). Even if everyone stopped going to google.com tomorrow they'd still be seeing ads that make Google money. Google the company would still be tracking much of the web's traffic feeding it into their ads platform.

Re: Google scrambles to manually remove weird AI answers in search

#249

Earlier quoted context omitted.

That reading of the comment did occur to me, but I think neither dictionaries nor LLMs are capable of belief, and the comment was about the status of beliefs derived from them.

Okay we are speaking past each other, and you are still misunderstanding the subtlety of the comment: A dictionary or a reputable Wikipedia entry or whatever is ultimately full of human-edited text where, presuming good faith, the text is written according to that human's rational understanding, and humans are capable of justified true belief. This is not the case at all with an LLM; the text is entirely generated by…

I think the parent comment ultimately concerned the reliability of /beliefs derived from text in reference works v text output by LLMs/, and that seems to be what the replies by the commenter concern. If the point is merely that the text output by LLMs does not really reflect belief but the text in a dictionary reflects belief (of the person writing it), it is well-taken. Since it is fairly obvious and I think the original comment really was about the first question, I address the first rather than second question.

The point you make might be regarded as an argument about the first question. In each case, the ‘chain of custody’ (as the parent comment put it) is compared and some condition is proposed. The condition explicitly considered in the first question was reliability; it was suggested that reliability is not enough, because it isn’t justification (which we can understand pretheoretically, ignoring the post-Gettier literature). My point was that we can’t circumvent the post-Gettier literature because at least one seemingly plausible view of justification is just reliability, and so that needs to be rejected Gettier-style (see e.g. BonJour on clairvoyance). The condition one might read into your point here is something like: if in the ‘chain of custody’ some text is generated by something that is incapable of belief, the text at the end of the chain loses some sort of epistemic virtue (for example, beliefs acquired on reading it may not amount to knowledge). Thus,

> text from an LLM is more suspect than text from a dictionary.

I am not sure that this is right. If I have a computer generate a proof of a proposition, I know the proposition thereby proved, even though ‘the text is entirely generated by an entity which is not capable of having justified true beliefs’ (or, arguably, beliefs at all). Or, even more prosaically, if I give a computer a list of capital cities, and then write a simple program to take the name of a country and output e.g. ‘[t]he capital of France is Paris’, the computer generates the text and is incapable of belief, but, in many circumstances, it is plausible to think that one thereby comes to know the fact output.

I don’t think that that is a reductio of the point about LLMs, because the output of LLMs is different from the output of, for example, an algorithm that searches for a formally verified proof, and the mechanisms by which it is generated also are.

Re: Google scrambles to manually remove weird AI answers in search

#250
Perhaps they could run each search result through ChatGPT. It's pretty skilled at spotting bad results. For example, I asked it whether the glue-on-pizza result was "valuable and should be shown to a user" and it returned "No, this response should not be shown to the user. The suggestion to add non-toxic glue to the sauce is inappropriate and potentially harmful."
Post reply on HN