Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google scrambles to manually remove weird AI answers in search

theverge.com

Post reply on HN
Print view View original

381–387 of 387 posts

Re: Google scrambles to manually remove weird AI answers in search

#381

Post by anonymousab »

nomel wrote: post #244
leptons wrote: post #203

Earlier quoted context omitted.

The right answer is no rocks. Some mentally ill person could type that in and get "eat 1000 rocks" and then die from eating rocks, and that would be Google's fault. It's not funny. I have no doubt right now there are at least 50 youtube videos being made testing different glue's effectiveness holding cheese on a pizza. And some of those idiots are going to taste-test it, too. And then people will try it at home, some…

Google is not responsible, and should never be responsible, for protecting mentally ill people from themselves. It would be at a severe detriment to the rest of us if they took on that responsibility. Society should set the bar to “a reasonable person”, otherwise you’re doomed, with no possible alternative to a nanny state.

They absolutely should bear responsibility for authoritatively telling people wrong and unsafe cooking temperatures for meats or spreading lies about people on their results page, in their own voice. "It's just a random text generator!" isn't a protection against, say, gross libel.

They pass the threshold of criminal negligence when they keep up a system that they know will actively mislead countless people in subtly dangerous ways. The problem being hard or the tech being fundamentally unsound doesn't wave away their culpability - if anything it destroys any reason why they should be given some leeway.

Re: Google scrambles to manually remove weird AI answers in search

#382

Post by seadan83 »

seadan83 wrote: post #85
kredd wrote: post #72

Earlier quoted context omitted.

It’s a bit weird since Google is taking over the “burden of proof”-like liability. Up until now, once user clicked on a search result, they mentally judged the website’s credibility, not Google’s. Now every user will judge whether data coming from Google is reliable or not, which is a big risk to take on, in my opinion.

That latter point might be illuminating for a number of additional ideas. Specifically, should people have questioned Google's credibility from the start? Ie: these are the search results, vs this is what google chose. Google did well in the old days for reasons. It beat alta vista and Yahoo by having better search results and a clean loading page. Since perhaps 08 (based on memory, that date might be off) or so, Goo…

My point was not well stated. My hope is that if people learn to question AI summaries, they might also learn to question the rest of the search page.

Re: Google scrambles to manually remove weird AI answers in search

#383

Post by fragmede »

noncoml wrote: post #373
fragmede wrote: post #369

Earlier quoted context omitted.

Why would pigs have wings and fly to the moon be in the training data as a data source marked as serious? We can no true scotsman both sides here. No true human, free of mental/cognitive disorder would be that wrong, but neither would an LLM, with properly annotated training data would be that wrong either.

Did you miss the part that “put glue in pizza to make the cheese stick” was in the training set?

Did you miss the part where I said properly annotated training data?

Re: Google scrambles to manually remove weird AI answers in search

#384

Post by noncoml »

fragmede wrote: post #383
noncoml wrote: post #373

Earlier quoted context omitted.

Did you miss the part that “put glue in pizza to make the cheese stick” was in the training set?

Did you miss the part where I said properly annotated training data?

“Properly annotated data” has nothing to do with the original context.

We were discussing about the current state of affairs. Of course I am not stupid to think what I said in my original reply if we are taking about an LLM trained on “perfect data”

But that was not the premise.

Re: Google scrambles to manually remove weird AI answers in search

#385

Post by fragmede »

noncoml wrote: post #384
fragmede wrote: post #383

Earlier quoted context omitted.

Did you miss the part where I said properly annotated training data?

“Properly annotated data” has nothing to do with the original context. We were discussing about the current state of affairs. Of course I am not stupid to think what I said in my original reply if we are taking about an LLM trained on “perfect data” But that was not the premise.

Your claim was that "LLMs will claim that it’s normal for pigs to have wings and fly to the moon" and that humans free of mental/cognitive disorder would not. Which is to say, humans with a mental/cognitive disorder might claim that it’s normal for pigs to have wings and fly to the moon. If we're carving out such a section for humans to be so wrong, then we should also carve out a section for LLMs to be so wrong.

Fwiw, ChatGPT-4o can write a lengthy essay as to how pigs don't have wings and couldn't fly to the moon even if they did, but if we're more interested in them being nothing more than just a statistical model and that those mere statistics can't possibly result in something that looks like reasoning then we've got to disregard the fact that it "knows" that pigs don't have wings.

Of course pigs having wings is a stand in for whatever else wrong thing that LLMs might "believe", so I agree it's very important for everyone that uses an LLM to understand their limitations especially around hallucinations, but where there are books written about how flat the Earth is and are in the training data, the current state of affairs is that ChatGPT and Gemini both know it's not flat. That Google search AI results, which is a different model, is telling users to use glue on pizza, or to drink urine only serves to say that Google Search's bot using Reddit as unannotated training data is as representative of LLMs as a human with a mental/cognitive disorder.

Re: Google scrambles to manually remove weird AI answers in search

#386

Post by charleslmunger »

fragmede wrote: post #325
charleslmunger wrote: post #324

Earlier quoted context omitted.

A similar story: https://issues.chromium.org/issues/41088357#comment32 Years ago I bought some Korean phone with a foot long antenna and TV tuner on eBay because it was crashing at a disproportionate rate. It was just the nature of Android development at the time.

That was an amazing read, how'd you come across it?

At the time I was working on a prototype of what would eventually be open sourced as Cronet, which was Chromium's http stack repackaged to be embedded in Android apps, so I monitored chromium bugs in the network stack component that mentioned android.

Cronet is still around, now open source and as far as I know is still the best all-around network stack for Android apps - fast, secure, supports modern protocols.

Re: Google scrambles to manually remove weird AI answers in search

#387

Post by noncoml »

fragmede wrote: post #385
noncoml wrote: post #384

Earlier quoted context omitted.

“Properly annotated data” has nothing to do with the original context. We were discussing about the current state of affairs. Of course I am not stupid to think what I said in my original reply if we are taking about an LLM trained on “perfect data” But that was not the premise.

Your claim was that "LLMs will claim that it’s normal for pigs to have wings and fly to the moon" and that humans free of mental/cognitive disorder would not. Which is to say, humans with a mental/cognitive disorder might claim that it’s normal for pigs to have wings and fly to the moon. If we're carving out such a section for humans to be so wrong, then we should also carve out a section for LLMs to be so wrong. Fwi…

Well the whole conversation started by me saying that I think even when I am wrong I am not “put glue in your pizza” wrong. And by I, I did mean the average human. Which is unannotated data from Reddit.
Post reply on HN