Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google’s AI is being manipulated. The search giant is quietly fighting back

bbc.com

Post reply on HN
Print view View original

51–60 of 235 posts

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#51

Post by ulrashida »

simmerup wrote: post #9
63 wrote: post #2

Seems like a lot of entities are "quietly" doing things these days. The llm-ification of every piece of text on the internet is driving me crazy

I hate it. I was on a history subreddit yesterday, reading a submission that was an AI generated history piece —- but seemed to be sourced entirely from a fictional hollywood movie I only knew that because i saw the movie, but it’s a clear sign that the internet is going to shit for quality information

I wonder if this will mean a resurgence of encyclopedias or other authoritative digital records that are known to be verified.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#53

Post by svachalek »

dmortin wrote: post #5

There should be some warning if some "fact" is only supported by one or very few obscure sources. The strength of the sources should be clearly indicated in the answers to help users gauge how trustworthy the info is.

We need a 2026 version of PageRank, some fully game-theory-maxed transitive trust model. And we need it a few years ago already.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#54

Post by ChuckMcM »

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#55

Post by ajross »

JKCalhoun wrote: post #31
ajross wrote: post #18

Earlier quoted context omitted.

> Training on the whole internet feels like citing the National Enquirer It's not, though, because the refutations are in the training data too. This isn't actually the problem being described. The weights in the LLM are fine. It's that the task the LLM is being asked to do is to search and summarize new content that isn't in its training data. And it does it too much like a naive reader and not enough like a cynical…

"…the LLM is being asked to do is to search and summarize new content that isn't in its training data…" If it fails at that then it is a pretty significant problem. As you say earlier "the refutations are in the training data too", then the LLM should in fact be able to use "both sides" and land with a little better confidence when presented with new data. (Hopefully your point regarding prompting issues is resolved…

Well, yeah, "should be" and "does" are different and this is new technology and has bugs and misfeatures and different limitations than what came before, and the market will have a learning curve as we all adapt.

I was just refuting your contention that this is somehow inherent in the idea of "training", and it's not.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#56

Post by galaSerge »

dijksterhuis wrote: post #6

> I was able to demonstrate the problem by publishing a single article on my personal website about my hot-dog-eating prowess. One blog post ... that's all it takes. i'm actually surprised it's that bad . i would have thought it'd take more effort, but i guess it could depend on some sort of purposeful weighting based on search rank during training? > If a company or website is caught breaking the rules, it could be…

[flagged]

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#57

Post by simmerup »

ulrashida wrote: post #51
simmerup wrote: post #9

Earlier quoted context omitted.

I hate it. I was on a history subreddit yesterday, reading a submission that was an AI generated history piece —- but seemed to be sourced entirely from a fictional hollywood movie I only knew that because i saw the movie, but it’s a clear sign that the internet is going to shit for quality information

I wonder if this will mean a resurgence of encyclopedias or other authoritative digital records that are known to be verified.

Well, I suspect the non-LLM ones will become much more expensive than they are now due to the specialist knowledge they’d require to make combined with the smaller pool of people willing to pay for the difference

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#58

Post by coffeefirst »

WarmWash wrote: post #25

My worry dropped significantly when I saw that the result they manipulated was a query for: >2026 South Dakota International Hot Dog Eating Champion If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for. But it looks more like they are doing the equivalent of creating a made u…

Right. So that's what one guy can do.

When you realize how much astroturf is going into Reddit, most social media platforms, and the efforts to manipulate wikipedia for political gain, this is a very real problem.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#59

Post by skywhopper »

WarmWash wrote: post #25

My worry dropped significantly when I saw that the result they manipulated was a query for: >2026 South Dakota International Hot Dog Eating Champion If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for. But it looks more like they are doing the equivalent of creating a made u…

It was a proof of concept and one intended to cause as little collateral damage as possible. But if Google's AI can't tell the difference between a little joke and something real (and of course, it can't, and never will be able to do so), that's a weakness that can be exploited both on a bigger scale and more subtly.

If you don't think bad actors are already attempting this sort of thing (and have been, ever moreso the past four years, including with the help of the very LLM tools they are trying to subvert!) and learning how to manipulate these systems, you are being naive.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#60

Post by [deleted] »

WarmWash wrote: post #28
moparts wrote: post #26

Earlier quoted context omitted.

The article also said this: “ But our investigation also found the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement.” That’s a lot more alarming than just hotdogs.

They should provide the queries then, because it's likely the same trick people have used for decades now with SEO'ing blog posts to appear as "3rd party review" for their shitty products. I create a supplement called Xanatewthiuy, I write blogs/make websites that appear totally unaffiliated saying positive things about "Xanatewthiuy", and then when people see my ads and search for "Xanatewthiuy", the only results ar…

[deleted]
Post reply on HN