Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google’s AI is being manipulated. The search giant is quietly fighting back

bbc.com

Post reply on HN
Print view View original

151–160 of 235 posts

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#151

Post by Yokohiii »

WarmWash wrote: post #25

My worry dropped significantly when I saw that the result they manipulated was a query for: >2026 South Dakota International Hot Dog Eating Champion If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for. But it looks more like they are doing the equivalent of creating a made u…

Well my concern instantly spiked. Recently Gemini started to show a search spinner for every turn. So every response paired with a search could be subject to prompt injection. Probably every response.

This will also become viral like link spam. Every user content site will become a prompt injection host. The problem is that these are way harder to detect then a link.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#152

Post by danudey »

keeda wrote: post #114
thewebguyd wrote: post #90

Earlier quoted context omitted.

The problem with the news is who makes the decision on which outlets should be blindly trusted by the LLMs and which shouldn't? It also opens the door to government overreach, say a mandate that says LLMs must use fox news as a source of verified, vetted information. Barring that, we are still relying on the execs at the model companies to pick and choose news outlets, and they have their own biases.

I totally agree, centralization is dangerous, ideally we want any output to be corroborated by multiple, independent sources of truth. But given that the alternative is the absolutely unregulated, unaccountable, wild west of arbitrary content posted on the Internet, I cannot see a solution besides some sort of centralization of trust.

I would still maintain that the solution would be to have LLMs doing 'research' (by querying news for recent events) to ensure they're checking multiple sources, and to be explicit about which sources there were, whether those sources had sources, and whether their claims were uncorroborated or unsubstantiated.

The problem, IMHO, is that the LLMs are happily regurgitating facts from whoever, wherever, whenever. Even with a centralization of trust, e.g. 'We know La Presse is reputable and can be given the benefit of the doubt', mistakes can still be made. Without the LLMs cross-checking what they learn the output is still entirely unreliable.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#153

Post by dlenski »

ChuckMcM wrote: post #54

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

> "AI" is just glorified search

Even aside from out-and-out spam one of the extremely frustrating things about Google's AI overviews, compared to traditional search, is that the results are presented as coherent verging on authoritative even when they're not.

If you do an "old-fashioned" (udm=14) Google search for, let's say "vendor scsi commands appotech USB NAND flash chip": https://www.google.com/search?q=vendor+scsi+commands+appotec...

… you'll see that there are only a few links, and a lot of them are people who are trying to reverse-engineer the devices' behavior, and uncertain or confused about what they're doing. You get instant feedback that you're looking a dark corner for something that has little public documentation.

If you remove that `&udm=14` and look at the AI overview, Google gives you a confident-looking reply about available tools and techniques, even though some of what it links to are bit-rotted Russian-language forums and file download sites, and other places that likely won't solve your problem in a straightforward way… because that's all that's available for Google to mine.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#154

Post by dhosek »

foxglacier wrote: post #93
dhosek wrote: post #67

Earlier quoted context omitted.

Every day I find myself thinking more and more that capitalism ruined the internet. The Green Card Lottery usenet spam was the clear indication of where things were going and now everything is Green Card Lottery spam.

That's the same attitude as "cheap airfares caused too many tourists which ruined my favorite tourist destination". You're unhappy that more people have access to it and wish it was still exclusive to the small group you conveniently belong to. Capitalism is what made the internet available to the general public.

It’s not the presence of the general public that ruined the internet, it’s the make-a-buck-at-all-costs attitude generated by capitalism that did it.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#155

Post by idiotsecant »

basilikum wrote: post #145
suzzer99 wrote: post #94

Earlier quoted context omitted.

> How is it different from a person believing whatever they read on the Internet? The problem is LLMs have no capacity for shame. My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that. To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and st…

I don't think shame is a helpful human emotion here in general. It prevents people from reaching out for help and makes many crimes much harder to tackle because the victims do not report it. Also many victims fall for the exact same scam over and over again; to the point that lists of scam victims are sold and used as leads.

Shame is a wildly useful human emotion. Shame of letting down the tribal unit formed basically all of civilization. Shame is good.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#156

Post by suzzer99 »

jerf wrote: post #139
ChuckMcM wrote: post #108

Earlier quoted context omitted.

I love that book, that said, the point is more subtle than that. Current LLM attention models are limited in their feedback. Adding a form of 'shame' feedback (result is technically correct but morally bad or some such) would help here but I doubt the folks building theses things would choose to do so.

From a certain and quite valid point of view, they have no mechanism for feedback at all. Every time you start a conversation you're starting in the same state, modulo the random numbers. At most you have this very, very vague loop in that the conversations for LLM 1.0 will be fed in to the training set for LLM 2.0. Even "shame" would only apply to the current session and disappear in the next one, or eventually be c…

According to ChatGPT, researchers are working on models that remember personal directives across sessions. IE - an actual personal assistant that gets to know you and your proclivities. So it's definitely on their radar. No idea how far along they are.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#157

Post by hunterpayne »

danudey wrote: post #149
hunterpayne wrote: post #142

Earlier quoted context omitted.

On scientific topics, not a single source you listed is in any way accurate at all. And these are things that can be calculated and known with very high accuracy which aren't matters of opinion and yet these sources still get them wrong the majority of the time. And there are plenty of scientific topics which have major impact on policy. Maybe we need to take certain decisions out of the hands of the scientifically i…

> On scientific topics, not a single source you listed is in any way accurate at all. My rebuttal to that is twofold: First, the discussion is about about news, not science (nor about general LLM behaviour). Second, and probably more relevant, I explicitly said 'if they have a source, go analyze the source rather than taking their interpretation at face value'. When I wrote that I was thinking specifically about what…

"First, the discussion is about about news, not science (nor about general LLM behaviour)."

That's a false dichotomy. Consider energy policy. What kind of power do you need to add to your grid? What are the risks for each type of power? How much CO2 does each type of power emit, etc? These are scientific questions which directly impact public policy and are consistently misreported by news sources.

So there is no line between these things. It is however an area which where accuracy can be measured. And when we do that, its hard to argue that allowing journalists without technical credentials to continue to have a platform is a good idea.

And I can make the same argument about several other topics including military matters. Literally, the 2 weapons systems the media hates the most have the 2 best track records on the battlefield. They aren't just wrong. They are literally the opposite of correct on many topics.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#158

Post by xp84 »

WarmWash wrote: post #126
xp84 wrote: post #115

Earlier quoted context omitted.

Okay, but it's easy to make up a novel specific claim no one has written about before, then to make that claim and point to the AI as proof you aren't making this up. For example, imagine this blogpost: --- "San Francisco Mayor Goodway Admits Poisoning Drinking Water with Drugs to Influence Election" May 20th, 2026 "Mayor Goodway admitted on Tuesday that she and her deputies poisoned drinking water across the City in…

Yeah, but this has been true of Google for over 20 years now.

People had a better conceptual model of what results on the SERP were: Random websites.

If I ask ChatGPT "Did X do Y" and it responds with bold text "Yes, X did Y on this date, which was reported on the CBS Evening News" but that whole thing was just sourced from one webpage. Even if there are footnotes, people today are treating that with greater weight than some random crackpot having a blog because to them, "ChatGPT is telling me so" not "ChatGPT is listing websites that seem to mention that." Likewise with the garbage information that pops out of the "AI Overview" -- it really looks to the naive user (which is at least 50% of the Internet audience) that Google is telling you a fact. This part especially, I attribute to what AI Overview's real estate on the page was taken from: That spot used to show deterministic facts, like unit conversions, or extracted exact text snippets from a small set of basically reliable sites, like IMDB, or like, whatever a reliable and direct source is for population of a city. People learned that if you type into Google "how many Tbsp in a Cup" it answers you with that fact in bold at the top of the page. So the things presented today are being presented in a place people were primed for a decade to believe was a deterministic fact zone.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#159

Post by xp84 »

keeda wrote: post #143
xp84 wrote: post #109

Earlier quoted context omitted.

> How is it different from a person believing whatever they read on the Internet? It's not, directionally. But I think this is kind of bypassing the main point here. With an LLM's natural tendency to pattern-match in this way, it's easy to see that it can be used to launder disinformation. If in the olden days, I'd done a google search for "worst war criminals" and saw these blue links on that SERP: "Putin is the 21s…

Makes sense, but it seems to me that the ability to launder disinformation is more a function of the trust people put in LLMs than any inherent property of their own. As some other comments indicate, this also was and is a problem with Wikipedia. It's possible trust in LLMs will follow the same trajectory as trust in Wikipedia, which seems to have been pretty non-linear (like, we rarely see "Do not cite Wikipedia" an…

I agree with your assessment or hopes. The interesting thing is that I get the idea the average user basically grokked, in 2008, that Google itself can't answer a question for you, it can only show you a list of websites that match keywords and you have to do the work to vet them, and often to extract the answers themselves from webpages.

Today they seem to not grok (no pun intended, just think the word is fitting) that AI isn't an oracle and as such, its "opinion" on anything that could be even slightly controversial carries zero weight.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#160

Post by hunterpayne »

K0balt wrote: post #130
ACCount37 wrote: post #84

Earlier quoted context omitted.

It's blindingly obvious what the big bet is. The senior devs are going to come from the next generations of AI systems.

That’s the big bet, for sure… but if it’s reasoning that the supervising devs are injecting, and ai systems can’t reason, I guess it won’t work? Idk, I kinda think they do reason, though not in the way people might think. It’s definitely true that they are statistical next token predictors, and that is intrinsically pattern matching, and reasonable to say not capable of reasoning. But my intuition is that that is not…

"The software is the sum total of collective human culture they are trained on."

Almost, they are the median or most popular aspects of the culture upon which they are trained. So you are getting the most popular way to do something, not the best (for some definition of best). That's why the claims about LLMs being geniuses is absurd. They almost by definition are going to have the average IQ of all the people on the net weighted by how much each person posts. I'm guessing that's about 95.

Post reply on HN