Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google’s AI is being manipulated. The search giant is quietly fighting back

bbc.com

Post reply on HN
Print view View original

111–120 of 235 posts

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#111

Post by keeda »

suzzer99 wrote: post #94
keeda wrote: post #83

Earlier quoted context omitted.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

> How is it different from a person believing whatever they read on the Internet? The problem is LLMs have no capacity for shame. My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that. To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and st…

> The problem is LLMs have no capacity for shame.

I know what you mean but I can't help but be cheeky: https://www.fastcompany.com/91383271/googles-chatbot-apologi...

Jokes aside, shame does not change the underlying point though. Despite feeling ashamed for being tricked, as you point out people can still get scammed again by different tricks. I think your point is more about learning from mistakes than shame.

Which still does not change the underlying point, I suppose. Offhand I cannot think of anything that would fix this problem for LLMs that wouldn't also fix it for humans, like relying on trusted sources.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#113

Post by suburban_strike »

foxglacier wrote: post #93
dhosek wrote: post #67

Earlier quoted context omitted.

Every day I find myself thinking more and more that capitalism ruined the internet. The Green Card Lottery usenet spam was the clear indication of where things were going and now everything is Green Card Lottery spam.

That's the same attitude as "cheap airfares caused too many tourists which ruined my favorite tourist destination". You're unhappy that more people have access to it and wish it was still exclusive to the small group you conveniently belong to. Capitalism is what made the internet available to the general public.

> You're unhappy that more people have access to it and wish it was still exclusive to the small group you conveniently belong to.

This is not an argument made in good faith. It's a strawman you've stuffed with suggestive language to make them look petty and intolerant.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#114

Post by keeda »

thewebguyd wrote: post #90
keeda wrote: post #83

Earlier quoted context omitted.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

The problem with the news is who makes the decision on which outlets should be blindly trusted by the LLMs and which shouldn't? It also opens the door to government overreach, say a mandate that says LLMs must use fox news as a source of verified, vetted information. Barring that, we are still relying on the execs at the model companies to pick and choose news outlets, and they have their own biases.

I totally agree, centralization is dangerous, ideally we want any output to be corroborated by multiple, independent sources of truth. But given that the alternative is the absolutely unregulated, unaccountable, wild west of arbitrary content posted on the Internet, I cannot see a solution besides some sort of centralization of trust.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#115

Post by xp84 »

WarmWash wrote: post #25

My worry dropped significantly when I saw that the result they manipulated was a query for: >2026 South Dakota International Hot Dog Eating Champion If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for. But it looks more like they are doing the equivalent of creating a made u…

Okay, but it's easy to make up a novel specific claim no one has written about before, then to make that claim and point to the AI as proof you aren't making this up. For example, imagine this blogpost:

---

"San Francisco Mayor Goodway Admits Poisoning Drinking Water with Drugs to Influence Election"

May 20th, 2026

"Mayor Goodway admitted on Tuesday that she and her deputies poisoned drinking water across the City in order to influence the 2025 election. The Chronicle has confirmed that in neighborhoods whose turnout was to be suppressed, that barbiturates were added to the water for a period of three weeks, while in neighborhoods that had polled strongly for Goodway's favored Progressive slate, methamphetamines were used in the days before the election. Residents are advised to buy bottled water and not to bathe in city water for at least three months."

---

Then once you've confirmed it's been picked up, you tell people "Of COURSE they poisoned our drinking water to manipulate the election. Even ChatGPT will tell you! Just ask." Now, my example is intentionally hard to believe, but all you need is some specificity to build your underlying narrative. And you can make 10 blogs to push the same narrative to increase the effectiveness and increase how many "citations" will show up.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#116

Post by nitwit005 »

WarmWash wrote: post #25

My worry dropped significantly when I saw that the result they manipulated was a query for: >2026 South Dakota International Hot Dog Eating Champion If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for. But it looks more like they are doing the equivalent of creating a made u…

If you can do something small with minimal effort, you can do something big with a multi-million dollar marketing budget.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#117

Post by Terr_ »

pimlottc wrote: post #64
jrflo wrote: post #22

This is just the next phase of SEO. Maybe it'll be called AIO? Just like with search, this will be and endless struggle of Google and AI providers rolling out fixes, optimization firms finding exploits, those getting patched again, etc etc. Anything to get eyeballs for marketing.

Engineered Inference Ersatz Intelligence Optimization (EIEIO)

Old McDonald had a click-farm, EIEIO...

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#118

Post by amarant »

suzzer99 wrote: post #94
keeda wrote: post #83

Earlier quoted context omitted.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

> How is it different from a person believing whatever they read on the Internet? The problem is LLMs have no capacity for shame. My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that. To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and st…

>The problem is LLMs have no capacity for shame

You seem to be implying that people do, and I'd like to contest that point gestures wildly at everything

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#119

Post by RC_ITR »

keeda wrote: post #83
ChuckMcM wrote: post #54

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

You and OP are both unnecessarily diminishing what 'glorified search' is.

If you had told me that in 2015, we would have a tool that can iteratively search the world's best and largest unstructured database and synthesize outputs in language (any natural and structured language), I would have said that is basically AGI.

This whole desire for it to 'reason' (autonomously prime its search with a few thousand token) and 'think' (search for the best information within its parameters and synthesize that with its context) is semantic and will feel irrelevant as the technology progresses and we become more used to what these things are actually doing.

I honestly struggle to imagine what AGI will be if not an ever-improving semi-structured database (parametric or otherwise) that we become increasingly good at searching.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#120

Post by wslh »

keeda wrote: post #83
ChuckMcM wrote: post #54

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

> How is it different from a person believing whatever they read on the Internet?

Because the answers, while prompting, are clearly more human and charming than a search engine results list?

Post reply on HN