Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google’s AI is being manipulated. The search giant is quietly fighting back

bbc.com

Post reply on HN
Print view View original

141–150 of 235 posts

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#141

Post by sparqlittlestar »

tveita wrote: post #15

Would love to read specific examples of "the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement", but the relevant link in the article currently goes to file:///Users/GermaTW1/BBC%20Dropbox/Thomas%20Germain/A%20Downloads%20and%20Documents/2026/And%20there's%20evidence%20that%20AI%20tools%20are%20being%20manipulated%20o…

I've only ever had `first.last@company` as a username or email address, so this `last[:5]initials#` scheme is bewildering. Must lead to strange looking usernames.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#142

Post by hunterpayne »

danudey wrote: post #127
thewebguyd wrote: post #90

Earlier quoted context omitted.

The problem with the news is who makes the decision on which outlets should be blindly trusted by the LLMs and which shouldn't? It also opens the door to government overreach, say a mandate that says LLMs must use fox news as a source of verified, vetted information. Barring that, we are still relying on the execs at the model companies to pick and choose news outlets, and they have their own biases.

Simplest path to the most generally reliable results: * Trust consensus across publicly-funded news outlets from outside of the US the most * Then consensus across private news agencies from outside of the US (across countries) * Then individual trust from publicly-funded news outlets, then private * Then multinational non-profit advocacy groups based outside of the US * Then public broadcasters in the US * Then loca…

On scientific topics, not a single source you listed is in any way accurate at all. And these are things that can be calculated and known with very high accuracy which aren't matters of opinion and yet these sources still get them wrong the majority of the time. And there are plenty of scientific topics which have major impact on policy. Maybe we need to take certain decisions out of the hands of the scientifically illiterate.

PS The BBC (which would be in your highest level) has had to retract stories so often over the last 3 or 4 years that it became a meme to have them apologize for being wrong because they didn't know some video source came from a ML model.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#143

Post by keeda »

xp84 wrote: post #109
keeda wrote: post #83

Earlier quoted context omitted.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

> How is it different from a person believing whatever they read on the Internet? It's not, directionally. But I think this is kind of bypassing the main point here. With an LLM's natural tendency to pattern-match in this way, it's easy to see that it can be used to launder disinformation. If in the olden days, I'd done a google search for "worst war criminals" and saw these blue links on that SERP: "Putin is the 21s…

Makes sense, but it seems to me that the ability to launder disinformation is more a function of the trust people put in LLMs than any inherent property of their own. As some other comments indicate, this also was and is a problem with Wikipedia. It's possible trust in LLMs will follow the same trajectory as trust in Wikipedia, which seems to have been pretty non-linear (like, we rarely see "Do not cite Wikipedia" anymore.)

I think eventually things would settle on an approach similar to your example of the links: look at multiple sources and arrive at a balanced overview that includes the trust level and biases of each sources. I think the pieces are in place, just need to be put together. E.g. already AI overviews (especially on Amazon product reviews) are essentially of the form "Some say A but others say B" which has the benefit of a) clearly being second-hand information, and b) not sounding so authoritative, letting readers draw their own conclusions.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#144

Post by antran22 »

keeda wrote: post #83
ChuckMcM wrote: post #54

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

People are gullible. LLMs generate tokens based on the previous tokens given to it. The LLM in Google's search box doesn't believe anything it was given; it is a Markov-esque chain that go from "Summarize the next sentences: $SEARCH_RESULTS" to the output.

I agree that there's a problem with searching today. The line between actual meaningful content and spam is blurring, all the meaningful indicators of the olden days to distinguish between good and bad contents are now gone/unreliable (polished proses, author's reputation). The signal/noise ratio is decreasing.

The approach to improving SNR should have been reducing/eliminating noise (flag spam sites, reputation system) and boost signal (also maybe reputation system, whitelist/blacklist). It's a hard problem simply because of entropy — the more content you have on the internet, the more random it will all seems from the top down.

I'm not saying I have the answer to this problem, I'm really just a noob when it comes to data science. I'm just thinking that mixing a bunch of text together and let a statistical model rehash that pile of grub into a professional, vindictive sounding response will *not* help providing users with enough signal to make sense of what they are looking for.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#145

Post by basilikum »

suzzer99 wrote: post #94
keeda wrote: post #83

Earlier quoted context omitted.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highligh…

> How is it different from a person believing whatever they read on the Internet? The problem is LLMs have no capacity for shame. My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that. To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and st…

I don't think shame is a helpful human emotion here in general. It prevents people from reaching out for help and makes many crimes much harder to tackle because the victims do not report it.

Also many victims fall for the exact same scam over and over again; to the point that lists of scam victims are sold and used as leads.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#146

Post by freejazz »

K0balt wrote: post #133
freejazz wrote: post #97

Earlier quoted context omitted.

>I can have my agentic system read a few data sheets, then I explain the project requirements and have it design driver specifications, protocols, interfaces, and state machines. Taking those, develop an implementation plan. Working from that, write the skeleton of the application, then fill it in to create a functional system using a novel combination of hardware. When you put it that way, isn't it crazy you have to…

To be fair, I also have had to explain this same basic workflow to junior devs in the past, so I guess not?

> I also have had to explain this same basic workflow to junior devs

That would not surprise me.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#147

Post by [deleted] »

sparqlittlestar wrote: post #141
tveita wrote: post #15

Would love to read specific examples of "the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement", but the relevant link in the article currently goes to file:///Users/GermaTW1/BBC%20Dropbox/Thomas%20Germain/A%20Downloads%20and%20Documents/2026/And%20there's%20evidence%20that%20AI%20tools%20are%20being%20manipulated%20o…

I've only ever had `first.last@company` as a username or email address, so this `last[:5]initials#` scheme is bewildering. Must lead to strange looking usernames.

[deleted]

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#148

Post by ACCount37 »

K0balt wrote: post #130
ACCount37 wrote: post #84

Earlier quoted context omitted.

It's blindingly obvious what the big bet is. The senior devs are going to come from the next generations of AI systems.

That’s the big bet, for sure… but if it’s reasoning that the supervising devs are injecting, and ai systems can’t reason, I guess it won’t work? Idk, I kinda think they do reason, though not in the way people might think. It’s definitely true that they are statistical next token predictors, and that is intrinsically pattern matching, and reasonable to say not capable of reasoning. But my intuition is that that is not…

If the easiest pathway to high performance next token prediction lies through reasoning, then training for better next token prediction ends up training for reasoning implicitly.

By now, there's every reason to believe that this is what's happening in LLMs.

"Reasoning primitives" are learned in pre-training - and SFT and RL then assemble them into high performance reasoning chains, converting "reasoning as a side effect of next token prediction" to "reasoning as an explicit first class objective".

The end result is quite impressive. By now, it seems like the gap between human reasoning and LLM reasoning isn't "an entirely different thing altogether" - it's "humans still do it better at the very top end of the performance curve - when trained for the task and paying full attention".

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#149

Post by danudey »

hunterpayne wrote: post #142
danudey wrote: post #127

Earlier quoted context omitted.

Simplest path to the most generally reliable results: * Trust consensus across publicly-funded news outlets from outside of the US the most * Then consensus across private news agencies from outside of the US (across countries) * Then individual trust from publicly-funded news outlets, then private * Then multinational non-profit advocacy groups based outside of the US * Then public broadcasters in the US * Then loca…

On scientific topics, not a single source you listed is in any way accurate at all. And these are things that can be calculated and known with very high accuracy which aren't matters of opinion and yet these sources still get them wrong the majority of the time. And there are plenty of scientific topics which have major impact on policy. Maybe we need to take certain decisions out of the hands of the scientifically i…

> On scientific topics, not a single source you listed is in any way accurate at all.

My rebuttal to that is twofold:

First, the discussion is about about news, not science (nor about general LLM behaviour).

Second, and probably more relevant, I explicitly said 'if they have a source, go analyze the source rather than taking their interpretation at face value'. When I wrote that I was thinking specifically about what I assume is your point, which is how often news articles about scientific discoveries or science news can often miss, misunderstand, or exaggerate the point of the original research, sometimes to the point of being as useful to society as celebrity gossip.

> And there are plenty of scientific topics which have major impact on policy. Maybe we need to take certain decision out of the hands of the scientifically illiterate.

I would be in favour of mandating that governments make decisions based on established scientific fact rather than the vibes they wish existed, restricting the decision making to 'how do we react to these facts as a society' and not 'which facts should we imagine are true to justify the policies we want'.

> PS The BBC (which would be in your highest level) has had to retract stories so often over the last 3 or 4 years that it became a meme to have them apologize for being wrong because they didn't know some video source came from a ML model.

Aside from being a good reason to support AI fingerprinting on generated media, this is covered by my existing point:

"consensus across publicly-funded news outlets"

"the idea should be to analyze consensus across multiple sources with different biases and agendas. Don't trust any one story from any one source, but look for multiple stories from multiple sources and synthesize results from that"

If the BBC reports on something because they got duped but they're the only ones who did, then there's a distinct lack of consensus which is my main argument in my post.

Lastly, and this is generally off-topic, but at least the BBC issues retractions (which LLMs could then also consume and use in their results). There's a lot of 'news media' out there that will happily parrot talking points they wish were true, or blindly report what they're told, but have no interest in publishing retractions after they push falsehoods, deliberately or not, to their customers.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#150

Post by suzzer99 »

basilikum wrote: post #145
suzzer99 wrote: post #94

Earlier quoted context omitted.

> How is it different from a person believing whatever they read on the Internet? The problem is LLMs have no capacity for shame. My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that. To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and st…

I don't think shame is a helpful human emotion here in general. It prevents people from reaching out for help and makes many crimes much harder to tackle because the victims do not report it. Also many victims fall for the exact same scam over and over again; to the point that lists of scam victims are sold and used as leads.

If a junior developer makes a dumb mistake that causes a mini-disaster, their brain makes it a priority to never make that same mistake again. They physically feel anxiety the next time they get into a similar situation, which serves as a very effective reminder not to do the same dumb thing.

LLMs make the same mistakes over and over. And even if/when they have the capacity to learn on the fly, they have no capacity to prioritize. It's all just a big haze of tokens.

That's my overall point. Humans have mistakes and then they have MISTAKES. And a whole continuum in between. LLMs just have a mish-mash of training data. I think before LLMs are more than just fancy parrots, we need a find an analogue to pain, shame, joy, fear, and the myriad other emotions that factor into human decision-making.

Post reply on HN