Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google’s AI is being manipulated. The search giant is quietly fighting back

bbc.com

Post reply on HN
Print view View original

171–180 of 235 posts

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#171

Post by jerf »

suzzer99 wrote: post #156
jerf wrote: post #139

Earlier quoted context omitted.

From a certain and quite valid point of view, they have no mechanism for feedback at all. Every time you start a conversation you're starting in the same state, modulo the random numbers. At most you have this very, very vague loop in that the conversations for LLM 1.0 will be fed in to the training set for LLM 2.0. Even "shame" would only apply to the current session and disappear in the next one, or eventually be c…

According to ChatGPT, researchers are working on models that remember personal directives across sessions. IE - an actual personal assistant that gets to know you and your proclivities. So it's definitely on their radar. No idea how far along they are.

Unless that's something more than the already-common practice called "memories" that are text files held off to the side, that doesn't change what I meant. You can do all sorts of interesting things within the context window, but there's no feedback beyond that.

Even if an frontier-LLM-sized neural net could do something that would somehow change its net on a pervasive level in response to things that happen to it, nobody could possibly serve that in a cost-effective manner.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#172

Post by ChuckMcM »

ge96 wrote: post #167
ChuckMcM wrote: post #54

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

I did notice I had made videos/reddit post about vintage lenses and I was trying to figure out how old it was. The LLM would say an age eg. "made in 1940s" and reference my post which never mentioned the manufacturer date.

I've seen this happen when the backend image searches a picture, gets a description of what is in the picture, and adds that description to the bag of things it will produce as a summary. The whole 'put some text in the image frame that misleads the AI' lead to some hilarious results (man holding a puppy which has a postit stuck to it saying "Siamese kitten" for example, results saying "this man is holding a Siamese kitten."

That led to some changes but it would be interesting to see if you could still poison results that way.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#173

Post by kristianp »

ChuckMcM wrote: post #54

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

> "AI" is just glorified search

Google's AI overview seems to be using RAG of their search snippets that is summarised by a very fast LLM. I wouldn't call that glorified search.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#174

Post by rhdunn »

danudey wrote: post #149
hunterpayne wrote: post #142

Earlier quoted context omitted.

On scientific topics, not a single source you listed is in any way accurate at all. And these are things that can be calculated and known with very high accuracy which aren't matters of opinion and yet these sources still get them wrong the majority of the time. And there are plenty of scientific topics which have major impact on policy. Maybe we need to take certain decisions out of the hands of the scientifically i…

> On scientific topics, not a single source you listed is in any way accurate at all. My rebuttal to that is twofold: First, the discussion is about about news, not science (nor about general LLM behaviour). Second, and probably more relevant, I explicitly said 'if they have a source, go analyze the source rather than taking their interpretation at face value'. When I wrote that I was thinking specifically about what…

> First, the discussion is about about news, not science (nor about general LLM behaviour).

What if science is the news, such as:

1. advancements in fusion power; or

2. progress/status of the Artemis missions; or

3. new LLM models and/or capabilities (e.g. Project Glasswing).

With things like that you typically have a press announcement/briefing, a research paper/publication, or both. That information is then presented in newspapers/media that may obscure, misrepresent, or overly generalize the original finding/announcement.

There may also be clarifications, retractions, etc. after publication, such as with the initial announcement/publication of the proof to Fermat's Last Theorem that initially had an error that was later corrected.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#175

Post by lelandfe »

realmofthemad wrote: post #70
coffeefirst wrote: post #58

Earlier quoted context omitted.

Right. So that's what one guy can do. When you realize how much astroturf is going into Reddit, most social media platforms, and the efforts to manipulate wikipedia for political gain, this is a very real problem.

It's very hard to tell how much is actually fake though. Are there any good statistics on this?

The nature of effective manipulation sort of precludes the ability to get good stats.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#178

Post by incr_me »

idiotsecant wrote: post #155
basilikum wrote: post #145

Earlier quoted context omitted.

I don't think shame is a helpful human emotion here in general. It prevents people from reaching out for help and makes many crimes much harder to tackle because the victims do not report it. Also many victims fall for the exact same scam over and over again; to the point that lists of scam victims are sold and used as leads.

Shame is a wildly useful human emotion. Shame of letting down the tribal unit formed basically all of civilization. Shame is good.

Some shame is good and other shame is bad. Some guilt/shame is indicative of the development of the self, other guilt/shame is a cause and effect of stunted development of the self. I like Winnicott on this:

> How important it is, therefore, for a baby to have his mother consistently looking after him, looking after him over a period of time, surviving his attacks, and eventually there to be the object of the tender feeling and the guilt feeling and sense of concern for her welfare which come along in the course of time. Her continuing to be a live person in the baby’s life makes it possible for the baby to find that innate sense of guilt which is the only valuable guilt feeling, and which is the main source of the urge to mend and to re-create and to give.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#179

Post by jacobgkau »

sparqlittlestar wrote: post #141
tveita wrote: post #15

Would love to read specific examples of "the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement", but the relevant link in the article currently goes to file:///Users/GermaTW1/BBC%20Dropbox/Thomas%20Germain/A%20Downloads%20and%20Documents/2026/And%20there's%20evidence%20that%20AI%20tools%20are%20being%20manipulated%20o…

I've only ever had `first.last@company` as a username or email address, so this `last[:5]initials#` scheme is bewildering. Must lead to strange looking usernames.

I've had several usernames/emails more similar to the `last[:5]initials#` example at universities and large companies. It's more secure (harder to guess based on the name alone), more private (harder for outsiders to tie back to a person from email alone), and reduces or removes the possibility of duplication (especially important for schools that let alumni keep their emails). It actually surprised me when a school gave me first.last once.

Re: Google’s AI is being manipulated. The search giant is quietly fighting back

#180

Post by jorvi »

suzzer99 wrote: post #150
basilikum wrote: post #145

Earlier quoted context omitted.

I don't think shame is a helpful human emotion here in general. It prevents people from reaching out for help and makes many crimes much harder to tackle because the victims do not report it. Also many victims fall for the exact same scam over and over again; to the point that lists of scam victims are sold and used as leads.

If a junior developer makes a dumb mistake that causes a mini-disaster, their brain makes it a priority to never make that same mistake again. They physically feel anxiety the next time they get into a similar situation, which serves as a very effective reminder not to do the same dumb thing. LLMs make the same mistakes over and over. And even if/when they have the capacity to learn on the fly, they have no capacity…

Much worse, you can tell an LLM, "actually, humans can survive without oxygen because blah blah blah", and with enough force of will it'll 'believe' you. If you then tell it it was wrong to think that, it'll 'believe' that, and when you tell it that actually research indicates the first opinion was right, it'll flipflop again.

Not intelligent mind would ever behave like that, not even a 5 year old kid. Or hell, if you trick a dog a few times it'll get annoyed by your antics and go back to sleep on its pillow. An LLM, you can trick for aeons.

Yet somehow most of the AI industry has deluded itself into thinking that LLMs are on the threshold of general intelligence instead of being nothing but fancy stochastic parrots.

Post reply on HN