Live data from Hacker News

Bad Actors Are Grooming LLMs to Produce Falsehoods

americansunlight.substack.com

51–60 of 302 posts

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#51
post #12

> But here’s the thing, current models “know” that Pravda is a disinformation ring, and they “know” what LLM grooming is (see below) but can’t put two and two together. Of course they can't, no surprises here. That's just not how LLMs work.

I agree that it's a much bigger problem for LLMs, but to be fair it's also not how humans work. A long lasting, high volume stream of propaganda will have considerable effect on a human even if he is aware that it is false.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#52
post #9

If actions by these bad actors accelerate the rate at which people lose trust in these systems and lead to the AI bubble popping faster then they have my full support. The entire space is just bad actors complaining about other bad actors while they're collectively ruining the web for everyone, each in their own way.

Thats naive. Look at all the tabloids thriving. The kind of people that bad actors target will continue to believe everything it says. They won't lose trust, or magazines like New York Post, the Sun or BILD would already have crossed to exist with their lies and deception. And Russia would not have so many cult members believing the lies they spread.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#53
post #48
post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

This is in fact the goal of Russian style propaganda. You have successfully been targeted. The idea is to spread so much confusion that you just throw up your hands and say, I'm not going to try and figure out what's going on any more. That saps your will to be political, to morally judge actions and support efforts to punish wrongdoers. https://www.rand.org/pubs/perspectives/PE198.html https://en.wikipedia.org/wiki/…

[flagged]

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#54
post #18

Earlier quoted context omitted.

[flagged]

> There are plenty of cases (like in Ukraine, or vaccines, or climate change) where there is unquestionable truth on one side The problem is that most people are like you, and live in psycho-informational ecosystems in which there are "unquestionable truths" -- it is in these very states of comfortable-certainty that we are often most subject to propaganda. All of the issues you mention are identity markers for being…

Thank you for proving my point.

On one hand, we have the unquestionable and undeniable facts that Russia invaded Ukraine and is committing atrocities against its civilian population, up to and including literal genocide (kidnapping children).

On the other, we have:

> Scientists who are aware of the full history of ukraine, western interventionism, russian geostrategic concerns, the full details of the 2013 collapse of the ukrainian govenrment, the terms underwhich russian naval bases in crimea had been leased, the original colour revolution, the role of US diplomats in the overthrow of democratically elected Ukrainian leadership -- etc.

Trying to muddy the waters with at best exaggerations, at worst flat out lies, trying to sow doubt with things which, if true (and usually they aren't) are relevant only to help contextualise the events. But don't in any way change the core facts of the Russian invasion and subsequent war crimes. How does American diplomats supporting a popular protest against the current government which led to that government fleeing (and three elections have happened since, btw), in any way change or minimise the war crimes? It doesn't, you're just muddying the waters. "Oh Russia is justified in kidnapping children and bombing civilians because diplomats did support a popular protest that led to the Russian puppet running away to Russia, 10 years ago, even though multiple elections since have confirmed the people of Ukraine are not for Russian puppets anymore".

You're just repeating Russian propaganda talking points. And we've known since the 80s that they operate in a "firehose" manner, drowning everyone in nonsense to sow doubt. How many different excuses have they provided for their "special military operation" now? Which one is it, is Ukraine ruled by Nazis or are Ukrainians just confused Russians or did America coup Ukraine to install a guy who was elected on a platform of peace with Russia? And how does it in any way explain the war crimes? It's like the downing of MH17, they drowned everyone in multiple conspiracy theories to make it seem there is some doubt in the official, proven, story.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#55
post #48
post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

This is in fact the goal of Russian style propaganda. You have successfully been targeted. The idea is to spread so much confusion that you just throw up your hands and say, I'm not going to try and figure out what's going on any more. That saps your will to be political, to morally judge actions and support efforts to punish wrongdoers. https://www.rand.org/pubs/perspectives/PE198.html https://en.wikipedia.org/wiki/…

What you're saying is certainly an established propaganda strategy of Russia (and others), but what parent is saying is also true, "truth" isn't always black and white, and what is the desired behavior in one country can be the opposite in another.

For example, it is the truth that the Golf of Mexico is called the Gulf of America in the US, but Golf of Mexico everywhere else. What is the "correct" truth? Well, there is none, both of truthful, but from different perspectives.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#56
The problem as I see it is that LLMs behave like bratty teenagers, believing any old rubbish they are told or read. However, their voice is that of a friendly and well meaning adult. If their voice was more in line with their 'age' then I think we'd treat their suggestions with the correct degree of scepticism.

Anyhow, overall this is an unsurprising result. I read it as 'LLMs trained on contents of internet regurgitate contents of internet'. Now that i'm thinking about it, i'd quite like to have an LLM trained on Pliny's encyclopedia, which would give a really interesting take on lots of questions. Anyone got a spare million dollars of compute time?

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#58
post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

There is no objective truth because humans are inherently ideological beings and what we consider objective is just a reflection of our ideology.

Consider markets - a capitalist's "objective truth" might be that they are the most efficient mechanism of allocating resources, a marxists "objective truth" might be that they are a mechanism for exploiting the working class and making the capitalist class even richer.

Here's Zizek, famous ideology expert, describing this mechanism via film analysis: https://www.youtube.com/watch?v=TVwKjGbz60k

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#59
I wonder if the next iteration of advertisements will be people paying to to semantically intertwine their brand to the desired product. This could be done in a very innocuous way by maybe just co-locating the words without any specific endorsement. Or maybe even finding more innocuous ways to semantically connect brand to product. Perhaps the next iteration of the web/advertising will be mass LLM grooming.

Here's a fun example: suppose I'm a developer with a popular software project. Maybe I can get a decent sum of money to put brand placement in my unit-tests or examples.

If such a future plays out, will LLMs find themselves in the same place that search engines in 2025 are?

Post reply on HN