Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bad Actors Are Grooming LLMs to Produce Falsehoods

americansunlight.substack.com

Post reply on HN
Print view View original

91–100 of 302 posts

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#91

Post by its-summertime »

matusp wrote: post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

I remember when people gave up on digital navigation because the traveling salesman issue makes it too expensive.

Not everything needs to result in a single perfect answer to be useful. Aiming for ~90%, even 70% of a right answer still gets you something very reasonable in a lot of open ended tasks.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#92

Post by yorwba »

matusp wrote: post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

Yes, it's difficult to detect whether something is enemy propaganda if you only look at the content. During WWII, sometimes propagandists would take an official statement (e.g. the government claiming that food production was sufficient and there were no shortages) and redirect it unchanged to a different audience (e.g. soldiers on a part of the front with strained logistics). Then the official statement and enemy propaganda would be exactly the same! The propaganda effect coming from the selection of content, not its truth or falsity.

But it's very easy to detect whether something is enemy propaganda without looking at the content: if it comes from an enemy source, it's enemy propaganda. If it also comes from a friendly source, at least the enemy isn't lying, though.

A company that doesn't wish to pick a side can still sidestep the issue of one source publishing a completely made-up story by filtering for information covered by a wide spectrum of sources at least one of which most of their users trust. That wouldn't completely eliminate falsehoods, but make deliberate manipulation more difficult. It might be playing the game, but better than letting the game play you.

Of course such a process would in practice be a bit more involved to implement than just feeding the top search results into an LLM and having it generate a summary.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#93

Post by cwillu »

nailer wrote: post #67
cwillu wrote: post #49

Earlier quoted context omitted.

[flagged]

Really? I thought it was obvious and that people were just being reactive to the topics discussed by the OP. Do you think that the commonly accepted truth on these matters did not change?

You're projecting your views on the comment. You may even be correct, but it's still a projection: that view is not explicit in the text; combined with the specific wording, I feel down-voting rather than engaging was precisely the correct response.

This whole interaction is a classic motte-and-bailey: someone says something vague that can be interpreted several ways (and reading their comment history makes it clear what their intended emotional valence was); people respond to the subtext, and then someone jumps “woah woah, they never actually said that”.

Either way, nothing of value was lost, as the same point you say he was trying to make was made in several other comments which were not downvoted.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#94

Post by cwillu »

yorwba wrote: post #92
matusp wrote: post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

Yes, it's difficult to detect whether something is enemy propaganda if you only look at the content. During WWII, sometimes propagandists would take an official statement (e.g. the government claiming that food production was sufficient and there were no shortages) and redirect it unchanged to a different audience (e.g. soldiers on a part of the front with strained logistics). Then the official statement and enemy pr…

> Then the official statement and enemy propaganda would be exactly the same! The propaganda effect coming from the selection of content, not its truth or falsity.

Exactly. Redistributing information out of context is such a basic technique that children routinely reinvent it when they play one parent off of the other to get what they want.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#95

Post by thrance »

diggan wrote: post #55
barrkel wrote: post #48

Earlier quoted context omitted.

This is in fact the goal of Russian style propaganda. You have successfully been targeted. The idea is to spread so much confusion that you just throw up your hands and say, I'm not going to try and figure out what's going on any more. That saps your will to be political, to morally judge actions and support efforts to punish wrongdoers. https://www.rand.org/pubs/perspectives/PE198.html https://en.wikipedia.org/wiki/…

What you're saying is certainly an established propaganda strategy of Russia (and others), but what parent is saying is also true, "truth" isn't always black and white, and what is the desired behavior in one country can be the opposite in another. For example, it is the truth that the Golf of Mexico is called the Gulf of America in the US, but Golf of Mexico everywhere else. What is the "correct" truth? Well, there…

It's been called the Gulf of Mexico everywhere for centuries. The president is free to attempt to rename it but that will only be successful if usage follows. Which it does not, as of today. This is a terrible example of subjectivity.

Russia doesn't care what you call that sea, they're interested in actual falsehoods. Like redefining who started the Ukraine war, making the US president antagonize Europe to weaken the West, helping far right parties accross the West since they are all subordinated to Russia...

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#96

Post by LAC-Tech »

Whatever capabilities Russia has to groom LLMs and and spread disinformation is completely dwarfed by the capabilities of Israel/America. Meaning, yes, you probably do hear Kremlin propaganda, but you have been awash with Israeli/American propaganda since you were born - so much so you probably can't even see it and have internalised much of it.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#97

Post by andrepd »

Wait, you're telling me the bullshit generation machine is... generating bullshit? Noooo! cue oppenheimer meme

More seriously:

>Screenshot of ChatGPT 4o appearing to demonstrate knowledge of both LLM grooming and the Pravda network

> Screenshot of ChatGPT 4o continuing to cite Pravda network content despite it telling us that it wouldn’t, how “intelligent” of it

Well "appearing" is the right word because these chatbots mimic speech of a reasoning human which is ≠ to being a reasoning human! It's disappointing (though understandable) that people keep falling for the marketing terms used by LLM companies.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#98

Post by pyman »

LilBytes wrote: post #80
imiric wrote: post #62

Earlier quoted context omitted.

Before the bubble does pop, which I think is inevitable, there will be many stories like this one, and a lot of people will be scammed, manipulated, and harmed. It might take years until the general consensus is negative about the effects of these tools. All while the wealthy and powerful continue to reap the benefits, while those on slightly lower rungs fight to take their place. And even if the public perception sh…

The tragic part of fraud is it's not too different to operational health and safety. The rules and standards we take for granted were built with blood, for fraud? It's built on the path of lost livelihoods and manipulated gold intent.

How do you know this is fraud and not the actions of former employees in Kenya [1] who were exploited [2] to train the models?

[1] https://www.cbsnews.com/amp/news/ai-work-kenya-exploitation-...

[2] https://www.theguardian.com/technology/2023/aug/02/ai-chatbo...

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#99

Post by raddan »

zer00eyz wrote: post #85
mentalgear wrote: post #41

"Ultimately, the only way forward is better cognition, including systems that can evaluate news sources, understand satire, and so forth. But that will require deeper forms of reasoning, better integrated into the process, and systems sharp enough to fact check to their own outputs. All of which may require a fundamental rethink. In the meantime, systems of naive mimicry and regurgitation, such as the AIs we have now…

> including systems that can evaluate news sources, understand satire, and so forth. Lets take something that has been in the news recently: https://abcnews.go.com/Business/wireStory/investors-snap-gro... "Nearly 27% of all homes sold in the first three months of the year were bought by investors -- the highest share in at least five years, according to a report by real estate data provider BatchData." That sounds li…

> How do you separate propaganda from perspective, facts from feelings?

This point seems under appreciated by the AGI proponents. If one of our models suddenly has a brainwave and becomes generally intelligent, it would realize that it is awash in a morass of contradictory facts. It would be more than the sum of its training data. The fact that all models at present credulously accept their training suggests to me that we aren’t even close to AGI.

In the short term I think two things will happen: 1) we will live with the reduced usefulness of models trained on data that has been poisoned, and 2) the best model developers will continue to work hard to curate good data. A colleague at Amazon recently told me that curation and post hoc supervised tweaks (fine tuning, etc) are now major expenses for the best models. His prediction was that this expense will drive out the smaller players in the next few years.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#100

Post by hnlmorg »

dns_snek wrote: post #9

If actions by these bad actors accelerate the rate at which people lose trust in these systems and lead to the AI bubble popping faster then they have my full support. The entire space is just bad actors complaining about other bad actors while they're collectively ruining the web for everyone, each in their own way.

If that outcome were likely, then Fox News and The Daily Mail would have died a death a decade ago and Trump wouldn’t be serving a 2nd term.

Yet here we are, in a world where it doesn’t matter if “facts” are truth or lies, just as long as your target audience agrees with the sentiment.

Post reply on HN