Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bad Actors Are Grooming LLMs to Produce Falsehoods

americansunlight.substack.com

Post reply on HN
Print view View original

41–50 of 302 posts

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#41

Post by mentalgear »

"Ultimately, the only way forward is better cognition, including systems that can evaluate news sources, understand satire, and so forth. But that will require deeper forms of reasoning, better integrated into the process, and systems sharp enough to fact check to their own outputs. All of which may require a fundamental rethink.

In the meantime, systems of naive mimicry and regurgitation, such as the AIs we have now, are soiling their own futures (and training databases) every time they unthinkingly repeat propaganda."

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#42

Post by qu4z-2 »

jahsome wrote: post #25
sofixa wrote: post #19

Earlier quoted context omitted.

Because that would only reinforce the already problematic bubbles where people only see what feeds their opinions, often to disastrous results (cf. the various epidemics and deaths due to anti-vaxxers or even worse, downright genocides).

People have done this on their own behalf since the dawn of time, so it's not really clear to me why it's so often framed as an AI issue.

Quantity has a quality all its own.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#43

Post by d4rkn0d3z »

matusp wrote: post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

"different and dynamic truths" = fictions

We can not play the game.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#45

Post by fc417fc802 »

jahsome wrote: post #25
sofixa wrote: post #19

Earlier quoted context omitted.

Because that would only reinforce the already problematic bubbles where people only see what feeds their opinions, often to disastrous results (cf. the various epidemics and deaths due to anti-vaxxers or even worse, downright genocides).

People have done this on their own behalf since the dawn of time, so it's not really clear to me why it's so often framed as an AI issue.

The core underlying issue isn't due to LLMs but they greatly exacerbate it. So does the current form of social media.

People used to live in bubbles, sure, but when that bubble was the entire local community, required human interaction, and radio had yet to be invented the implications were vastly different.

I'm optimistic that carefully crafted algorithms could send things back in the other direction but that isn't how you make money so seemingly no one is making a serious effort.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#46

Post by jachee »

charcircuit wrote: post #16
matusp wrote: post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

There are personalized social media feeds, so why not have personalized LLMs that align with how people want their LLM to act.

[flagged]

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#47

Post by ClumsyPilot »

perihelions wrote: post #38
matusp wrote: post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

There's a more basic problem: it's two very different questions to ask "can the machine reason about the plausibility of things/sources?", and "how does it score on an evaluation on a list of authoritative truths and proven lies?" A machine that thinks critically will perform poorly on the latter, since, if you're able to doubt a bad-actor's falsehood, you're just as capable of doubting an authoritative source (often…

> The system that would score best tested against a list of known-truths and known-lies, isn't the perceptive one that excels at critical thinking: it's the ideological sycophant

This is an excellent point

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#48

Post by barrkel »

matusp wrote: post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

This is in fact the goal of Russian style propaganda. You have successfully been targeted. The idea is to spread so much confusion that you just throw up your hands and say, I'm not going to try and figure out what's going on any more.

That saps your will to be political, to morally judge actions and support efforts to punish wrongdoers.

https://www.rand.org/pubs/perspectives/PE198.html

https://en.wikipedia.org/wiki/Firehose_of_falsehood

https://jordanrussiacenter.org/blog/propaganda-political-apa...

https://www.newyorker.com/news/annals-of-communications/insi...

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#49

Post by cwillu »

nailer wrote: post #23
GardenLetter27 wrote: post #14

Earlier quoted context omitted.

> What we consider enemy propaganda today might be an official statement tomorrow. Remember when worrying about COVID was sinophobia? Or when the lab leak was a far-right conspiracy theory? When masks were deemed unnecessary except for healthcare professionals, but then mandated for everyone?

People seem to be voting this down as it seems like political advocacy, but the point seems to be that the commonly accepted truth does indeed change.

[flagged]
Post reply on HN