Live data from Hacker News

Bad Actors Are Grooming LLMs to Produce Falsehoods

americansunlight.substack.com

61–70 of 302 posts

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#62
post #9

If actions by these bad actors accelerate the rate at which people lose trust in these systems and lead to the AI bubble popping faster then they have my full support. The entire space is just bad actors complaining about other bad actors while they're collectively ruining the web for everyone, each in their own way.

Before the bubble does pop, which I think is inevitable, there will be many stories like this one, and a lot of people will be scammed, manipulated, and harmed. It might take years until the general consensus is negative about the effects of these tools. All while the wealthy and powerful continue to reap the benefits, while those on slightly lower rungs fight to take their place. And even if the public perception shifts, the power might be so concentrated that it could be impossible to dislodge it without violent means.

What a glorious future we've built.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#63
post #48
post #11

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

This is in fact the goal of Russian style propaganda. You have successfully been targeted. The idea is to spread so much confusion that you just throw up your hands and say, I'm not going to try and figure out what's going on any more. That saps your will to be political, to morally judge actions and support efforts to punish wrongdoers. https://www.rand.org/pubs/perspectives/PE198.html https://en.wikipedia.org/wiki/…

[flagged]

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#64
post #59

I wonder if the next iteration of advertisements will be people paying to to semantically intertwine their brand to the desired product. This could be done in a very innocuous way by maybe just co-locating the words without any specific endorsement. Or maybe even finding more innocuous ways to semantically connect brand to product. Perhaps the next iteration of the web/advertising will be mass LLM grooming. Here's a…

[dead]

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#65
AI summaries are information deodorant. When you stumble on a misinformation site via Google, usually there are some signals you can smell. Like how they word their titles or how frequently they post similar topics. The 'style' alone implies the quality of the 'substance'. But if you read the same substance summarized by LLMs you can't smell shit.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#66
post #55
post #48

Earlier quoted context omitted.

This is in fact the goal of Russian style propaganda. You have successfully been targeted. The idea is to spread so much confusion that you just throw up your hands and say, I'm not going to try and figure out what's going on any more. That saps your will to be political, to morally judge actions and support efforts to punish wrongdoers. https://www.rand.org/pubs/perspectives/PE198.html https://en.wikipedia.org/wiki/…

What you're saying is certainly an established propaganda strategy of Russia (and others), but what parent is saying is also true, "truth" isn't always black and white, and what is the desired behavior in one country can be the opposite in another. For example, it is the truth that the Golf of Mexico is called the Gulf of America in the US, but Golf of Mexico everywhere else. What is the "correct" truth? Well, there…

The correct truth is to go to a higher level of abstraction and explain that there's a naming controversy.

I get the general point, but I disagree that you have to choose between one of the possibilities instead of explaining what the current state of belief is. This won't eliminate grey areas but it'll sure get us closer than picking a side at random.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#67
post #49
post #23

Earlier quoted context omitted.

People seem to be voting this down as it seems like political advocacy, but the point seems to be that the commonly accepted truth does indeed change.

[flagged]

Really? I thought it was obvious and that people were just being reactive to the topics discussed by the OP.

Do you think that the commonly accepted truth on these matters did not change?

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#69
post #66
post #55

Earlier quoted context omitted.

What you're saying is certainly an established propaganda strategy of Russia (and others), but what parent is saying is also true, "truth" isn't always black and white, and what is the desired behavior in one country can be the opposite in another. For example, it is the truth that the Golf of Mexico is called the Gulf of America in the US, but Golf of Mexico everywhere else. What is the "correct" truth? Well, there…

The correct truth is to go to a higher level of abstraction and explain that there's a naming controversy. I get the general point, but I disagree that you have to choose between one of the possibilities instead of explaining what the current state of belief is. This won't eliminate grey areas but it'll sure get us closer than picking a side at random.

What about straight up ideological disagreements?

Are markets a driver of wealth and innovation or of exploitation and misery?

Is abortion an important human right or murder?

Etc etc

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#70

What is propaganda for one is truth for another, how could LLM tell the difference ? LLM are not journalist fact checking stuff, they are merely programs that regurgitate what it reads. The only way to counter that would be to feed your LLM only on « safe » vetoed source but of course it would limit your LLM capacities so it’s not really going to happen.

> What is propaganda for one is truth for another, how could LLM tell the difference?

"How do you discern truth from falsehood" is not a new question, and there are centuries of literature on the answer. Epistemology didn't suddenly stop existing because we have Data(TM) and Machine Learning(TM), because the use of data depends fundamentally on modeling assumptions. I don't mean that in a hard-postmodernist "but can you ever really know anything bro" sense, I mean it in a "out-of-model error is a practical problem" way.

And yeah, sometimes you should just say "nope, this source is doing more harm than good". Most reasonable people do this already - or do you find yourself seriously considering the arguments of every "the end is nigh" sign holder you come across?

Post reply on HN