It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.
Bad Actors Are Grooming LLMs to Produce Falsehoods
11–20 of 302 posts
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#12> But here’s the thing, current models “know” that Pravda is a disinformation ring, and they “know” what LLM grooming is (see below) but can’t put two and two together.
Of course they can't, no surprises here. That's just not how LLMs work.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#13I've been using "off-by-one" errors to describe one of my biggest concerns with LLMs replacing search, or acting as research agents, or functionally being expected to be reliable narrators in general. If you ask ChatGPT when George Washington was born, and it comes back with March 4th, 2017, you'll reject that outright and recognize it's hallucinated a garbage response, presuming you have enough context to have under…
Maybe, just maybe people will learn they can’t trust everything that’s written online wether it’s done by a bot or even human.
Hell, they might learn that even real life authorities may lies, cheat and not have everyone’s interest in their mind.
Hope for the best, prepare for the worst.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#14It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.
> What we consider enemy propaganda today might be an official statement tomorrow.
Remember when worrying about COVID was sinophobia? Or when the lab leak was a far-right conspiracy theory? When masks were deemed unnecessary except for healthcare professionals, but then mandated for everyone?
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#15LLMs are “taught” two kinds of “truth”. One is 100% adherence to a reference text. If the text says the Coliseum is in Antarctica or 1+1=716, model must too. The other is adherence to reputable outside sources.
Not sure if it’s embarrassing or a fundamental limitation that grooming and misunderstanding satirical articles defeat the models.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#16It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.
There are personalized social media feeds, so why not have personalized LLMs that align with how people want their LLM to act.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#17The biggest problem here is the differentiation between objective and relative truth. As long as relative truth is part of ai we can't fully trust it's output. The relative truth for one individual might be perceived as propaganda by another individual, relative to their surroundings and the narrative that is dominant in their social group. It's problematic that truth is not a neutral object but exactly this when it comes to non logical subjects.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#18What is propaganda for one is truth for another, how could LLM tell the difference ? LLM are not journalist fact checking stuff, they are merely programs that regurgitate what it reads. The only way to counter that would be to feed your LLM only on « safe » vetoed source but of course it would limit your LLM capacities so it’s not really going to happen.
[flagged]
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#19It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.
There are personalized social media feeds, so why not have personalized LLMs that align with how people want their LLM to act.
Because that would only reinforce the already problematic bubbles where people only see what feeds their opinions, often to disastrous results (cf. the various epidemics and deaths due to anti-vaxxers or even worse, downright genocides).
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#20I've been using "off-by-one" errors to describe one of my biggest concerns with LLMs replacing search, or acting as research agents, or functionally being expected to be reliable narrators in general. If you ask ChatGPT when George Washington was born, and it comes back with March 4th, 2017, you'll reject that outright and recognize it's hallucinated a garbage response, presuming you have enough context to have under…
At every point, during a knowledge/data search for reaching a particular goal, the onus is _always_ on the person searching to do their best to ensure that the sources they use are accurate, and they do the effort required to ensure that they translate that properly to fit that goal.
The education system I grew up in was not perfect. Teachers were not experts in their field, but would state factual inaccuracies - as you say LLMs do - with authority. Libraries didn't have good books; the ones they had were too old, or too propaganda-driven, or too basic. The students were not too interested in learning, so they rote-learned, copied answers off each other and focussed on results than the learning process. If I had today's LLMs then, I'd have been a lot better off, and would've been able to learn a lot more (assuming that I went through the effort to go through all the sources the LLM cited).
The older you grow, you know that there is no arbiter of T-Truth; you can make someone/something that for yourself, but times change, "actual, factual history" could get proven incorrect, and you will need to update your knowledge stores and beliefs along with it, all the while being ready to be proved incorrect again. This has always been the case, and will continue to be, even with LLMs.