Bad Actors Are Grooming LLMs to Produce Falsehoods
americansunlight.substack.com
Bad Actors Are Grooming LLMs to Produce Falsehoods
1–10 of 302 posts
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#2It's agendas all the way down
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#3It's agendas all the way down
What happened to all the turtles?
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#4[deleted]
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#5Seems like the general problem is consistency within the model. To people working in the field : what are the current options explored for solving this problem ?
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#6Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#7I've been using "off-by-one" errors to describe one of my biggest concerns with LLMs replacing search, or acting as research agents, or functionally being expected to be reliable narrators in general. If you ask ChatGPT when George Washington was born, and it comes back with March 4th, 2017, you'll reject that outright and recognize it's hallucinated a garbage response, presuming you have enough context to have understood who George Washington was in the first place and that your brain hasn't completely succumbed to rot yet.
But if it returns February 20th, 1731... that... man, that sounds close? Is that right? It sounds like it _could_ be right... Isn't Presidents' Day essentially based on Washington's birthday? And _that's_ in February, right? So, yeah, February 20th, 1731. That's probably Washington's birthday.
And so the LLM becomes an arbiter of capital-T Truth and we lose our shared understanding of actual, factual data, and actual, factual history. It'll take less than a generation for the slop factories to poison the well, and while the idea is obviously that you train your models on "known good", pre-slop content, and that you weight those "facts" more heavily, a concerted effort to degrade the Truthfulness of various facts could likely be more successful than we anticipate, and more importantly: dramatically more successful than any layperson can easily understand.
We already saw that with the early Bard Google AI proto-Gemini results, where it was recommending glue as a pizza topping, _with authority_. We've been training ourselves to treat responses from computers (and specifically Google) as if they have authority, we've been eroding our own understanding and capabilities around media literacy, journalism, fact-checking, and what constitutes an actual "fact", and we've had a shared understanding that computers can _calculate_ things with accuracy and fidelity and consistency. All of that becomes confounded with an LLM that could reasonably get to a place where it reports that 2+2=5.
The worst part about the nature of this particular pathway to ruin is that the off-by-one nature of these errors are how they'll infiltrate and bury themselves into some system, insidiously, and below the surface, until days or months or years later when the error results in, I don't know, mega-doses of radiation because of a mis-coded rounding error that some agentic AI got wrong when doing a unit conversion and failed to catch it. We were already making those errors as humans, but as our dependence and faith on LLMs to be "mostly right" increases, and our willingness and motivation to check it for errors dwindles, especially when results "look" right, this will go from being a hypothetical issue to being a practical one extremely quickly and painfully, and probably faster than we can possibly defend against it.
Interesting times ahead, I suppose, in the Chinese-curse sense of the word.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#8What is propaganda for one is truth for another, how could LLM tell the difference ?
LLM are not journalist fact checking stuff, they are merely programs that regurgitate what it reads.
The only way to counter that would be to feed your LLM only on « safe » vetoed source but of course it would limit your LLM capacities so it’s not really going to happen.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#9If actions by these bad actors accelerate the rate at which people lose trust in these systems and lead to the AI bubble popping faster then they have my full support. The entire space is just bad actors complaining about other bad actors while they're collectively ruining the web for everyone, each in their own way.
Re: Bad Actors Are Grooming LLMs to Produce Falsehoods
#10Could stop with "grooming LLMs"