51–60 of 73 posts
Is this surprising? LLMs are trained to produce likely word/tokens in a dataset. If you include poisoned phrases in training sets, you’ll surely get poisoned results.
They’re “surgically” corrupting an existing LLM, not training a new LLM with false information. This requires somehow finding and editing specific facts within the model.
Earlier quoted context omitted.
> So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing. And the dissenting opinion will be able to do the same. Twelve year old kids will be running swarms of these for fun, and the technology will be so widely proliferated that everyone will encounter it dail…
I don’t disagree, but fools will still be fooled. And there are a lot of fools. I do wonder what it means for the future of the internet. I don’t think net good is coming out of this.
I really wonder what this will do to human culture as a whole, in the long term. So far we have relied on cultural artefacts and practices being mostly the work of other humans (directly or through tools). We are about to find out what happens when that is no longer the case.
Earlier quoted context omitted.
I think it’s a bigger problem than fake news. Sure, LLMs can generate that, but what they can do much better than prior disinformation automation is have tailored, context-aware conversations. So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing. It’s complete…
> Russian disinformation tactics but massively scaled up. And those were already wildly effective. Russian disinformation's success in the 2016 election is massively over hyped for the usual partisan sour grapes reasons. You cannot move the world with six figures of Facebook ads, if you could, everyone would spend a lot more money on Facebook ads.
Of course, nations still have a right to sovereignty and to be upset when another nation interferes in their internal affairs. I really hope the American public remembers how it felt going forward.
Earlier quoted context omitted.
I think it’s a bigger problem than fake news. Sure, LLMs can generate that, but what they can do much better than prior disinformation automation is have tailored, context-aware conversations. So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing. It’s complete…
> So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing. And the dissenting opinion will be able to do the same. Twelve year old kids will be running swarms of these for fun, and the technology will be so widely proliferated that everyone will encounter it dail…
if they have the money
> Twelve year old kids
pfft
Earlier quoted context omitted.
I don’t disagree, but fools will still be fooled. And there are a lot of fools. I do wonder what it means for the future of the internet. I don’t think net good is coming out of this.
Realistically it means Facebook style log in on all sites worth commenting on. The only way to prevent legions of bots, and the only way govs can keep enemy psyops at bay, will be online persona's tied to real life identitys.
Earlier quoted context omitted.
I think it’s a bigger problem than fake news. Sure, LLMs can generate that, but what they can do much better than prior disinformation automation is have tailored, context-aware conversations. So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing. It’s complete…
> Russian disinformation tactics but massively scaled up. And those were already wildly effective. Russian disinformation's success in the 2016 election is massively over hyped for the usual partisan sour grapes reasons. You cannot move the world with six figures of Facebook ads, if you could, everyone would spend a lot more money on Facebook ads.
Earlier quoted context omitted.
> So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing. And the dissenting opinion will be able to do the same. Twelve year old kids will be running swarms of these for fun, and the technology will be so widely proliferated that everyone will encounter it dail…
> And the dissenting opinion will be able to do the same. if they have the money > Twelve year old kids pfft
Earlier quoted context omitted.
Probably. And you could surround specific communities en masse. And it’s coming soon to every single site near you.
More scary: you could target individuals and surround them with a bunch of fake persons that they have no way of differentiating from real ones and slowly push them in a direction of your choosing.
This could be pervasive through not just online discussion forums, but also online news articles, auto-generated YouTube/TikTok feeds, pages inserted into search results that are custom generated on-demand, conversations on dating apps, etc.
Earlier quoted context omitted.
More scary: you could target individuals and surround them with a bunch of fake persons that they have no way of differentiating from real ones and slowly push them in a direction of your choosing.
Even more scary: You can tailor each individual's completely unique online universe to dovetail with the equally unique online universes of their IRL social connections / networks so that when they get together again at the holidays or meet for the first time at a bar they both frequent, they have serendipitous conversations about discovering the same hyper-niche thing recently, reinforcing their online conditioning…
What could go wrong?