Live data from Hacker News

Disrupting a covert Iranian influence operation

openai.com

101–110 of 155 posts

Re: Disrupting a covert Iranian influence operation

#101
post #80

CNN just did a piece on private Israeli groups doing the exact same thing. It is pretty scary the shear scale of this. Literally any entity or group can spin up a ton of bots and use any AI service locally or otherwise, to attempt to sway public opinion.

It’s frustrating too seeing some people in AI say the technology has no negative implications. Of course it does. Foreign actors are using it to influence elections. But when Jeremy Howard was asked if he had done impact assessment of a large open source model release, he replied (roughly) “No. Impact is about the same as a new release of pencil and paper.”

Which… actually that’s not true. Iran isn’t using pencil and paper to influence US elections, they’re using LLMs. And if all the big cloud providers cut them off, they’ll move to using home brew models. That is the impact and putting our head in the sand about it isn’t going to be helpful.

This is NOT to say we should not release to models! I think we should release open models, and that in addition to that we should also care about written impact assessments so everyone is on the same page about risks! I think it’s great that Jeremy Howard is releasing open foundation models, but I wish he wasn’t so flippant about the real world implications. I’ve always looked up to him for the work he has done, and found that attitude very disappointing.

Re: Disrupting a covert Iranian influence operation

#102
post #61

Earlier quoted context omitted.

https://www.nbcnews.com/tech/security/meta-openai-say-disrup... https://openai.com/index/disrupting-deceptive-uses-of-AI-by-... OpenAI has also banned Israeli influence operations. Do you think the above isn't going far enough? It's also strange that you call out all pro-Israel comments as a cause for concern.

>OpenAI has also banned Israeli influence operations. Do you think the above isn't going far enough? How is that far enough when their head of research is a genocidal zionist?[1] >It's also strange that you call out all pro-Israel comments as a cause for concern. What's so strange about calling pro-Israel comments as a cause for concern when Israel has been on what jewish holocaust scholars like Amos Goldberg & Raz S…

> How is that far enough when their head of research is a genocidal zionist?[1]

I was thinking that this is probably just a little bit extreme view for supporting Israel but boy this guy place is prison [1]

[1] https://x.com/StopArabHate/status/1806450543230857697

Re: Disrupting a covert Iranian influence operation

#103

One area that OpenAI did not comment on, was the likelihood that the AI-generated content here was itself used in the training data for a later model. Let's say the OpenAI engineers are working on ChatGPT 5 and spent last month scraping Teorater and X/Twitter, where this material ended up. How does OpenAI know that the new model is not poisoned? This isn't just OpenAI's problem of course. Anyone training on the open…

> This isn't just OpenAI's problem of course. Anyone training on the open Internet now has this problem.

'Low-background Steel' has been a nerd trivia tidbit for a while now, will we have to consider a 'Pre' and 'Post' AI internet era for training uses?

Re: Disrupting a covert Iranian influence operation

#104

Nothing like 900 million articles about secret Iranian plots from government entangled dystopian megacorps and the pundits who love them, soon after the US starts moving troops into the Middle East to defend the progress of an ongoing genocide. I suppose this is a great opportunity for the people whose entire income comes from the fact that the US overpays insiders to supply its own military; if Trump wins, he gets t…

US is going to protect Hamas?

Re: Disrupting a covert Iranian influence operation

#105

One area that OpenAI did not comment on, was the likelihood that the AI-generated content here was itself used in the training data for a later model. Let's say the OpenAI engineers are working on ChatGPT 5 and spent last month scraping Teorater and X/Twitter, where this material ended up. How does OpenAI know that the new model is not poisoned? This isn't just OpenAI's problem of course. Anyone training on the open…

Maybe one of the uses of the generated content detection method that OpenAI supposedly has is they can filter at least some of it out?

Re: Disrupting a covert Iranian influence operation

#106
post #80

CNN just did a piece on private Israeli groups doing the exact same thing. It is pretty scary the shear scale of this. Literally any entity or group can spin up a ton of bots and use any AI service locally or otherwise, to attempt to sway public opinion.

There is a pro Russian bot that someone used prompt injection to prove it was a bot.

https://www.reddit.com/r/interestingasfuck/comments/1dlg8ni/...

Re: Disrupting a covert Iranian influence operation

#107
post #80

CNN just did a piece on private Israeli groups doing the exact same thing. It is pretty scary the shear scale of this. Literally any entity or group can spin up a ton of bots and use any AI service locally or otherwise, to attempt to sway public opinion.

Reddit especially is full of LLMs. I would imagine there are ones from every major country plus several privately funded ones.

they are here as well.

Re: Disrupting a covert Iranian influence operation

#108
post #61

Earlier quoted context omitted.

https://www.nbcnews.com/tech/security/meta-openai-say-disrup... https://openai.com/index/disrupting-deceptive-uses-of-AI-by-... OpenAI has also banned Israeli influence operations. Do you think the above isn't going far enough? It's also strange that you call out all pro-Israel comments as a cause for concern.

>OpenAI has also banned Israeli influence operations. Do you think the above isn't going far enough? How is that far enough when their head of research is a genocidal zionist?[1] >It's also strange that you call out all pro-Israel comments as a cause for concern. What's so strange about calling pro-Israel comments as a cause for concern when Israel has been on what jewish holocaust scholars like Amos Goldberg & Raz S…

[flagged]

Re: Disrupting a covert Iranian influence operation

#109
post #80

CNN just did a piece on private Israeli groups doing the exact same thing. It is pretty scary the shear scale of this. Literally any entity or group can spin up a ton of bots and use any AI service locally or otherwise, to attempt to sway public opinion.

Reddit especially is full of LLMs. I would imagine there are ones from every major country plus several privately funded ones.

Yeah, all of the major subreddits are gamed, especially subs like r/politics. Even going back to the 2016 election it was clear that much of the activity on there was not genuine.

Re: Disrupting a covert Iranian influence operation

#110
post #30

Earlier quoted context omitted.

They want to sow discord. Manipulating US politics in any meaningful way is difficult to do. It's easier to divide and conquer and turn your enemy against itself. In my opinion the West is already doing a good enough job at polarising itself, doesn't need much help from the enemy.

> Manipulating US politics in any meaningful way is difficult to do. Well empirically it's actually pretty cheap and easy. What's expensive and hard are the countermeasures against it. To manipulate politics you mainly need to generate fake social proof. To do it effectively you mainly need to target a relatively small number of counties in battleground states.

Do you have examples in mind of successful foreign manipulation of US politics?
Post reply on HN