Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disrupting malicious uses of AI by state-affiliated threat actors

openai.com

Post reply on HN
Print view View original

61–70 of 94 posts

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#61

Post by jjcm »

This whack-a-mole approach, while definitely good, is likely already dead as far as a way to prevent these actions. Local LLMs that have no restrictions will continue to get better.

If anything the best thing about this post is not the actions they've taken, but simply that they've shown us a snapshot of what the future will look like for state-affiliated actors' actions. The research aspect I think is a good thing - giving people full access to more information for how things are made and architected will likely be a net positive. The phishing aspect though is terrifying - it's going to be crazy seeing what the next decade looks like for phishing. I do wonder how long it takes before there's some sort of "verified as a human" type function in communications to try and combat this.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#62

Post by nonethewiser »

NLPaep wrote: post #56
nonethewiser wrote: post #52

Earlier quoted context omitted.

It's amazing to me how there are really only 4 named countries (China, North Korea, Iran, Russia). I guess it's probably just more political and state sponsored than I would have guessed. Or it's not based on prevalence of attack sources and it's dictated more directly by some US policy? For example, you might also expect India because 1) it's a massive country with many people, 2) it's fairly independent and at leas…

India’s an ally. It shares military bases with the USA and they’re also in each others favorite countries

That would be a very oversimplified conclusion. They are categorized as a major defense partner but are also rather close with Russia (getting 65% of their weapons from them [0]). They cooperate strategically but are rather independent. None of this is a slight to India - I'm just contrasting it to Europe which is decidedly in the US camp.

[0] https://www.reuters.com/world/india/india-pivots-away-russia...

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#63

Post by pphysch »

49874539 wrote: post #25
pphysch wrote: post #16

Earlier quoted context omitted.

[flagged]

> "Russia influenced the 2016 election" = some guy with a distant relationship to the Russian govt bought some really crappy Facebook ads in Florida. I'm not sure this is true. I'd highly recommend reading the Muller report. It provides a lot of detailed, specific, evidence of direct communication with the Russian government.

No it does not. Have you actually read it?

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#64

Post by nozzlegear »

ithkuil wrote: post #59
nozzlegear wrote: post #55

Earlier quoted context omitted.

In this case, if we're the store owner, we can just say "I don't like what you're doing in my store" and ban them. We don't have to play games where the person in our store gets to stay because they're not technically doing anything illegal. We just kick them out, it's our store. Although with Capone, your results probably would've varied. =P Anyway, to do away with the analogy, OpenAI isn't obligated to let these gr…

So you can kick out anybody because you don't like them (e.g. a minority you don't like the color of)? Or does it work only when somebody is "famously an universally bad" person (al Capone, Hitler etc)?"

I'm not interested in having bad faith arguments.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#65

Post by armchairhacker »

Aerbil313 wrote: post #46
armchairhacker wrote: post #34

Earlier quoted context omitted.

I mean, this is essentially how current AI poses a threat. And it is a threat. Imagine a massive flood of low-quality images relevant to current events (e.g. war), which are obviously fabricated to anyone remotely aware of AI generation. But there are a lot of people who aren't aware of AI generation, or just not paying attention, who will take the photos at face value. (You'd probably be one of the latter. How many…

> Imagine a massive flood of low-quality images relevant to current events (e.g. war), which are obviously fabricated to anyone remotely aware of AI generation. Have you seen photorealistic Midjourney images? I can never distinguish many of them in a million years.

https://www.reddit.com/r/ChatGPT/. They range from practically-indistinguishable to ridiculously wrong, but the more details, the more they lean towards ridiculous.

It does make it easier, and who knows about the future. But right now if you generate an image you have to really look and there's a good chance at least something's off.

Specific example: https://www.reddit.com/r/ChatGPT/comments/1apyrwv/which_vide.... They are video games so already not realistic, but no gym has people stand around like that and all of them have some nonsensical equipment.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#66

Post by RecycledEle »

I am concerned that in repressive regimes, the state will control what AI the people have access to. This will make it easy to create music that supports the regime and almost impossible to create music that is critical of the regime.

It will be like the pro-state people having assistants to create memes they imagine for them, while the anti-state people are left with paper and crayons, until the AI paper and AI crayons refuse to draw anti-state memes and then we find anti-state activists trying to learn bee keeping so they can get wax to make crayons to draw anti-state memes.

If you thought the election interference of the past was bad, that's nothing compared to what we will see in the near future.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#67

Post by 2OEH8eoCRo0 »

pphysch wrote: post #63
49874539 wrote: post #25

Earlier quoted context omitted.

> "Russia influenced the 2016 election" = some guy with a distant relationship to the Russian govt bought some really crappy Facebook ads in Florida. I'm not sure this is true. I'd highly recommend reading the Muller report. It provides a lot of detailed, specific, evidence of direct communication with the Russian government.

No it does not. Have you actually read it?

https://www.justice.gov/archives/sco/file/1373816/download

> The Internet Research Agency (IRA) carried out the earliest Russian interference operations identified by the investigation—a social media campaign designed to provoke and amplify political and social discord in the United States.

> At the same time that the IRA operation began to focus on supporting candidate Trump in early 2016, the Russian government employed a second form of interference: cyber intrusions (hacking) and releases of hacked materials damaging to the Clinton Campaign

> On June 9, 2016, senior representatives of the Trump Campaign met in Trump Tower with a Russian attorney expecting to receive derogatory information about Hillary Clinton from the Russian government

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#68

Post by __loam »

yusml wrote: post #15
maldev wrote: post #6

Fuck openai, I'm a security researcher and if you dare ask it about anything Windows related it tells you to screw off. Linux? Fine. But ask about some undocumented Windows behavior and it says it can't. Ask it about patchguard internals as a reference? Tells you it can't assist. Absolutely crazy, I can understand asking it to write straight up malware, oh wait, it does that no issue! Lord help you if you want to use…

Agreed. Open AI defo need to tune their models so that it can provide enough information but not too much info which can be malicious. Currently, anything remotely close is flagged as malicious.

Who gets to decide what is and isn't malicious

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#69

Post by webdoodle »

GaggiX wrote: post #5
chpatrick wrote: post #3

You'd think the smart thing to do would be to let them use it, see what they're doing and subtly mess with the results.

That's probably what happened. OpenAI should already have a big chunk of data.

This is a well established intelligence tactic. It was used by the Clinton administration to poison the well of nuclear development in Iran.

https://en.wikipedia.org/wiki/Operation_Merlin

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#70

Post by pphysch »

2OEH8eoCRo0 wrote: post #67
pphysch wrote: post #63

Earlier quoted context omitted.

No it does not. Have you actually read it?

https://www.justice.gov/archives/sco/file/1373816/download > The Internet Research Agency (IRA) carried out the earliest Russian interference operations identified by the investigation—a social media campaign designed to provoke and amplify political and social discord in the United States. > At the same time that the IRA operation began to focus on supporting candidate Trump in early 2016, the Russian government emp…

None of those constitute "direct communication [of Trump team] with the Russian government".
Post reply on HN