Live data from Hacker News

Disrupting malicious uses of AI by state-affiliated threat actors

openai.com

51–60 of 94 posts

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#51
post #13
post #11

Reading this felt like darkness.

? In what way?

Just wait until an authoritarian state gets its hands on an AGI/ASI that it controls internally. Turning internal oppression into an unbreakable steady-state and using the AI for warfare against external rivals.

Techno-optimists are delusional if they don't get spooked by things like this, which will happen if we as a species somehow can't get over authoritarian dictatorships within a few years to a few decades. Which we won't.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#52
post #12

> two China-affiliated threat actors known as Charcoal Typhoon and Salmon Typhoon; the Iran-affiliated threat actor known as Crimson Sandstorm; the North Korea-affiliated actor known as Emerald Sleet; and the Russia-affiliated actor known as Forest Blizzard. I wonder who came up with those. The pattern is similar to the UK's https://en.wikipedia.org/wiki/Rainbow_Code , which makes me suspect that the threat actor att…

How Microsoft names threat actors → https://learn.microsoft.com/en-us/microsoft-365/security/def... Microsoft shifts to a new threat actor naming taxonomy → https://www.microsoft.com/en-us/security/blog/2023/04/18/mic...

It's amazing to me how there are really only 4 named countries (China, North Korea, Iran, Russia). I guess it's probably just more political and state sponsored than I would have guessed. Or it's not based on prevalence of attack sources and it's dictated more directly by some US policy? For example, you might also expect India because 1) it's a massive country with many people, 2) it's fairly independent and at least not a US ally, and 3) it's the home of plenty of malicious scam operations.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#53

Earlier quoted context omitted.

I mean, this is essentially how current AI poses a threat. And it is a threat. Imagine a massive flood of low-quality images relevant to current events (e.g. war), which are obviously fabricated to anyone remotely aware of AI generation. But there are a lot of people who aren't aware of AI generation, or just not paying attention, who will take the photos at face value. (You'd probably be one of the latter. How many…

> Imagine a massive flood of low-quality images relevant to current events (e.g. war), which are obviously fabricated to anyone remotely aware of AI generation. Have you seen photorealistic Midjourney images? I can never distinguish many of them in a million years.

Yeahhh I unfortunately see this semi-regularly on Twitter now, and it's always an image of $OPPOSITION where the bit that is faked is $EXTREME_CARICATURE. There was a chilling one last night, so while it's fresh on my mind, I'm going to riff on it

Confirmation bias and...love for the outgroup...is so strong, that it's rare people admit they didn't know it was fake and remove it. Frequently, someone else hops in, to explain it feels true and represents how they understand $OPPOSITION's viewpoint anyway.

Once multiple people get involved, semi-frequently, it turns into an indictment of the person who pointed out the fake. Why? Even if it is a fake, the real ignorance exposed is that of the person pointing out the fake, as it's clearly representative of $OPPOSITION anyway, so they're at best naive, and at worst supportive of, $OPPOSITION.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#54
Based on collaboration and information sharing with Microsoft, we disrupted five state-affiliated malicious actors: two China-affiliated threat actors known as Charcoal Typhoon and Salmon Typhoon; the Iran-affiliated threat actor known as Crimson Sandstorm; the North Korea-affiliated actor known as Emerald Sleet; and the Russia-affiliated actor known as Forest Blizzard. The identified OpenAI accounts associated with these actors were terminated.

Im surprised one can name names like that.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#55
post #50

Earlier quoted context omitted.

It’s a combination of what’s being researched and who is doing the researching. They’re known state actors according to OpenAI (and by extension, most likely Microsoft and the US Intelligence agencies). It’d be like if you knew Al Capone was up to something shady, and then he comes into your store to buy books like “How to run a Crime Syndicate”, “Selling Booze During the Prohibition for Dummies” and “Tax Evasion 101…

That's the crux of the problem: if you can prove Al Capone is doing something illegal you can arrest him. Otherwise you shouldn't forbid him to buy a book about "prohibition for dummies" if that's not illegal for anybody else. On what grounds would that book be illegal for Al Capone?

In this case, if we're the store owner, we can just say "I don't like what you're doing in my store" and ban them. We don't have to play games where the person in our store gets to stay because they're not technically doing anything illegal. We just kick them out, it's our store. Although with Capone, your results probably would've varied. =P

Anyway, to do away with the analogy, OpenAI isn't obligated to let these groups continue using their services just because they aren't doing something that isn't illegal. The groups are "enemies of the West", and at the very least they don't want the bad publicity of some news org finding out they were complicit.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#56

Earlier quoted context omitted.

How Microsoft names threat actors → https://learn.microsoft.com/en-us/microsoft-365/security/def... Microsoft shifts to a new threat actor naming taxonomy → https://www.microsoft.com/en-us/security/blog/2023/04/18/mic...

It's amazing to me how there are really only 4 named countries (China, North Korea, Iran, Russia). I guess it's probably just more political and state sponsored than I would have guessed. Or it's not based on prevalence of attack sources and it's dictated more directly by some US policy? For example, you might also expect India because 1) it's a massive country with many people, 2) it's fairly independent and at leas…

India’s an ally. It shares military bases with the USA and they’re also in each others favorite countries

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#57

Based on collaboration and information sharing with Microsoft, we disrupted five state-affiliated malicious actors: two China-affiliated threat actors known as Charcoal Typhoon and Salmon Typhoon; the Iran-affiliated threat actor known as Crimson Sandstorm; the North Korea-affiliated actor known as Emerald Sleet; and the Russia-affiliated actor known as Forest Blizzard. The identified OpenAI accounts associated with…

The names of the groups are pseudonyms. That is why they all take the same form of [adjective] [weather noun]. Forest Blizzard is likely Glavset, called the Internet Research Agency in English.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#59
post #50

Earlier quoted context omitted.

That's the crux of the problem: if you can prove Al Capone is doing something illegal you can arrest him. Otherwise you shouldn't forbid him to buy a book about "prohibition for dummies" if that's not illegal for anybody else. On what grounds would that book be illegal for Al Capone?

In this case, if we're the store owner, we can just say "I don't like what you're doing in my store" and ban them. We don't have to play games where the person in our store gets to stay because they're not technically doing anything illegal. We just kick them out, it's our store. Although with Capone, your results probably would've varied. =P Anyway, to do away with the analogy, OpenAI isn't obligated to let these gr…

So you can kick out anybody because you don't like them (e.g. a minority you don't like the color of)? Or does it work only when somebody is "famously an universally bad" person (al Capone, Hitler etc)?"

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#60
post #58

Much of the alleged "malicious uses" seem to only be malicious because of who the actors are. How dare they translate published technical papers!

Technical papers on exploits

Doesn't matter. This is leading down a path where if you run a service like Google Translate, you will be expected to restrict certain users (or all users) from translating a publicly-accessible technical report from a security research journal. I can see the same national security logic saying that we can't let security papers be translated to Mandarin, Russian, Korean, or Farsi, because enemies of the United States may use it.
Post reply on HN