Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disrupting malicious uses of AI by state-affiliated threat actors

openai.com

Post reply on HN
Print view View original

71–80 of 94 posts

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#71

Post by perihelions »

webdoodle wrote: post #69
GaggiX wrote: post #5

Earlier quoted context omitted.

That's probably what happened. OpenAI should already have a big chunk of data.

This is a well established intelligence tactic. It was used by the Clinton administration to poison the well of nuclear development in Iran. https://en.wikipedia.org/wiki/Operation_Merlin

Also by the Reagan administration to sabotage economic development in the USSR,

https://www.washingtonpost.com/archive/politics/2004/02/27/r...

- "In January 1982, President Ronald Reagan approved a CIA plan to sabotage the economy of the Soviet Union through covert transfers of technology that contained hidden malfunctions, including software that later triggered a huge explosion in a Siberian natural gas pipeline, according to a new memoir by a Reagan White House official."

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#72

Post by Natfan »

ithkuil wrote: post #59
nozzlegear wrote: post #55

Earlier quoted context omitted.

In this case, if we're the store owner, we can just say "I don't like what you're doing in my store" and ban them. We don't have to play games where the person in our store gets to stay because they're not technically doing anything illegal. We just kick them out, it's our store. Although with Capone, your results probably would've varied. =P Anyway, to do away with the analogy, OpenAI isn't obligated to let these gr…

So you can kick out anybody because you don't like them (e.g. a minority you don't like the color of)? Or does it work only when somebody is "famously an universally bad" person (al Capone, Hitler etc)?"

As long as you're not kicking them out for reasons surrounding a protected class (race, religious, sexuality etc), and you own the property, then yes? It's your land, you can ask them to leave and if they refuse then they're trespassing.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#73

Post by justsomehnguy »

paxys wrote: post #18
mise_en_place wrote: post #17

If this is the case, there must be a serious competency crisis in foreign intelligence agencies. It’s trivial to run your own local model.

> It’s trivial to run your own local model With what GPUs?

The ones made in China?

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#74

Post by [deleted] »

RecycledEle wrote: post #66

I am concerned that in repressive regimes, the state will control what AI the people have access to. This will make it easy to create music that supports the regime and almost impossible to create music that is critical of the regime. It will be like the pro-state people having assistants to create memes they imagine for them, while the anti-state people are left with paper and crayons, until the AI paper and AI cray…

[deleted]

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#75

Post by mschuster91 »

nonethewiser wrote: post #52
guessmyname wrote: post #28

Earlier quoted context omitted.

How Microsoft names threat actors → https://learn.microsoft.com/en-us/microsoft-365/security/def... Microsoft shifts to a new threat actor naming taxonomy → https://www.microsoft.com/en-us/security/blog/2023/04/18/mic...

It's amazing to me how there are really only 4 named countries (China, North Korea, Iran, Russia). I guess it's probably just more political and state sponsored than I would have guessed. Or it's not based on prevalence of attack sources and it's dictated more directly by some US policy? For example, you might also expect India because 1) it's a massive country with many people, 2) it's fairly independent and at leas…

There aren't that many countries in the world with nation-scale hacker groups - the only ones I'd add is Israel with all their involvement into commercial spyware (and Mossad, whose capabilities likely surpass even the NSA), but they're heavily allied with the US.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#76

Post by miohtama »

jjcm wrote: post #61

This whack-a-mole approach, while definitely good, is likely already dead as far as a way to prevent these actions. Local LLMs that have no restrictions will continue to get better. If anything the best thing about this post is not the actions they've taken, but simply that they've shown us a snapshot of what the future will look like for state-affiliated actors' actions. The research aspect I think is a good thing -…

ChatGPT will tell only information that it has indexed from Internet.

This means the same information you get from ChatGPT is available from a Google search. Based on the listed queries ("programming help") ChatGPT does not create much value for national security threat actors here.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#77

Post by miohtama »

NLPaep wrote: post #58
Cheer2171 wrote: post #8

Much of the alleged "malicious uses" seem to only be malicious because of who the actors are. How dare they translate published technical papers!

Technical papers on exploits

You can use Google Translate to do the same. No one is worried about Google Translate.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#78

Post by blueblimp »

Cheer2171 wrote: post #9

What this tells me is that if you think you have any privacy with OpenAI by turning off chat history in ChatGPT or exercising your California or EU privacy rights, you are kidding yourself. In the name of national security, anything you send to OpenAI can be used against you.

Yep. Unfortunately, all the major LLM APIs do surveillance on your prompts and responses.

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#79

Post by t_mann »

NameError wrote: post #10

Looking over the specifics, the striking thing about this to me is that it seems like these supposedly-sophisticated covert operatives are just going to ChatGPT (or similar) and basically asking "how do I make good malware?"

Isn't that how a lot of software gets written now? Why would malware be different?

Re: Disrupting malicious uses of AI by state-affiliated threat actors

#80

Post by saagarjha »

Cheer2171 wrote: post #60
NLPaep wrote: post #58

Earlier quoted context omitted.

Technical papers on exploits

Doesn't matter. This is leading down a path where if you run a service like Google Translate, you will be expected to restrict certain users (or all users) from translating a publicly-accessible technical report from a security research journal. I can see the same national security logic saying that we can't let security papers be translated to Mandarin, Russian, Korean, or Farsi, because enemies of the United States…

I don’t think Google Translate does this, though?
Post reply on HN