Disrupting malicious uses of AI by state-affiliated threat actors

Using it to research a target and also generate phishing content for that target are pretty big IMO.

I mean, this is essentially how current AI poses a threat. And it is a threat.

Imagine a massive flood of low-quality images relevant to current events (e.g. war), which are obviously fabricated to anyone remotely aware of AI generation. But there are a lot of people who aren't aware of AI generation, or just not paying attention, who will take the photos at face value. (You'd probably be one of the latter. How many times did you "inspect" a photo in a news article to make sure it wasn't faked? How many of those photos slipped past inspection into your subconscious? Subliminal messaging has never been easier or more widespread.)

Also imagine a lot of vigilantes who are stupid enough to be vigilantes in this day and age, who ordinarily couldn't do the bare minimum amount of research to cause real harm. But now they can ask ChatGPT "how do I rob a bank?" and get advice which is still pretty bad, but better than what they'd come up with on their own (if they wouldn't just give up entirely), so it causes more damage.

A trained professional who wants a quality deepfake can already use photoshop and video editing tools, and a trained criminal already knows how to do research. But there aren't a lot of those people. Massive low-quality spam and low-level crimes are useful even for a state actor with huge resources, because they cause general instability in ways a few quality hits can't.

There's another issue that a powerful state actor can simply build and deploy their own language model. However, it probably won't be as good as OpenAI's (quality may have some effect) and it won't be wasting their resources.

Surely failing to be useful to paying customers is worse?

Groups sponsored by governments can easily write content for phishing campaigns, research companies, and write malware without the help of AI. They can also afford the GPUs to run local models if that gives them a boost.

As a general subject of concern in terms of state-level threat actors, yes, but more specifically:

https://www.cnbc.com/2024/01/16/openai-quietly-removes-ban-o...

Let's say some state-level entity asks OpenAI for access to its best code generating models to help it build software for autonomous kill vehicles that use face recognition algorithms to assassinate human targets. Most people would classify the end product as "a thing that should be banned internationally".

Consider IBM's history - IBM supplied its machines and technology to just about any private or state entity willing to sign a contract - and in most cases the result was beneficial to every sector of the economy, and improved government efficiency as well. IBM survived the Great Depression in part with a large Social Security management contract from FDR, and had several large military contracts afterwards, including in Vietnam for a decade. But there was also the German arm of the business in the 1930s, which I'd hope IBM leadership regrets in hindsight.

As the LLM technology platform seems a bit difficult to monetize at present, it's likely that the sector will be looking at large government contracts to sustain its growth over the next decade (see AWS and $10 billion for the NSA's "WildandStormy" contract (yes really)). Thus it would be nice to hear industry leaders explicitly state that using AI systems to write code for autonomous kill vehicle operations or to mine phone records for automated generation of assassination lists is unacceptable.

Transparency is going to be an issue - secret contracts for AI services should not be allowed.