Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models in national security applications

arxiv.org

Post reply on HN
Print view View original

21–30 of 60 posts

Re: Large language models in national security applications

#21

Post by [deleted] »

joe_the_user wrote: post #15
perihelions wrote: post #5

The most obvious way the US national security industry could use LLM's right now is simply to spam foreign adversaries with chatbots. That's their greatest strength right now—a use-case they have amply proven themselves for. This paper comes off as eager to avoid this topic: they (briefly) talk about detecting foreign LLM spam, which is called propaganda, but sidestep the idea of our own side using it. If we were con…

Oh, national security professionals aren't going to be talking about psyops, offensive applications and etc, because such things make a given state look bad - they're an offense against democracy and respect for facts and they make the overt media of a given nation look bad. But hey, leave it to HN commentators to root for taking the gloves off. Not to worry post, I'd bet dollars to donuts the actual classified discu…

[deleted]

Re: Large language models in national security applications

#22

Post by psunavy03 »

gmaster1440 wrote: post #10
ben_w wrote: post #8

Earlier quoted context omitted.

LLMs can combine cross-domain insights, but the insights they have — that I've seen them have in the models I've used — are around the level of a second year university student. I would concur with what the abstract says: incredibly valuable (IMO the breadth of easily discoverable knowledge is a huge plus all by itself), but don't put them in charge.

The "second year university student" analogy is interesting, but might not fully capture what's unique about LLMs in strategic analysis. Unlike students, LLMs can simultaneously process and synthesize insights from thousands of historical conflicts, military doctrines, and real-time data points without human cognitive limitations or biases. The paper actually makes a stronger case for using LLMs to enhance rather tha…

But the aide won't have deeply analyzed every military campaign in history; it will only spout off answers from books about those campaigns. It will have little to no insight on how to apply principles and lessons learned from similar campaigns in the current problem. Wars are not won by lines on maps. They're not won by cool gear. They're won by psychologically beating down the enemy until they're ready to surrender or open peace negotiations. Can LLMs get in an enemy's head?

Re: Large language models in national security applications

#23

Post by photochemsyn »

It's rather conspicuous that the most well-known use of AI systems in warfare at present, the Lavender / Gospel / Where's Daddy systems used by the IDF, don't get any mention. It's true that LLMs are not the central component of these systems, which have much more in common with Google's targeted ad serving algorithms, in the broader category of machine learning, but a no-code LLM interface is a likely component.

In defensive red team scenarios, such an LLM system could be used in all kinds of nefarious ways, using prompts like "provide a list of everyone associated with the US nuclear weapons program, including their immediate friend and family circles, and ranking them by vulnerability to blackmail based on their personal web browsing history" and so on.

Re: Large language models in national security applications

#24

Post by fragmede »

psunavy03 wrote: post #22
gmaster1440 wrote: post #10

Earlier quoted context omitted.

The "second year university student" analogy is interesting, but might not fully capture what's unique about LLMs in strategic analysis. Unlike students, LLMs can simultaneously process and synthesize insights from thousands of historical conflicts, military doctrines, and real-time data points without human cognitive limitations or biases. The paper actually makes a stronger case for using LLMs to enhance rather tha…

But the aide won't have deeply analyzed every military campaign in history; it will only spout off answers from books about those campaigns. It will have little to no insight on how to apply principles and lessons learned from similar campaigns in the current problem. Wars are not won by lines on maps. They're not won by cool gear. They're won by psychologically beating down the enemy until they're ready to surrender…

Only if the enemy has provided a large corpus of writing and other data to submit to train the LLM on.

Re: Large language models in national security applications

#25

Post by ben_w »

psunavy03 wrote: post #22
gmaster1440 wrote: post #10

Earlier quoted context omitted.

The "second year university student" analogy is interesting, but might not fully capture what's unique about LLMs in strategic analysis. Unlike students, LLMs can simultaneously process and synthesize insights from thousands of historical conflicts, military doctrines, and real-time data points without human cognitive limitations or biases. The paper actually makes a stronger case for using LLMs to enhance rather tha…

But the aide won't have deeply analyzed every military campaign in history; it will only spout off answers from books about those campaigns. It will have little to no insight on how to apply principles and lessons learned from similar campaigns in the current problem. Wars are not won by lines on maps. They're not won by cool gear. They're won by psychologically beating down the enemy until they're ready to surrender…

> Can LLMs get in an enemy's head?

That may be much easier for an LLM than all the other things you listed.

Read their socials, write a script that grabs the voices and faces of their loved ones from videos they've shared, synthesise a video call… And yes, they can write the scripts even if they don't have the power to clone voices and faces themselves.

I have no idea what's coming. But this is going to be a wild decade even if nothing new gets invented.

Re: Large language models in national security applications

#26

Post by JoshTriplett »

bilbo0s wrote: post #16
perihelions wrote: post #5

The most obvious way the US national security industry could use LLM's right now is simply to spam foreign adversaries with chatbots. That's their greatest strength right now—a use-case they have amply proven themselves for. This paper comes off as eager to avoid this topic: they (briefly) talk about detecting foreign LLM spam, which is called propaganda, but sidestep the idea of our own side using it. If we were con…

Or, to rephrase this as a question: Is it ethical to spam another democracy with millions of chatbots pretending to be citizens of that country—if the effect is to manipulate those citizens to [take any action advantageous to US national interest]...? Just, Devil's Advocate, but ethical or not, that's what we should be doing and what we are doing. Every nation has its sock puppets out there, our job is to stop everyo…

That's not inherently true. If there were a way to reliably destroy all the sock puppets, we should, and the world would be better off. For instance, reliable bot detection, or mechanisms by which major social networks could detect and prohibit bot-like activity that isn't labeled as such.

Re: Large language models in national security applications

#27

Post by beardedwizard »

gmaster1440 wrote: post #4

The paper argues against using LLMs for military strategy, claiming "no textbook contains the right answers" and strategy can't be learned from text alone (the "Virtual Clausewitz" Problem). But this seems to underestimate LLMs' demonstrated ability to reason through novel situations. Rather than just pattern-matching historical examples, modern LLMs can synthesize insights across domains, identify non-obvious patter…

A language model isn't a model of strategic conflict or reasoning, but may contain text in its training data related to these concepts. I'm unclear why (and it seems the paper agrees) you would use the llm to reason when there are better models for reasoning about the problem domain - and the main value from llm is ability to consume unstructured data to populate the other models.

Re: Large language models in national security applications

#28

Post by Onavo »

perihelions wrote: post #5

The most obvious way the US national security industry could use LLM's right now is simply to spam foreign adversaries with chatbots. That's their greatest strength right now—a use-case they have amply proven themselves for. This paper comes off as eager to avoid this topic: they (briefly) talk about detecting foreign LLM spam, which is called propaganda, but sidestep the idea of our own side using it. If we were con…

The platform you are referring to is called "Reddit", one of YC's portfolio companies.

Re: Large language models in national security applications

#29

Post by exe34 »

JoshTriplett wrote: post #26
bilbo0s wrote: post #16

Earlier quoted context omitted.

Or, to rephrase this as a question: Is it ethical to spam another democracy with millions of chatbots pretending to be citizens of that country—if the effect is to manipulate those citizens to [take any action advantageous to US national interest]...? Just, Devil's Advocate, but ethical or not, that's what we should be doing and what we are doing. Every nation has its sock puppets out there, our job is to stop everyo…

That's not inherently true. If there were a way to reliably destroy all the sock puppets, we should, and the world would be better off. For instance, reliable bot detection, or mechanisms by which major social networks could detect and prohibit bot-like activity that isn't labeled as such.

charge one penny for every post. most people can afford it. bots become less cost effective and you'd be able to trace the source of funds.

Re: Large language models in national security applications

#30

Post by psunavy03 »

ben_w wrote: post #25
psunavy03 wrote: post #22

Earlier quoted context omitted.

But the aide won't have deeply analyzed every military campaign in history; it will only spout off answers from books about those campaigns. It will have little to no insight on how to apply principles and lessons learned from similar campaigns in the current problem. Wars are not won by lines on maps. They're not won by cool gear. They're won by psychologically beating down the enemy until they're ready to surrender…

> Can LLMs get in an enemy's head? That may be much easier for an LLM than all the other things you listed. Read their socials, write a script that grabs the voices and faces of their loved ones from videos they've shared, synthesise a video call… And yes, they can write the scripts even if they don't have the power to clone voices and faces themselves. I have no idea what's coming. But this is going to be a wild dec…

Creating chaos and confusion is great, but it's only part of what a military campaign needs. You have to be able to use all levers of government power to put the other government or the adversary organization in a point where they feel compelled to quit or negotiate.
Post reply on HN