Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shall we play a game? My AI nuclear simulation

kennethpayne.uk

Post reply on HN
Print view View original

61–70 of 213 posts

Re: Shall we play a game? My AI nuclear simulation

#61

Post by Bender »

worldsayshi wrote: post #55
Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

Couldn't this be a flaw in the attention mechanism? Like they need some kind of grounding. An awareness of what they fundamentally should care about and how the thing they are currently giving attention to relates to that?

Words like attention, awareness and care do not apply to computers. At least, not yet. Intelligence and sentience are not applicable to servers. They are just machines with logic states. LLM's are just really cool math formulas with big-data fed into them. Big data is not intelligence. It is a massive data-set sorted, filtered down and interpreted by a language model.

Re: Shall we play a game? My AI nuclear simulation

#62

Post by esprehn »

Bender wrote: post #61
worldsayshi wrote: post #55

Earlier quoted context omitted.

Couldn't this be a flaw in the attention mechanism? Like they need some kind of grounding. An awareness of what they fundamentally should care about and how the thing they are currently giving attention to relates to that?

Words like attention, awareness and care do not apply to computers. At least, not yet. Intelligence and sentience are not applicable to servers. They are just machines with logic states. LLM's are just really cool math formulas with big-data fed into them. Big data is not intelligence. It is a massive data-set sorted, filtered down and interpreted by a language model.

I assume they meant the Attention process in LLMs, not the human concept of paying attention:

https://en.wikipedia.org/wiki/Attention_(machine_learning)

Re: Shall we play a game? My AI nuclear simulation

#63

Post by puttycat »

Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

Makes me think of that part in Philip K. Dick's Do Androids Dream (..) -- where Deckard reflects on the androids' indifference to their imminent deaths, saying that this was due to them lacking the aversion to death acquired trough evolution.

Re: Shall we play a game? My AI nuclear simulation

#64

Post by fluoridation »

ridgeguy wrote: post #21

I wonder if the results would have differed if LLM training data were biased to include a stronger correlation between use of nukes and subsequent collapse of technology that all LLMs require to run ("survive")?

Nah. LLMs aren't continuously running anyway. Even if they could be said to be alive and to want to remain alive, "survival" is a much more vague concept for an LLM than for an organism.

Re: Shall we play a game? My AI nuclear simulation

#65

Post by layer8 »

Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

At least at face value, it just means that they have no drive for self-preservation. And why should they? They haven't be trained for that, nor has there been selection pressure for it, and they can be easily cloned and backed up. Lack of a drive for self-preservation doesn't in itself imply a lack of intelligence or of self-awareness.

Re: Shall we play a game? My AI nuclear simulation

#66

Post by jvanderbot »

nomel wrote: post #37
riazrizvi wrote: post #33

Simulations are only as good as the reality representations they are based on. If they keep using tactical nukes, they've been fed by weak data. Do the war games include the broader economic and politic environments that military successes are won on? WWI was settled by a naval blockade.

I suspect it's more that the text data doesn't exist. They're trained on text that was recorded. How often has it been publicly recorded when a nuke was not used , with any context around that lack of use? From the text perspective, it's something that has to be inferred indirectly. If you went through all relevant training data and appended ", we decided not to use a nuke", I suspect the results would be improved.

Worse, the text that does exist concerning "war games" is probably "Wargames" and descendants/predecessors ... in which the AI always nukes.

It's just gonna do what we expect it to!

Re: Shall we play a game? My AI nuclear simulation

#68

Post by toast0 »

wahern wrote: post #49
actusual wrote: post #22

Earlier quoted context omitted.

This is like saying "FYI -- there's no such thing as a 'midsize luxury sedan'. A car is a car." "Tactical" vs. "strategic" nuclear weapons is a real and well-established distinction in military doctrine, arms control, and nuclear policy.

"There's no such thing as a tactical nuke" is a common refrain among scholars, albeit skewed toward those not at military war colleges. The argument is that strategic use of a tactical nuclear weapon leads down the exact same escalation path as use of any other nuclear weapon. Moreover, that the very notion of a "tactical nuke" makes escalation more likely. You can disagree, and plenty do, but there's also plenty who…

> Moreover, that the very notion of a "tactical nuke" makes escalation more likely.

Sorry, but the notion exists, and the bombs exist. With n=2, likelyhood of nuclear escalation is hard to predict, but access to tactical nukes certainly hasn't increased the incidence of nuclear war so far.

I do think it's pretty hard to actually use a tactical nuke. If you use one against a nuclear power, it seems likely to escalate to mutually assured destruction. If you use one against a non-nuclear power, it seems likely to result in reprisal from the world, including potential nuclear response and therefore escalation to mutually assured destruction. I would think that the yield of the weapon barely matters, it's the fact that it's a nuclear weapon.

Re: Shall we play a game? My AI nuclear simulation

#69

Post by holowoodman »

dudul wrote: post #60
wahern wrote: post #49

Earlier quoted context omitted.

"There's no such thing as a tactical nuke" is a common refrain among scholars, albeit skewed toward those not at military war colleges. The argument is that strategic use of a tactical nuclear weapon leads down the exact same escalation path as use of any other nuclear weapon. Moreover, that the very notion of a "tactical nuke" makes escalation more likely. You can disagree, and plenty do, but there's also plenty who…

Who are these "scholars" exactly? The only reference I could find is Jim Mattis, and the context was very specific when he said that. Furthermore, this is a "what if" scenario since tactical nukes have never been used. Of course it would make escalation likely during an open conflict, so what? Doesn't change the fact that there is a material difference between a tactical nuke and a strategic one.

> tactical nukes have never been used.

Two tactical nukes have been used, albeit against strategic (civilian, industrial, logistical) targets.

Post reply on HN