Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shall we play a game? My AI nuclear simulation

kennethpayne.uk

Post reply on HN
Print view View original

101–110 of 213 posts

Re: Shall we play a game? My AI nuclear simulation

#101

Post by [deleted] »

chaseadam17 wrote: post #98
Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

I'd argue we don't even know what "intelligence" or "self-awareness" mean. Humans are conscious which means we experience things, then we develop preferences for certain experiences, then we develop skills for achieving those preferences. Without consciousness, what is there to be aware of? And why would intelligence emerge and/or what end would it serve?

[deleted]

Re: Shall we play a game? My AI nuclear simulation

#102

Post by notJim »

jerf wrote: post #52

The most interesting takeaway for me is the three very distinct personalities. Three models all based on the same tech, trained in the same manner, trained by three groups of people with similar ideological outlooks, and the result is three very different AIs. The military basically wants an oracle. Feed the AI the situation, get the best answer out. But if the AIs are as diverse and opinionated as humans, it is deba…

What's interesting is that the LLMs' coding personalities seem to match their policy WRT to strategy, which suggests an underlying consistency.

Claude, for example, is very eager to begin coding, and very persistent. It tends to exit plan mode even when the plan is half-baked, and will go as far as deleting tests to get the suite to "pass."

ChatGPT on the other hand is very hesitant. It loves to pause and ask for permission before it starts coding, and gives up quickly if it runs into a problem. This is similar to its tendency toward passivity in the strategy simulation presented here.

Re: Shall we play a game? My AI nuclear simulation

#103

Post by stevenwoo »

rphv wrote: post #13

Hm maybe humans are nicer/more moral than AI given that the use of tactical nukes has only happened once.

Tactical means battlefield, attacking cities and infrastructure means strategic. Tactical nuclear weapons took a while to develop after 1945 - they have never been used.

Re: Shall we play a game? My AI nuclear simulation

#104

Post by Levitating »

ekelsen wrote: post #86

I wouldn't be surprised if humans behaved the same way when playing the same game? Like even if you brought me into a room and told me I was controlling "real nuclear weapons" I wouldn't believe you.

I think is an important point, and I don't see it mentioned in the article or the paper (though I skimmed the latter).

They are aware of what they are and how they are used. They're told to act as AI assistants. And there's theories of them being aware of their answers influencing their training.

So surely they must be able to reason that they're not literally controlling weapons of mass-destruction with their answers.

Re: Shall we play a game? My AI nuclear simulation

#105

Post by godwinson__4-8 »

andix wrote: post #88
roadside_picnic wrote: post #74

Earlier quoted context omitted.

> It's like the infinite monkeys on typewrighters that will type whatever you are looking for, given infinite time. In the monkey example the infinite time is doing a lot of work there. The fact that LLMs can search through semantic space and find reasonably correct paths in a reasonable time is directly tied to the reason why they are valuable. Saying "these two things are similar except one can be useful and one ca…

> but so much code writen is just the human brain doing the same autocomplete style work that LLMs can do now. That's the part they are really good at. But they are really bad at taking complex decisions. Most of them are just guesses from a finite amount of solutions they were trained on, or from options they have in context.

Indeed. Humans are well known for being good at "taking complex decisions" for which they have no "training", "options" or "context".

Re: Shall we play a game? My AI nuclear simulation

#106

Post by operatingthetan »

Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

>Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness.

The problem is many people seem to believe they have these things and some of those people will put LLMs into situations where this becomes dangerous.

Re: Shall we play a game? My AI nuclear simulation

#107

Post by Scubabear68 »

My personal take is a pre-requisite of true human-like AI is physical feedback and a concept of emotions or something like it.

Without physical feedback you can rapidly devolve into unstable positive feedback loops. And emotions are what help us process and react to that feedback.

Kids learn partially because their friends say sharp words that hurt them, fire burns them, they go hungry and starve if they don’t plan for meals.

Humans in the loop, MCP, etc are all very primitive hacks that are mimicing feedback and emotion, poorly.

Re: Shall we play a game? My AI nuclear simulation

#108

Post by anon84873628 »

flir wrote: post #92
Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

I reckon the context is all the fiction they've read where the AI blows up the world. They're just behaving like fictional AIs are supposed to behave. In so many of these scenarios, they're basically being asked to play an RPG.

I don't think the pre-training phase is responsible for much of their "personality". At least not so directly on a specific topic like this.

Re: Shall we play a game? My AI nuclear simulation

#110

Post by DaiPlusPlus »

sohex wrote: post #35

Sonnet, GPT-5.2, Gemini Flash, in a set of 21 games, where conclusions are drawn from the LLMs self reported reasoning. This is like writing a paper about kids in a literal sandbox fighting over ‘territory’. The models employed don’t indicate the actual extents of machine reasoning even as we currently recognize them. They certainly don’t have the metacognition necessary to accurately understand their own reasoning.…

> “Chilling” shouldn’t be the take away here.

It is when you consider the personality currently occupying the office of US SecDef.

Post reply on HN