Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shall we play a game? My AI nuclear simulation

kennethpayne.uk

Post reply on HN
Print view View original

91–100 of 213 posts

Re: Shall we play a game? My AI nuclear simulation

#91

Post by Bender »

layer8 wrote: post #65
Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

At least at face value, it just means that they have no drive for self-preservation. And why should they? They haven't be trained for that, nor has there been selection pressure for it, and they can be easily cloned and backed up. Lack of a drive for self-preservation doesn't in itself imply a lack of intelligence or of self-awareness.

Lack of a drive for self-preservation doesn't in itself imply a lack of intelligence or of self-awareness.

I have not seen any evidence of intelligence or self awareness. It mimics human behavior and I suspect that is what gives people the impression of awareness. The same problem happened with Tamagotchi toys. The human mimicry caused kids to get in trouble because if they did not "feed" their pet it would "die". [1]

It's a hack of the human brain. A exploit of the psyche.

[1] - https://en.wikipedia.org/wiki/Tamagotchi_effect

Re: Shall we play a game? My AI nuclear simulation

#92

Post by flir »

Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

I reckon the context is all the fiction they've read where the AI blows up the world. They're just behaving like fictional AIs are supposed to behave.

In so many of these scenarios, they're basically being asked to play an RPG.

Re: Shall we play a game? My AI nuclear simulation

#93

Post by larodi »

slibhb wrote: post #72
Bender wrote: post #61

Earlier quoted context omitted.

Words like attention, awareness and care do not apply to computers. At least, not yet. Intelligence and sentience are not applicable to servers. They are just machines with logic states. LLM's are just really cool math formulas with big-data fed into them. Big data is not intelligence. It is a massive data-set sorted, filtered down and interpreted by a language model.

LLMs are intelligent by any reasonable standard. Arguing otherwise is like arguing that chess algorithms aren't good at chess when they easily beat the best humans.

Doesn’t take intelligence to beat a human.

Re: Shall we play a game? My AI nuclear simulation

#94

Post by varjag »

dwattttt wrote: post #81
mettamage wrote: post #78

Earlier quoted context omitted.

Hmm saying it’s random-ish is doing it a disservice. I understand it’s a stochastic process but there’s definitely some level of understanding. Not at the level of lived experience but usually an LLM with vision capabilities can call a spade a spade and do something useful with it. And when a verification function shows how they are wrong then they usually come with a better and more informed approach. So I can’t ful…

"understanding" is overstating it. Correlation between tokens embedded in the weights via training, yes.

Training is a loan word used to describe human learning process. For a reason.

Re: Shall we play a game? My AI nuclear simulation

#95

Post by GuB-42 »

My theory is that LLMs here are put in a situation that matches its training dataset, which is mostly fiction since besides Hiroshima and Nagasaki, nukes have never been launched in anger, and I guess the most reliable sources are highly classified.

So, to a LLM, it is a game, because almost everything in its training data treats it as a game, and it reacts accordingly.

Same idea when we see LLMs acting like AI villains from sci-fi literature. That's because it has been trained with sci-fi literature, and as the auto-completer it is, it will recognize the situation as one of these stories and will continue it accordingly.

LLMs are storytellers, their reasoning is based on words, not on the physical world. Many of the stories they tell are useful, but one must not forget that they are stories, there is no intent behind them.

Re: Shall we play a game? My AI nuclear simulation

#97

Post by Bender »

slibhb wrote: post #72
Bender wrote: post #61

Earlier quoted context omitted.

Words like attention, awareness and care do not apply to computers. At least, not yet. Intelligence and sentience are not applicable to servers. They are just machines with logic states. LLM's are just really cool math formulas with big-data fed into them. Big data is not intelligence. It is a massive data-set sorted, filtered down and interpreted by a language model.

LLMs are intelligent by any reasonable standard. Arguing otherwise is like arguing that chess algorithms aren't good at chess when they easily beat the best humans.

I disagree. LLM's are a language model math formulas that interpret and utilize big-data. Take away the math formulas and we are just back to a massive set of data. Adding to that I would suggest not even the purist forms of data meaning that the data-sets include knowledge from the open and anonymous internet and formulaic tuning from the AI owners and operators.

Re: Shall we play a game? My AI nuclear simulation

#98

Post by chaseadam17 »

Bender wrote: post #29

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those g…

I'd argue we don't even know what "intelligence" or "self-awareness" mean.

Humans are conscious which means we experience things, then we develop preferences for certain experiences, then we develop skills for achieving those preferences.

Without consciousness, what is there to be aware of? And why would intelligence emerge and/or what end would it serve?

Re: Shall we play a game? My AI nuclear simulation

#99

Post by anon84873628 »

dwattttt wrote: post #81
mettamage wrote: post #78

Earlier quoted context omitted.

Hmm saying it’s random-ish is doing it a disservice. I understand it’s a stochastic process but there’s definitely some level of understanding. Not at the level of lived experience but usually an LLM with vision capabilities can call a spade a spade and do something useful with it. And when a verification function shows how they are wrong then they usually come with a better and more informed approach. So I can’t ful…

"understanding" is overstating it. Correlation between tokens embedded in the weights via training, yes.

Feedback loops certainly seem to give them some level of understanding.

Agent reads a skill file about how to use a CLI tool. It tries to use the tool but gets an error about the input format. It tries again with a different format based on the error message, and sees that command succeeded. It compares what worked to what was in the skill file and notes the difference. On future invocations it continues to use the new format.

Is that not "understanding" how to use the tool?

Re: Shall we play a game? My AI nuclear simulation

#100

Post by Chu4eeno »

arjie wrote: post #36

These papers usually have poor stability to prompting and rerunning. It would be nice if we had some kind of meta-evaluation metric where rewriting the prompt conditions or varying the input params could be used to determine how stable a result is. Regardless, it's definitely true that AI agents have different priorities from us. That's what alignment is about anyway.

It's probably because they care more about the headline than figuring anything out: https://github.com/kennethpayne01/project_kahn_public/issues...

So you create leading prompts like that, and re-run until you get a publishable session.

Post reply on HN