Lot of people trying to psychoanalyze why a statistical machine will include something in its training set with a nonzero probability... It includes this option because it will eventually include all options it is trained on if given enough time. Given all the historical, fictional, academic, and sensational content on the topic, it is not surprising it emerges so readily.
Shall we play a game? My AI nuclear simulation
191–200 of 213 posts
Re: Shall we play a game? My AI nuclear simulation
#192[flagged]
Re: Shall we play a game? My AI nuclear simulation
#193Earlier quoted context omitted.
I'd argue we don't even know what "intelligence" or "self-awareness" mean. Humans are conscious which means we experience things, then we develop preferences for certain experiences, then we develop skills for achieving those preferences. Without consciousness, what is there to be aware of? And why would intelligence emerge and/or what end would it serve?
Intelligence is the ability to have an internal world model then run simulations on that model to choose an optimal course of action. This is true for humans down to flies. Most of what humans do is still the boring innate stuff; it's just that fancy abstract things like "skydiving" get the most attention. Clearly other animals have "phenomenological experience" i.e. consciousness / qualia without being as intelligen…
How would you determine “an optimal course of action” without consciousness?
Eg we optimize for survival but without the ability to experience things, why survive?
Why would that instinct or any other preference for a course of action emerge?
Re: Shall we play a game? My AI nuclear simulation
#194Earlier quoted context omitted.
The literally churn out random garbage and are trained over time for that garbage to look more and more like an acceptable outcome to humans. It’s training monkeys at typewriters through reinforcement.
> trained over time So not random. > acceptable outcome to humans And not garbage. It’s real weird to see people argue that LLM output is no different than random gibberish and then handwave over the fact that it’s clearly not with terms like “training”, as if a steam of random garbage is trainable.
Hey dpark.
I quite literally created and productized predictive linguistics and behavioral vectors at Google.
If you had stopped to consider what I explained; you’d understand that it’s the process of turning random garbage into increasingly acceptable outputs.
Ie training the monkeys.
The insight you are missing is the rule of networked scale. It turns out that any reactive node scaled enough can form sophisticated predictive system given reward over a training topography, even if it starts out at garbage or is literally made of monkeys.
So it is garbage. And you can turn garbage into semi-intelligence.
Re: Shall we play a game? My AI nuclear simulation
#195Earlier quoted context omitted.
Intelligence is the ability to have an internal world model then run simulations on that model to choose an optimal course of action. This is true for humans down to flies. Most of what humans do is still the boring innate stuff; it's just that fancy abstract things like "skydiving" get the most attention. Clearly other animals have "phenomenological experience" i.e. consciousness / qualia without being as intelligen…
How would you determine “an optimal course of action” without consciousness? Eg we optimize for survival but without the ability to experience things, why survive? Why would that instinct or any other preference for a course of action emerge?
Well that is the "hard problem", isn't it :-)
But your body is just a robot suit for replicating nucleic acids and you can follow an unbroken chain of those chemical reactions all the way to the first microbes. Which were still "optimizing for survival" without any intelligence beyond following chemical gradients.
If you say that means even microbes have some amount of qualia then I can't dispute it. Pure physicalism may necessitate panpsychism of some form.
Re: Shall we play a game? My AI nuclear simulation
#196Earlier quoted context omitted.
Who are these "scholars" exactly? The only reference I could find is Jim Mattis, and the context was very specific when he said that. Furthermore, this is a "what if" scenario since tactical nukes have never been used. Of course it would make escalation likely during an open conflict, so what? Doesn't change the fact that there is a material difference between a tactical nuke and a strategic one.
I don't know what to tell you. You clearly haven't studied International Affairs, or at least read the scholarly literature. Even some cursory research through Wikipedia citations will bring this up. But in any case, here are some freebies: https://armscontrolcenter.org/why-tactical-nuclear-weapons-a... https://www.armscontrolwonk.com/archive/403540/brodies-weake... If you have a real interest in this area, a subscri…
First link was written by an intern, let's be serious. The second seems to have more legitimacy, ok, so we have this one guy, but the article is very weak. He doubts that tactical nuclear weapons would help control escalation. OK. I don't think that's the center of the argument here. I also think using tactical nukes would lead to escalation, so what? Doesn't change the fact that tactical and strategic nukes are different things.
Re: Shall we play a game? My AI nuclear simulation
#197Earlier quoted context omitted.
Who are these "scholars" exactly? The only reference I could find is Jim Mattis, and the context was very specific when he said that. Furthermore, this is a "what if" scenario since tactical nukes have never been used. Of course it would make escalation likely during an open conflict, so what? Doesn't change the fact that there is a material difference between a tactical nuke and a strategic one.
> tactical nukes have never been used. Two tactical nukes have been used, albeit against strategic (civilian, industrial, logistical) targets.
[flagged]
Re: Shall we play a game? My AI nuclear simulation
#198The most interesting takeaway for me is the three very distinct personalities. Three models all based on the same tech, trained in the same manner, trained by three groups of people with similar ideological outlooks, and the result is three very different AIs. The military basically wants an oracle. Feed the AI the situation, get the best answer out. But if the AIs are as diverse and opinionated as humans, it is deba…
They all have conditioning prompts that precede your input; presumably, most of the detected "personality" comes from the differences in these inputs.
My point is more-or-less orthogonal to why it happens. The military, and honestly, a lot of people, want AI to just give the answer. If it is highly dependent on a prompt, or the follow-on training, and the AI could be passive or friendly or aggressive or hostile or all those other wonderful attributes of individual humans and there's no sort of AI convergence on "correct" answers, then they aren't going to be able to fulfill that "oracle" role that so many people are looking for.
Re: Shall we play a game? My AI nuclear simulation
#199Earlier quoted context omitted.
How would you determine “an optimal course of action” without consciousness? Eg we optimize for survival but without the ability to experience things, why survive? Why would that instinct or any other preference for a course of action emerge?
Well that is the "hard problem", isn't it :-) But your body is just a robot suit for replicating nucleic acids and you can follow an unbroken chain of those chemical reactions all the way to the first microbes. Which were still "optimizing for survival" without any intelligence beyond following chemical gradients. If you say that means even microbes have some amount of qualia then I can't dispute it. Pure physicalism…
Haha yeah, we're in "who knows" territory. It could just be a foundational truth that all life has an innate unconscious will to survive.
However, it makes more sense to me that consciousness precedes the will to survive. And that doesn't mean the way a microbe experiences life is the same as us. E.g. the fact that we have so much control over our environment might mean we have a much more sensitive and reactive way of engaging with consciousness, whereas maybe a microbe is like an enlightened monk, just chillin because it can't do much about what it experiences anyways.
Re: Shall we play a game? My AI nuclear simulation
#200Earlier quoted context omitted.
Interesting. The reasoning models were super weird and robotic. They toned that down a bit in GPT-5.x, especially the later ones. I always assumed the strange style was an artefact of the RLVR.
I think they've brought back a "personality" of sorts to ChatGPT 5.x. I've caught it more than once explaining something to me and saying "In my personal opinion", or "I personally enjoy the most". Which is always jarring, it doesn't "personally" or "enjoy" anything. We could be discussing videogames and it tells me which games "it personally enjoys the most". Bizarre.
[deleted]