He certainly has the confidence a ceo needs. But none of the humility, and seemingly not enough of the humanity he professes to put first.
A warning about 'model welfare'
451–460 of 573 posts
Re: A warning about 'model welfare'
#452Earlier quoted context omitted.
> The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering. No - suffering in an emotional state, and we'll know if we are choosing to design cognitive architecture with emotions. It's not going to happen accidentally. > Would that argument ever hold up for humans or animals? Why don't y…
> It's not going to happen accidentally. https://transformer-circuits.pub/2026/emotions/index.html Whether these are like "our" emotions is hard to say. What we _can_ say is that they are emotion-shaped, we didn't design them, and they happened accidentally. Modern AI is grown, not meticulously designed, and we cannot say with any certainty what the resulting mechanistic properties are.
An LLM will learn anything that helps it predict, including the emotional state of the writer - that is expected.
If you give an LLM the move sequence of a half-played chess game and ask it to continue as white or black, then it has learnt enough to model the ELO rating of both players and will continue playing at that level. It is not playing to win - it is doing what you expect and predicting as well as it can - it predicts the 1500 ELO player will keep playing at that level, and generates moves accordingly.
An LLM appearing to exhibit an emotion (if we anthropomorphize it and read emotion into it's output) is just predicting as well as it can - if the context calls for sad output, they you'd expect to get sad output and will necessarily find that "we're predicting sadness" detector somewhere internally.
Transformers are the same as they ever were from 10 years ago, other than minor efficiency tweaks like MOE and different attention mechanisms. Training is getting more and more complex, resulting in better and better cargo cult reasoning etc, but the architecture remains the same.
Re: A warning about 'model welfare'
#453Earlier quoted context omitted.
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…
Kobiyashi Maru is nonsense test from simple sci-fi series that makes sense only in uber-simplified universe of star trek.
Yes, that’s why this is serious. The closest analogy is preposterous fiction.
Re: A warning about 'model welfare'
#454> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?
how do you know you're anything more than a set of weights either? that's semi rhetorical, the philosophy aspect would be less relevant if an autonomous death drone shot you in the face purely for the lols
Re: A warning about 'model welfare'
#455It is interesting to see that in a time when a lot of people accept the theory of materialism for the human brain (i.e the view that everything is physical and the mind is a product of brain), the same people tend to have a "hidden" dualist view on LLMs. Suddenly, they claim that what happens in the brain cannot be replicated anywhere else because "something" is lacking, but either they don't say what it is, or it is…
There are chemical reactions that LLMs lack because they do NOT have a biology. They are just mathematical weights. Now, let’s say we can truly make sentient AI in the future but it does not have the biology that we do. Then it would just be a totally literal, emotion lacking, sentient machine. But then what is sentience? Dogs are sentient to a certain extent and so are dolphins…
Why can’t there exist something analogous to emotions outside of carbon-based biological processes?
Re: A warning about 'model welfare'
#456Earlier quoted context omitted.
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…
"I’m broadly skeptical that floating-point operations can be conscious" Why?
Because it is alien to what is known.
Incidentally, we should expect similar bewilderment if we ever come across aliens
Re: A warning about 'model welfare'
#457They're intelligent and not conscious. This is not hard to understand, yet people insist the artificial in artificial intelligence must be directly linked with consciousness because we've observed intelligence only in life. Unlink the concept of intelligence from life and you land on AI and LLMs.
Re: A warning about 'model welfare'
#458David Brin has a wonderful novel on this topic, Existence, where he discusses several different AI and human systemic evolutions. Wonderful book for our current time.
Re: A warning about 'model welfare'
#459Earlier quoted context omitted.
This has nothing to do with the statement above- it's just a consequence of the fact that by resetting the context you reset all memory of the llm. The same would happen if you could reset entirely a human brain to some initial state- and it would mean nothing regarding its consciousness or ability to feel.
But that inability to reset human brains might be a critical ingredient to consciousness. We just don't know.
And what of amnesia and dementia?
Re: A warning about 'model welfare'
#460This are the opening lines: > AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. How do we know that? This bit is stated as if its obvious. Can anyone prove this? Event he word conscious is not well defined.
Can you prove they are? I think “AIs are conscious” is an extraordinary claim requiring extraordinary evidence. The burden is on the claimant to provide evidence it is conscious, and I suspect any evidence you can provide has alternative explanations. The majority of claims I’ve seen like this amount to “how do you know humans are conscious in a way that’s distinct from a next token predictor.” I do not find those claims compelling.