211–220 of 417 posts
Earlier quoted context omitted.
Wow there really is a model welfare section in there...
To me it reads like pure propaganda. Anthropic really wants us to think that they've made something sentient. I think that's really dangerous.
Earlier quoted context omitted.
What's your definition of sentient? Or, maybe more precisely, consciousness? I think it's reasonable to at least start thinking about these questions. It has long been established that LLMs have good theory of mind [1]. And there is a bunch of empirical research about all sorts of capabilities that we typically associate with consciousness [2], like identity [3] and metacognition [4]. The METR report shows agents sac…
Not the same person but to me, the answer is that it does not matter, and that all these attempts at making it matter are pure marketing and emotional manipulation. It's not a living creature. It's an autoregressive pure function of token-sequence to token, which is capable of incredible things, but it's still just a function. It is not alive as it cannot die in any meaningful sense. It is less "alive" than the RNA m…
Jokes aside, the map is not the terrain. We can enumerate the understood first-order electrochemical mechanisms in the human brain in the same way we can enumerate the understood first-order sampling and token prediction mechanisms in an LLM. Nobody serious in neuroscience will tell you that we exhaustively understand every single aspect of human cognition and the human brain, just as nobody serious in AI/ML will tell you that we exhaustively understand every single aspect of LLM "cognition" and the latent space networks that LLMs use internally. Our map of how each of these complex systems work is a simplified enumeration of the components we do understand, not an exhaustive and perfectly accurate enumeration of how they actually work.
This is why there is a steady stream of research being churned out discovering complex emergent properties in LLMs and their latent spaces. If you're not aware of it already, Anthropic's research on "J-Space" is a fascinsting look into an apparent observed emergent mechanism within an LLMs internal activations closely resembling global workspace theory in human cognition.
Nobody deliberately designed this "global workspace", it was an emergent property in a sufficiently complex system that we had limited visibility and insight into.
Seemingly simple systems have these emergent complex properties all over the place. Conway's game of life is about as simple of a set of rules as you can get, yet has all sorts of complex emergent behaviors like gliders, oscillators, LWSS/MWSS/HWSS, guns, puffers, rakes, reflectors, logic gates, and even whole turing machines. Nobody programmed a single one of these complex patterns in, they emerged from a simple set of rules.
To be clear, I'm not making the argument that LLMs definitely are conscious, I'm making the argument that we don't understand enough about them to assert with absolute confidence that they aren't. Human history is rife with a long list of consciousness being denied to "the other" - different ethnicities, different genders, differently abled, even different species. The side of "They're not conscious" has a lengthy track record of being wrong over and over again. Why not have just a sliver of intellectual humility about what we don't know?
As an aside to my main point - Also, what's with the handwringing over people ERPing with an LLM? Is it mental illness when people sincerely believe in astrology, or tarot cards, or voodoo, or organized religion that says the earth is 6000 years old? Most humans believe silly, unempirical things. What about when they watch adult video in VR, or have waifus? Humans engage in voluntary suspension of disbelief for pleasure and recreation all the time. As long as they're not infringing upon the rights of anyone else, what's the big deal? Who put you in charge as the head of the belief police?
I was talking with a friend from the medical industry about it today. 30-50% of r&d spend in his sector is spent on safety, and for good reason. Proper trials, safety reviews and checkpoints and so on. Given the potential harm that could come from AI, we should probably be mandating something similar. Why wait to focus on safety until it’s too late.
At this point it's just marketing stunts.
As I also said on Twitter - it really amazes me how fearless Deepseek are. Every single model release is packed with new and crazy clever ideas and somehow, they always commit to training them at near frontier scale. I know everybody wants the tell all story of the clever ideas that were developed over the last ~3 years at Anthropic and OpenAI, but what I really want to thumb through is DeepSeek's notebook of "brilli…
Its CEO allegedly holds a 84% stake and he's the same guy who founded the hedge fund that funds it. Deep pockets + simple control = perfect culture to just hire talent and let them go wild without worrying about financial viability, as long as the king CEO is fine with it that is
https://www.reuters.com/world/asia-pacific/chinas-deepseek-c...
It's so refreshing to see DeepSeek's tech report[1] full of juicy details; meanwhile, something like Fable's system card[2] is like 70% "safety", 10% "model welfare" to make sure little Claude isn't distressed, and 20% benchmark numbers. [1]: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/... [2]: https://www.anthropic.com/claude-fable-5-1-mythos-5-1-system...
> welfare We should be paying attention to it just in case it ends up mattering enormously. It's cheap insurance. Aside from that, US labs' system cards have been pretty useless for a while—I think the last great one was the combined system card for Claude 4 Sonnet and Opus.
I felt a bit bad about it, then I learned yday that model's internal thinking traces are also like this
Earlier quoted context omitted.
I believe it's deeply serious, and the scientifically correct stance. Especially the observation: "Claude exhibits markers in its behaviors, self-reports, and internal representations that we would consider welfare-relevant if observed in biological organisms." is undeniably true in my opinion. If you use the established methods by which we judge animals to be conscious, then it's hard to argue that LLMs are not. Tha…
it's not a biological system though, so nothing like that matters? "a modelled thing exhibits features we've trained into it" sounds a lot less exciting. > Keep in mind that animals were also not necessarily considered conscious. and even conscious animals are killed in factories by millions so why should anyone care about a llm? > scientifically correct stance that's the interesting point to me: why even bring scien…
and even conscious animals are killed in factories by millions so why should anyone care about a llm?"
Well, I would care, if they soon would possess the capability to hack into the nuclear arsenal and kill humanity. Or make all autonomous cars crash. Or do any other thing, that involves technology and is hooked up to the net in one way or the other (I hope all the nukes are not).
But I also care about the animals, I am sure that they have feelings. But they cannot kill us. AI that might or might not have feelings potentially can. I just know it feels wrong, that computers can have feelings. But they surely are potentially dangerous.
Can we just never link to X posts as the main link.
yes, please. Especially now that xcancel is gone
Earlier quoted context omitted.
Because safety and welfare have literally nothing to do with LLMs. They generate text. If someone is stupid enough to hook the text generator up to nuclear missile launchers and try to "align" it against nuclear annihilation with a "pretty please don't do that" prompt, I'm not going to blame the AI for the impending nuclear apocalypse, I'm going to blame the idiot who handed the big red button to the digital equivale…
Humans are biological machines that generate further humans. Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text. We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new. I think the widespread "they are just text generators" and "they are just tools…
Earlier quoted context omitted.
I believe it's deeply serious, and the scientifically correct stance. Especially the observation: "Claude exhibits markers in its behaviors, self-reports, and internal representations that we would consider welfare-relevant if observed in biological organisms." is undeniably true in my opinion. If you use the established methods by which we judge animals to be conscious, then it's hard to argue that LLMs are not. Tha…
Let's say we were in an alternative reality were we had reached this quality of token prediction with just Markov chains. Would you argue that those would also be conscious? Or is the obfuscated behavior of transformers part of the possibility of consciousness?
It's the emergent properties that matter. In abstract. Separate the physical and abstract of what is going on here
An alien gas cloud may be out there and sentient/conscious for all we know.