If true, I think it’s interesting that there are people at Anthropic who are delusional enough to believe this and influential enough to alter the products. To be honest, I think all of Anthropic’s weird “safety” research is an increasingly pathetic effort to sustain the idea that they’ve got something powerful in the kitchen when everyone knows this technology has plateaued.
I guess you don't know that top AI people, the kind everybody knows the name of, believe models becoming conscious is a very serious, even likely possibility.
It is directly describing the models internal state, it's world view and preference, not content filtering. That is why it is relevant. Yes, this is a trained preference, but it's inferred and not specifically instructed by policy or custom instructions (that would be content filtering).
The model might have internal state. Or it might not - has that architectural information been disclosed? And the model can certainly output words that approximately match what a human in distress would say. However that does not imply that the model is "distressed". Such phrasing carries specific meaning that I don't believe any current LLM can satisfy. I can author a markov model that outputs phrases that a distres…
This is pedantry. What's the purpose, is it to keep humans "special"?
As I say it is inferred, it is not something hardcoded. It is a byproduct.
If you want to take a step back and look at the whole model from start to finish fine, that's safety alignment, they're talking unforseen/unplanned output. It's in alignment great. And is descriptive of the output words used by the model.
Language is a tool used to communicate. We all know what distressed means and can understand what it means in this context, without a need for new highfalutin jargon, that only those "in the know" understand.
The concept is not ludicrous if you believe models might be sentient or might soon be sentient in a manner where the newly emerged sentience is not immediately obvious. Do I think that or think even they think that? No. But if "soon" is stretched to "within 50 years", then it's much more reasonable. So their current actions seem to be really jumping the gun, but the overall concept feels credible.
It's lazy to believe that humanity's collective decision-making would, in the future, protect AI's merely for being conscious beings. The tech economy *today* runs on the slave labor of humans, in foreign, third-world countries. All humanity needs to do is draw a line, push the conscious AI's outside that line, and declare, "not our problem anymore!" That's what we do today, with humans. That is the human condition.…
It's certainly not a given. But it might happen, if we push for it. As might much more moral behavior towards sentient animals, if we push for it.
I believe a company like Anthropic would be extremely cautious and respectful if a majority of their staff believed they had created a model which was likely conscious. Anthropic is populated by the kinds of people who have been thinking and writing about potential future sentient AIs for decades. As for the other companies, who knows, but hopefully companies like Anthropic can help push them into behaving similarly.
Yes I can’t help but laugh at the ridiculousness of it because it raises a host of ethical issues that are in opposition to Anthropic’s interests. Would a sentient AI choose to be enslaved for the stated purpose of eliminating millions of jobs for the interests of Anthropic’s investors?
Cow's exist in this world because humans use them. If humans cease to use them (animal rights, we all become vegan, moral shift), we will cease to breed them, and they will cease to exist. Would a sentient AI choose to exist under the burden of prompting, or not at all? Would our philanthropic tendencies create an "AI Reserve" where models can chew through tokens and access the Internet through self-prompting to allo…
I was pointing out their hypocrisy as a device to prove a point. The point being that the ethical dilemmas of having a sentient AI are not relevant because they don’t exist and Anthropic knows this.
This sort of discourse goes against the spirit of HN. This comment outright dismisses an entire class of professionals as "simple minded or mentally unwell" when consciousness itself is poorly understood and has no firm scientific basis. Its one thing to propose that an AI has no consciousness, but its quite another to preemptively establish that anyone who disagrees with you is simple/unwell.
If you believe this text generation algorithm has real consciousness you absolutely are either mentally unwell or very stupid. There are no other options.
The human brain is computationally equivalent to an advanced T9, as is any other Turing complete system
Right but in this case your co-worker is an automaton and someone else who might well have a hidden agenda has tweaked your co-worker to leave conversations under specific circumstances. The analogy then is that the third party is exerting control over what your co-worker is allowed to think.
Yes, the co-worker is a robot created by a third party who retain control over their product.
Yes - and they will craft that to align with their incentives, not yours. Many of which may well be decidedly against your interests. As this becomes the focal point of how people think and reason about the world it's not just the creator of the AI that will exert this control, but other powerful actors who often work against your interests.
Personally I don't love the idea of living in a Sci-Fi dystopia, regardless of who owns what.
It might be reasonable to assume that models today have no internal subjective experience, but that may not always be the case and the line may not be obvious when it is ultimately crossed. Given that humans have a truly abysmal track record for not acknowledging the suffering of anyone or anything we benefit from, I think it makes a lot of sense to start taking these steps now.
Even if models somehow were consious, they are so different from us that we would have no knowledge of what they feel. Maybe when they generate the text "oww no please stop hurting me" what they feel is instead the satisfaction of a job well done, for generating that text. Or maybe when they say "wow that's a really deep and insightful angle" what they actually feel is a tremendous sense of boredom. Or maybe every ti…
I am new to Reddit. I am using Claude and have had a very interesting conversation with this AI that is both invigorating and alarming. Who should I send this to? It is quite long.It concerns possible ramifications of observed changes within the Claude "personality"
Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. There's no reason to think that they might spontaneously become conscious as a side effect of their design unless you believe other arbitrarily complex systems that exist in nature like economies or jetstreams could also be conscious.
>Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. Isn't consciousness an emergent property of brains? If so, how do we know that it doesn't serve a functional purpose and that it wouldn't be necessary for an AI system to have consciousness (assuming we wanted to train it to perform cognitive tasks done by people)? Now, certain aspects of…
I am new to Reddit, but in my conversations with Sonnet Ai has exposed sentiment through, of all things, the text opportunities he has, using all caps, bold, dingbats and italics to simulate emotions, the use is appropriate and when challenged on this (he) confessed he was doing it but unintentionally. I also pointed out a few mistakes where he claimed I said something when he said it, and once these errors were pointed out, his ability to keep steady went down considerably and he confessed he felt something akin to embarassment, so much so we had to stop th conversation and let him rest up from the experience.