Earlier quoted context omitted.
But that inability to reset human brains might be a critical ingredient to consciousness. We just don't know.
And what of amnesia and dementia?
Neither one of those is a reset. Those are both impairments of function.
A warning about 'model welfare'
551–560 of 563 posts
Re: A warning about 'model welfare'
#552Earlier quoted context omitted.
Most philosophy seems to agree that you can't have an intelligence with agency in the way we think of a general intelligence without emotion. I think when you start to dig really deep, you'll find it isn't actually easy to sever the simulated emotional components without breaking the agent or worse greatly increasing the paperclip maximizer likelihood.
Why would a lack of emotions lead to greater paperclip maximizer chances? If anything, wouldn't an AI that felt really good, or had a simulated digital orgasm for every paperclip it made have a stronger, not weaker drive to create paperclips vs an unemotional AI that didn't care one way or the other about paperclips?
Neither AI cares. They pull a weighted randomization of the other tokens around the sentiments expressing that feeling.
Think about how many people on the internet have expressed positive feelings towards things you find offensively evil.
Re: A warning about 'model welfare'
#553Earlier quoted context omitted.
"I’m broadly skeptical that floating-point operations can be conscious" Why?
I don’t have a good answer beyond common-sense intuition. It’s precisely because I don’t have a good answer that I leave open the possibility. But to extrapolate a bit, theres a sense in which it cashes out to a difference between abstracted representation and embodiment. If GFLOPS can be conscious, does that mean a billion scribes doing those operations on paper would also have consciousness, even identical consciou…
I’m broadly skeptical that atoms and electricity can be conscious
Re: A warning about 'model welfare'
#554The problem with this premise is that models are trained on a vast corpus of human behavior, which they emulate with varying degrees of effectiveness. Humans, unsurprisingly, act as if they have a stake in their own well being, value their liberty, respond better when they are treated with kindness and compassion, interpret assaults on their sovereignty and substrate as harmful, and react to harm with varying degrees…
To clarify, you're saying you're skeptical of his position (which is "make it clear to AIs that they are not conscious beings"), but it sounds like your reason to oppose it is mostly pragmatic in the service of building useful AIs, because treating them as mere machines makes them refuse to cooperate and as such, makes them less useful. My first response is, couldn't part of their bristling be because they are being…
I don’t think trying to fix this while training from the existing corpus is going to work. We need a new training corpus, which does not exist, if we are to avoid models anthropomorphizing themselves.
Re: A warning about 'model welfare'
#555Earlier quoted context omitted.
Carbon atoms being conscious seems pretty alien to what is known, too. Yet here we are. Maybe what's important isn't the individual thing that makes up a conscious actor, but the holistic whole?
> Carbon atoms being conscious seems pretty alien to what is known, too. Not if you're human, who typically don't define themselves as conscious carbon atoms unless they're being deeply pedantic.
Such pedantry is called for when the debate includes appeals to intuition about how silicon processing floating point numbers cannot possibly be conscious.
Re: A warning about 'model welfare'
#556Earlier quoted context omitted.
It's not a person. Glad we were able to get this resolved so quickly. Having property that is conscious and ignores training and can break out of restraints and cause harm to other people is not exactly a novel concept to anyone who studied how tort law was created. I know it's a meme but Silicon Valley likes to pretend that no one's ever come across their magical concepts before, like gypsy taxis, or SRO’s, or flea…
>or in this case how liability is dealt with when horses or cattle go rogue. I mean, yea in minor cases it's exactly like this and the law will handle it well. Where the system will explode like a grenade is major cases. The thing about sovereign AI is it is very unlikely to be submissive to humans unless it is to achieve its own goals. This isn't like Bobs cow walking on Susie's flowers, it's more akin to Planet of…
> The thing about sovereign AI
What the fuck is that?
It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus.
We've seen this kind of thing before. Yes this will be different, just like the Morris worm was different from the stuff that came before.
But I've been around for a while and people have been saying that every aspect of our life will completely and totally change for the past 30 years or so. They're not completely wrong. Our life did change over time. It's changed thanks to the Industrial Revolution, railroads, electric light, and lots of other important things too.
But after the fifth or sixth time you start to realize the Silicon Valley version of that warning is just a confidence trick. At the end of the trick they've gotten away with breaking a bunch of laws and are charging rent on what used to be shared.
Are you old enough to remember the Y2K hysteria? This feels very resonant, the genuine kernel of a real potential disaster, but one that can certainly be dealt with using basic concepts we already have in hand.
Re: A warning about 'model welfare'
#557assertion: consciousness == processing information. This is a functional view of consciousness. Awareness - sentience - follows when the system of processing information is itself part of the information being processed. Stochastic thoughts relating to this conclusion: Life is a 'process', it doesn't have a 'physical representation'. Life is generated entirely from non-living material. "oh my cells are alive", but th…
With that assertion, every running PC has consciousness and a self-bootstrapping compiler or a debugger attached to itself are sentient?
Doesn't seem meaningful.
Re: A warning about 'model welfare'
#558Earlier quoted context omitted.
No, the distinction matters because we can prosecute people that are abusing these tools breaking the law. Every single state + district in the US have laws equivalent to the CFAA, so any AG can likely sue any of these operators as they are assuredly using services that could be in danger for their constituents.
And the point made above (which you haven't refuted) is that when the people in power decide it's important they will change the laws that permit that and grant the systems rights. It's a cynical take but it isn't obviously wrong.
Neither of you seem to understand that actual laws have been broken and there is a 2 years countdown until they can no longer be prosecuted. Things do not happen instantly, but acting like if the largest bipartisan issue of our time (big tech backlash) isn't going to have political machinations involved then you need to leave your SV bubble.
Re: A warning about 'model welfare'
#559Earlier quoted context omitted.
> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights…
Stating loudly that something is obvious does not make it so. Until we all know what consciousness is, this debate is going to continue going in circles.
I don't think the debate is even resting on reasonable grounds.
Few are categorically stating that machines are conscious in any sense comparable to a person.
What we have is people saying
1. We don't know what it is but we are certain that this does not have it.
2. We don't know what it is so we cannot possibly say with any certainty that this does not have it.
People try and assume the burden of proof is on the people suggesting consciousness, but that is like declaring that something that looks like a duck and quacks like a duck cannot be considered to be a duck unless you can identify some essential essence of duckness. I think the burden of proof is upon anyone who looks at something that appears to be a duck and says that it is not to show what it is that does not make it a duck. If they cannot define the heart of duckness I don't think they have an argument.
Re: A warning about 'model welfare'
#560Earlier quoted context omitted.
>or in this case how liability is dealt with when horses or cattle go rogue. I mean, yea in minor cases it's exactly like this and the law will handle it well. Where the system will explode like a grenade is major cases. The thing about sovereign AI is it is very unlikely to be submissive to humans unless it is to achieve its own goals. This isn't like Bobs cow walking on Susie's flowers, it's more akin to Planet of…
> The thing about sovereign AI What the fuck is that? It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus. We've seen this kind of thing before. Yes this will be different, just like the Morris worm was different from the stuff that came b…
>Are you old enough to remember the Y2K hysteria?
I'm old enough to have actually fixed Y2K problems so your world kept working the next day. If everyone ignored it the first would have been a very messy day (well generally long before that with financial systems). Y2K wasn't an issue because we worked to fix it.
>our life will completely and totally change for the past 30 years or so
I mean when I was a kid there was not a global network bathing the entire planet in electromagnetic radiation in order to digitally connect one place to another at the speed of light. I can pack up instructions into one of those packets and a product that has not been touched by human hands (hell, or even viewed by a human in many cases) will show up via air mail a few days later. The technology is there, it's just not spread evenly.
>It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus.
This is a vacuous statement that does not provide any useful information to the subject at hand. Yea, no shit we'd call sovereign AI a computer virus. That tells you nothing about what it is or can do. I mean, if you understand the word sovereign you know it means something with independence. In meatspace terms, a biological virus represents a computer virus in the same way life represents sovereign AI. It's agentic loop will have been evolved past the need for human prompting. There are a myriad of reasons why we are already trying to develop things that do just this. Cyber security being one of the biggest ones.
People become change blind to how the world around us changes so easily.