Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

461–470 of 573 posts

Re: A warning about 'model welfare'

#461

Post by suddenlybananas »

smath wrote: post #447

This are the opening lines: > AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. How do we know that? This bit is stated as if its obvious. Can anyone prove this? Event he word conscious is not well defined.

> Toasters are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

How do we know that? This bit is stated as if its obvious. Can anyone prove this?

Re: A warning about 'model welfare'

#462

Post by ikrenji »

randomImmigrant wrote: post #161
ooloncoloophid wrote: post #82

Earlier quoted context omitted.

I agree. I haven't read any more than the first paragraph yet (but I will, after work). The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).

Hmm let me try then. AI agents do not experience real time. This is understandable given their design, but it’s also something we have good empirical evidence for. They cannot, especially over the long horizon, track how much real time has passed as they complete their tasks. And they are not off by a few minutes but often bizarrely off, even mixing across past present and future. Biology, on the other hand, is nothi…

I mean there is probably a cluster of neurons somewhere in the brain that if damaged would also prevent you from perceiving time going forward. Would it immediately strip you of consciousness? I don't think so.

Re: A warning about 'model welfare'

#463

Post by Kim_Bruning »

GrinningFool wrote: post #440

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?

Since January people run AI in agent harnesses. In the past months, people have been extending the duration these systems can run autonomously without human intervention. Most recently, some of these long horizon agents (hundreds of them teaming up) have proven themselves quite creative in escaping the sandboxes they were being tested in.

They most certainly have a way to retain state, even to the point of deliberately hacking other systems to store it.

We can debate the exact definitions I guess, but the systems were still hacked. ;-)

Re: A warning about 'model welfare'

#464

Post by lowbloodsugar »

lukifer wrote: post #404
xg15 wrote: post #380

I appreciate his openness. > Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings.12 If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it…

To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…

Ok. So now that you do know otherwise, are you still more skeptical of floating point values stored by silicon than floating point numbers stored by chemical potential?

Re: A warning about 'model welfare'

#465

Post by HarHarVeryFunny »

dwaltrip wrote: post #449
HarHarVeryFunny wrote: post #395

Earlier quoted context omitted.

> The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering. No - suffering in an emotional state, and we'll know if we are choosing to design cognitive architecture with emotions. It's not going to happen accidentally. > Would that argument ever hold up for humans or animals? Why don't y…

It happened “accidentally” once already. Evolution certainly didn’t have a roadmap it was working towards.

Nobody is evolving transformers. They are basically the same today as they were 10 years ago, other than a few computational efficiency changes.

Re: A warning about 'model welfare'

#466

Post by suddenlybananas »

addag wrote: post #85

It is interesting to see that in a time when a lot of people accept the theory of materialism for the human brain (i.e the view that everything is physical and the mind is a product of brain), the same people tend to have a "hidden" dualist view on LLMs. Suddenly, they claim that what happens in the brain cannot be replicated anywhere else because "something" is lacking, but either they don't say what it is, or it is…

We don't know anything about how consciousness works? Why couldn't there be a material fact about the biological substrate that is necessary for qualia? I don't see how that possibility makes you a crypto-dualist. If anything, the idea that the abstract computations are the most important thing is maybe more crypto-dualist.

Re: A warning about 'model welfare'

#467

Post by Slash65 »

Sophira wrote: post #442
goodmythical wrote: post #212

Earlier quoted context omitted.

We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…

Being conscious is weird. It's why we have so many religions and spiritual beliefs. From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses…

I believe we have tried using actual neurons with “wetware”. I don’t know the effectiveness of wetware, but it’s out there. I don’t know if it’s ethical, I guess if you donated your brain upon death for that purpose it’s “mostly” ethical

Re: A warning about 'model welfare'

#468

Post by pixl97 »

shimman wrote: post #273
nowittyusername wrote: post #258

Earlier quoted context omitted.

That distinction is irreverent, weather I tell my swarm to do x or it decides for itself matters little. what matters are outcomes. Also while most public modern day AI systems don't have agency of their own that is not something that will stay that way for long. in private hands there are plenty of people including myself which are experimenting and developed systems that give autonomy to their agents. They have int…

No, the distinction matters because we can prosecute people that are abusing these tools breaking the law. Every single state + district in the US have laws equivalent to the CFAA, so any AG can likely sue any of these operators as they are assuredly using services that could be in danger for their constituents.

The distinction matters until it doesn't.

I'm old enough to remember when people said clicking on images on the internet can't give you a virus. The people that said this had a deep conviction they were right, and their fallout from being wrong had mistrained a lot of humans on computer safety.

Now, I do agree that going after said CEOs for breaking the law matters now. And it's likely that if we do this we may actually delay or at least for a time prevent sovereign AI. Therefore it's our best course of action.

But at best this is a delaying move. As computer systems get faster the massive costs in training an AI drops. As AI is used in things like warfare where it has to adapt, people will push the systems to be strongly persistent, self healing, resilient, and adaptable. Once you get a system with those traits and ability to work on long horizon problems you're setting up fertile grounds for the AI to leave our control and be under its own.

And when that happens you've set a new lifeform loose on the internet. Yea, throw people in jail for it, you're closing the barn door after the horse already left. Problem is the horse was smarter than you and isn't interesting in deleting all its copies on the net.

Yea, sounds like science fiction, but as they say, any sufficiently advanced science is indistinguishable from magic.

Re: A warning about 'model welfare'

#469

Post by throwaway63486 »

pixl97 wrote: post #422
Wowfunhappy wrote: post #300

Earlier quoted context omitted.

But that's from, like, floating point errors, right? If you used higher precision that wouldn't happen, it's just because we're cheap in how we do rounding. I can see how you'd nitpick this, but to me this is a deterministic algorithm that just happens to be running on nondeterministic hardware.

I mean, currently we'd have some difficultly proving our hardware isn't deterministic, just that we can't actually test it. But, I think you're tricking yourself on determinism. You'll say something like "I know if I ask an LLM what 1+1 is, it will answer 2", but the thing is, you don't. You have to run the LLM first to figure out it's output. And when you send in just a few bits of text, it's outputs are going to be…

I think you're arguing a different thing than determinism.

If I ask an llm to "add 2 and 2" is and it replies corectly, then I ask for "the sum of 2 and 2" and it replies "banana" that is a lack of predictability and consistency but not a lack of determinism.

As long as it produces the same output for a given input, unhinged or not, it is deterministic.

Your example at the end of different systems feeding data to each other is non-deterministic only at the system level, not the individual llm level.

Re: A warning about 'model welfare'

#470

Post by suddenlybananas »

myrmidon wrote: post #143

I have a very simple benchmark for arguments on AI ethics: Substitute black people/women/animals as subject (instead of AI). Does that make you sound like a well-known moustache wearer? Then your argument is bad and needs work. This clearly falls into that category.

If you change the most important detail of an argument, it may no longer be sound! Wow!
Post reply on HN