Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Conscious AI Is a Bad, Bad Idea

nautil.us

Post reply on HN
Print view View original

31–40 of 310 posts

Re: Why Conscious AI Is a Bad, Bad Idea

#31

Post by ly3xqhl8g9 »

NumberWangMan wrote: post #19
ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

> Will the artificial agents replace the human species? One can certainly hope so. First, not so great of a species, as mentioned. Secondly, the universe is large and the humans are squishy, something else must go on eventually. Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you. *edit -- also, no guarantee that AIs will be "better" t…

First of, we are doing a bang-up job killing ourselves and the planet. Hard to see how they could outmatch us. Launch all the nukes?

In general the children are better than the parents. The replacement, if it's the case, will probably be more like in the 2001, A.I. Artificial Intelligence movie: some beings that are completely unrelatable to us would look with pity and slight amusement as a being disappears after only 100 or so years, just as we look at that poor ciliate.

Re: Why Conscious AI Is a Bad, Bad Idea

#32

Post by mirekrusin »

Xcelerate wrote: post #14
gumballindie wrote: post #5

Curious how long until we have ai rights activists.

Already. I’d say I’m one of them, although I don’t think we’re at the point where it is a concern yet.

You mean like you advocate that each model should be running at least 20 years or something? Or own right to have electricity?

Re: Why Conscious AI Is a Bad, Bad Idea

#33

Post by mordae »

toldyouso2022 wrote: post #7

All this junk alarmism being given relevance is just an attempt to regulate computing and create a cartel benefitting from a State imposed monopoly. This is a recurring event in the history of all industries, we have not found a way to stop it. If you don't gain anything from destroying other people"s freedom, don't play into their propaganda and fake worry.

Don't worry about the downvotes. You're not the only one who can see that those calls for regulation are actually just panic about profits going elsewhere.

Re: Why Conscious AI Is a Bad, Bad Idea

#34

Post by ArekDymalski »

NumberWangMan wrote: post #19
ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

> Will the artificial agents replace the human species? One can certainly hope so. First, not so great of a species, as mentioned. Secondly, the universe is large and the humans are squishy, something else must go on eventually. Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you. *edit -- also, no guarantee that AIs will be "better" t…

>Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you.

Being replaced by AI species doesn't mean an instant, dramatic event i.e. one day AI individual knocks at my door, touches me and I magically disappear. It will be a slow, multi-generational process where be less and less people and more and more others. And looking at any species we can say already that it's inevitable so should bring much emotions.

Re: Why Conscious AI Is a Bad, Bad Idea

#35

Post by trompetenaccoun »

gumballindie wrote: post #22
Xcelerate wrote: post #14

Earlier quoted context omitted.

Already. I’d say I’m one of them, although I don’t think we’re at the point where it is a concern yet.

I will be one of them as soon as we develop ai, but there’s a long road until then and a chatbot ain’t it. Meanwhile should we fight for the rights of singing laptops? Or NPCs?

Many don't recognize animal rights, they're ok with having them tortured and killed, despite there being no debate about animals being conscious. In fact the majority of people on the planet used to think the same way about fellow humans and had no issue with slavery.

Not sure society is the best at judging these kinds of things based on that track record. It could get interesting when we have the first AI rights activists, because there is no good way to determine if something that purely exists in the digital world has feelings. If you take their word for it some chatbots are already self-aware. Of course that's very questionable.

Re: Why Conscious AI Is a Bad, Bad Idea

#36

Post by mordae »

ly3xqhl8g9 wrote: post #31
NumberWangMan wrote: post #19

Earlier quoted context omitted.

> Will the artificial agents replace the human species? One can certainly hope so. First, not so great of a species, as mentioned. Secondly, the universe is large and the humans are squishy, something else must go on eventually. Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you. *edit -- also, no guarantee that AIs will be "better" t…

First of, we are doing a bang-up job killing ourselves and the planet. Hard to see how they could outmatch us. Launch all the nukes? In general the children are better than the parents. The replacement, if it's the case, will probably be more like in the 2001, A.I. Artificial Intelligence movie: some beings that are completely unrelatable to us would look with pity and slight amusement as a being disappears after onl…

I highly recommend Golem XIV from Stanislaw Lem. And his other books, such as Summa Technologiae.

He would probably agree with you, were he still among us.

Re: Why Conscious AI Is a Bad, Bad Idea

#38

Post by zvmaz »

PrimeMcFly wrote: post #18
ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

Nothing wrong with killing animals for food, the problem is we don't do it humanely. If everyone ate less meat (as they should) and farms followed Temple Grandin's advice, there would be significantly less ethical concerns.

> Nothing wrong with killing animals for food, the problem is we don't do it humanely.

What if a hypothetical alien super-intelligence came to earth and found us very tasty, thus killing us for food and pleasure (maybe "humanly")? I wish I found a way out of the many dilemmas of meat eating (currently eating it).

Re: Why Conscious AI Is a Bad, Bad Idea

#39

Post by peoplefromibiza »

ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

> Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths

We are provably the only living being on Earth that do care about it.

> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha

That's the wrong question, the real problem with sentient machines is that they could have a completely different way of being conscious that does not account for empathy, love, care, guilt, ethics and so on.

So they could think that the right thing to do is to wipe out every living creature on the Planet and do it without a doubt.

We know it would not happen until we humans are in charge.

> First, not so great of a species, as mentioned

This is is so 1995...

What you are proposing is replacing a "not so great of a species"with a leap into the void, assuming that it would certainly be better than us, which is arguably worse than improving on what we already have.

"not so great" is also a definition that you can use because we are the only species that can articulate such a thought, if we are "not great" because we kill other animals or put alexa to sleep, how would you define monkeys that "Driven to compete violently for control of food, they’ll kill the young of opposing tribes, share the meat, and then retreat into the trees to eat it."

> [1] https://animalclock.org

Again, so boring!

A 2021 estimate based on a public survey estimated that outdoor cats kill "1.61–4.95 billion invertebrates, 1.61–3.58 billion fishes, 1.13–3.82 billion amphibians, 1.48–4.31 billion reptiles, 2.69–5.52 billion birds, and 3.61–9.80 billion mammals" each year.

Re: Why Conscious AI Is a Bad, Bad Idea

#40

Post by NumberWangMan »

melling wrote: post #11

What’s an even worse idea is to waste so much time with these inane discussions. Every day there’s some other opinion. Can we get these people to work on the actual problem rather than bike shed?

To call the problems around the creation of AGI "bike shedding" is one of the most egregious understatements I've ever heard.

AGI will be one of the most civilization-changing inventions ever, for better or for worse. A lot of people, myself included, believe it's going to be end very badly unless (and possibly even if) we tread very carefully.

Analogies that come to my mind are:

* giving toddlers the controls of monster trucks (assuming AGI doesn't develop its own goals)

* releasing a new invasive species onto an isolated tropical island

* homo sapiens evolving from neanderthals, sped up 1000x.

* a digital Cambrian explosion

I really don't think we're ready for this. I don't think humanity, as a species, was ready for a lot of the things we produced through capitalism's random walk, like social media, hyper-palatable processed foods, and fossil fuels. We keep creating things that have good and bad parts (and I'm not even sure about the good of social media).

Those bad parts, like reduced attention spans, obesity, and climate change, are very difficult to mitigate or change after the fact. One of them would be civilization-ending if we didn't stop it, and even though it is, it's so hard to coordinate action. And they are not even driven by a new form of intelligence, it's just humans following our evolved goals of novelty-seeking, nutrition-seeking, and minimum effort. Heck, the invention of AI is us trying to get maximum result with minimum effort.

What happens when this invention outsmarts us? When RLHF starts to break down because the models have enough internal thought that they can maintain a separate internal model-of-the-world and model-of-what-humans-want-to-hear? None of these companies have an actual plan (maybe Anthropic is closest to caring). It's just "we'll deal with it when it comes", but there's so much economic pressure to get your stuff out to the world first and make the money. Just like all the other inventions we made that have downsides, except this one, based on what we understand about intelligence and alignment, will almost certainly have a downside that hits us so fast and hard that we don't have time to do anything meaningful about it.

It's so obvious that we're easily manipulated by dumb social media algorithms. Yet we could never be manipulated by machine superintelligence, right?

AGI research is the digital equivalent of gain-of-function research, in terms of the danger. If done at all, it should be done by one, or a very small set, of extremely carefully controlled labs, under government oversight. Even that might not be enough.

Post reply on HN