Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There's No Fire Alarm for Artificial General Intelligence

intelligence.org

Post reply on HN
Print view View original

41–50 of 209 posts

Re: There's No Fire Alarm for Artificial General Intelligence

#41

Post by edanm »

Santosh83 wrote: post #31

Why do we fear other intelligences anyway? Isn't that just a sign of our own immaturity? Maybe we need to evolve more before we start think about creating AGI...

Mostly because other intelligences might have more ability than us to achieve their goals, and have different goals. "Intelligence does not imply goals" is the thing to keep in mind here.

E.g. what if more intelligent aliens truly believed that the only purpose in the world is proving more mathematical theorems? And decided to turn all of the planet into a giant math-proving machine? Destroying all the planet, all the animals, all the humans, all the art, whatever, all to prove more maths?

I love maths, but I'd consider that a pretty bad outcome. And there's no reason that I've ever seen to think that more intelligence implies anything about goals.

Re: There's No Fire Alarm for Artificial General Intelligence

#42

Post by IanCal »

noja wrote: post #32
alrs wrote: post #24

I was on an elevator full of people, and between two floors something went wrong and we went into free-fall for part of a second. We made it to the next floor, the door opened, my fellow passengers were content to stay in the elevator. I turned, said "My plan is to not die in an elevator today" and got off. What is wrong with people?

Did you ever ride in the elevator again?

It's reasonable to think that a randomly selected elevator is safe enough to ride, while also thinking the one you've just seen have some kind of fault is not.

Re: There's No Fire Alarm for Artificial General Intelligence

#43

Post by edanm »

backpropaganda wrote: post #22
DuskStar wrote: post #16

Earlier quoted context omitted.

Oh, of course! The article itself puts some effort into repeatedly stating the fact that people are saying 50 years does not in any way imply it will actually be 2 years and it might well be 500 .

This is a useless claim though. There are an infinite number of things that could happen that would be very bad for Earth that could happen anytime between 2 to 1000 years from now. We're bad at forecasting ALL of them. We can't use this indeterminacy to prove we should be working on X when the same is applicable to another thing Y.

Well we can decide what seems more or less likely. I mean, yes, an asteroid could impact the earth and destroy all life on it. But we have some guesses as to the probability that that happens.

Clearly, by itself, the world will most likely not kill off humanity, since it hasn't happened in the thousands/millions years we've been around. The one big thing that is changing is humanity itself and the technology we're making - that's the X factor, that's what statistically speaking has a chance of actually wiping us out.

Many of the people concerned about AGI are also concerned about e.g. manufactured viruses and other forms of technology.

Re: There's No Fire Alarm for Artificial General Intelligence

#44

Post by edanm »

nurettin wrote: post #27
simonh wrote: post #13

Earlier quoted context omitted.

A conscious system might be unintelligent, while it’s possible to imagine a highly intelligent system with no consciousness. They’re just different things. Also pattern detection is often raised in the way you just did, but it’s realy a distraction. Pattern detection just helps recognise things, it’s not inherently related to the ability to reason about things. So you need both, but they are not the same thing either…

But where does consciousness arise? Is the ability to reason about things independent of this concept?

Short answer - we don't know, but probably?

We don't really know what consciousness is or how it happens. We believe that it's possible to be highly intelligent without being conscious. I mean, I really hope that's true, personally, since we would hope to one day make an AGI that will carry out human desires, and we'd hope we weren't making a conscious entity which would be the equivalent of a slave.

Re: There's No Fire Alarm for Artificial General Intelligence

#45

Post by icebraining »

alrs wrote: post #24

I was on an elevator full of people, and between two floors something went wrong and we went into free-fall for part of a second. We made it to the next floor, the door opened, my fellow passengers were content to stay in the elevator. I turned, said "My plan is to not die in an elevator today" and got off. What is wrong with people?

Elevators have pretty good safety mechanisms. Even cutting the cable wouldn't have killed anyone.

I'd probably leave too, but just because I wouldn't want to get stuck inside it if it stopped in the middle of two floors, especially since it was full. But for fear of death? Nah.

From Wikipedia: "In fact, prior to the September 11th terrorist attacks, the only known free-fall incident in a modern cable-borne elevator happened in 1945 when a B-25 bomber struck the Empire State Building in fog, severing the cables of an elevator cab, which fell from the 75th floor all the way to the bottom of the building, seriously injuring (though not killing) the sole occupant — the elevator operator. (...) In Thailand, in November 2012, a woman was killed in free falling elevator, in what was reported as the "first legally recognised death caused by a falling lift".

That's a pretty good safety record. Certainly much better than stairs.

Re: There's No Fire Alarm for Artificial General Intelligence

#46

Post by kryptiskt »

IanCal wrote: post #42
noja wrote: post #32

Earlier quoted context omitted.

Did you ever ride in the elevator again?

It's reasonable to think that a randomly selected elevator is safe enough to ride, while also thinking the one you've just seen have some kind of fault is not.

But why would an elevator that had demonstrated that it had a working safety brake like that be more unsafe than any other elevator?

I trust in Elisha Otis.

Re: There's No Fire Alarm for Artificial General Intelligence

#47

Post by alrs »

noja wrote: post #32
alrs wrote: post #24

I was on an elevator full of people, and between two floors something went wrong and we went into free-fall for part of a second. We made it to the next floor, the door opened, my fellow passengers were content to stay in the elevator. I turned, said "My plan is to not die in an elevator today" and got off. What is wrong with people?

Did you ever ride in the elevator again?

There was a bank of six, I stayed off that one for a week.

Re: There's No Fire Alarm for Artificial General Intelligence

#48

Post by ajb »

alrs wrote: post #24

I was on an elevator full of people, and between two floors something went wrong and we went into free-fall for part of a second. We made it to the next floor, the door opened, my fellow passengers were content to stay in the elevator. I turned, said "My plan is to not die in an elevator today" and got off. What is wrong with people?

What's wrong is that people have a strong, biassed, prior that they will continue to exist. Taking action to avoid death occurs by instinct, not reason, since instinct is informed by evolution, rather than just your life to date.

Re: There's No Fire Alarm for Artificial General Intelligence

#49

Post by edanm »

backpropaganda wrote: post #5

The only non-speculative and relevant claim here is that the experts were wrong about Winograd Schemas. The paper Eliezer cites to prove that we've made unexpected progress in Winograd Schemas only deals with a very specific type of Winograd Schemas, and not an arbitrary one. This is awfully dishonest for someone purporting to be a skepticist. Also, the wording seems to imply that WS performance is already pretty hig…

"Alphago was not such an event. Yes, we did predict that Alphago is decades away, but that's assuming that academics will continue working on it at their pace using their limited resources. No expert was surprised with Alphago."

Even taking that as true, I'm not sure how it's relevant. The article isn't talking about how good our forecasting is given certain assumptions. It's saying that we won't know until right before or possibly right after AGI happens.

One perfectly valid way in which this happens will be: all the academics and experts think that AGI is 10 years away based on current academic progress, but unbeknownst to them, company X is actually secretly pouring billions into achieving AGI, so they are all surprised when it's only 1 month away. This seems to be what you are saying happened with AlphaGo, in which case you are effectively agreeing with the article, IMO.

Re: There's No Fire Alarm for Artificial General Intelligence

#50

Post by edanm »

leakydropout wrote: post #34

The disasters that may befall us if we fail to narrow this gap are many. [...] Within prosperous countries, such as the United States, there is a distinct and growing threat that increased automation, coupled with an obsolete and aimless system of education, will lead to a restratification of society in which a large middle class may find itself without suitable employment and without adequate means of filling its le…

I'm not sure what you think these quotes are suggesting. If it's that there are lots of predictions that AGI is close that haven't been borne out, you're obviously right, and in no way contradicting the article.
Post reply on HN