Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There's No Fire Alarm for Artificial General Intelligence

intelligence.org

Post reply on HN
Print view View original

181–190 of 209 posts

Re: There's No Fire Alarm for Artificial General Intelligence

#181

Post by YeGoblynQueenne »

I don't get this article. It keeps making the point that it's very hard to predict the future, even for specialists, then it uses this to argue that we should be preparing for AGI right now, precisely because we don't know if and when it will happen.

Well, if you have no way to tell whether something is going to happen, or not, you don't prepare for it- because you can't justify spending the resources to prepare. Or rather, in a world of limited resources, you can't prepare for every single event that may or may not happen, no matter how important.

To put it plainly: you don't take your umbrella with you because you don't know whether it will rain or not. You take it because you think it might. Otherwise, everyone would be going around with umbrellas all the time, just because it's impossible to make a completely accurate prediction about the weather and you don't know for sure when it will start raining until the first drops fall.

In the same sense, if there's no way to tell when, or if, AGI will arrive, then it doesn't make any sense to start preparing for it right now. We might as well prepare for an alien invasion. Or for grey goo, or a vacuum metastability event (er, not that you can prepare for the latter...).

In fact, if AGI is going to happen and we can't predict it in time then there's no point in even trying to prepare for it. Either we decide that the risk is too great and stop all AI research right now, or accept the risk and go on as we are.

Re: There's No Fire Alarm for Artificial General Intelligence

#182

Post by jarsta »

YeGoblynQueenne wrote: post #181

I don't get this article. It keeps making the point that it's very hard to predict the future, even for specialists, then it uses this to argue that we should be preparing for AGI right now, precisely because we don't know if and when it will happen. Well, if you have no way to tell whether something is going to happen, or not, you don't prepare for it- because you can't justify spending the resources to prepare. Or…

I don't think your analogies are that good. Do you have a fire detector? If yes, are you expecting your house to burn down?

You have to weigh the cost and the risk. Here the risk, how unlikely it might be, should warrant some extra preparation.

Re: There's No Fire Alarm for Artificial General Intelligence

#183

Post by YeGoblynQueenne »

jarsta wrote: post #182
YeGoblynQueenne wrote: post #181

I don't get this article. It keeps making the point that it's very hard to predict the future, even for specialists, then it uses this to argue that we should be preparing for AGI right now, precisely because we don't know if and when it will happen. Well, if you have no way to tell whether something is going to happen, or not, you don't prepare for it- because you can't justify spending the resources to prepare. Or…

I don't think your analogies are that good. Do you have a fire detector? If yes, are you expecting your house to burn down? You have to weigh the cost and the risk. Here the risk, how unlikely it might be, should warrant some extra preparation.

Let's talk about the risks then. The fire detector is not a good example because where I live, they're mandatory (and completely useless- they go off when I boil spaghetti).

Let's instead look at the risks of boarding a plane. There's a very small chance that when you board a flight, instead of a plane that will fly you to your destination safe and sound, you're boarding a Flying Death Trap that will crash and burn, taking everybody onboard it to their deaths.

The chances of boarding an FDT is very small, infinitisemal. The cost however may as well be infinite- if you are killed, it's game over, no more rewards, no way to recoup the cost.

What is the rational behaviour then? To not board your flight, because if you do board an FDT you will certainly die and pay an infinite cost? Most people -if they consider the question at all- seem to think that if the chance of paying X cost is really small, it doesn't matter how large X is.

So people keep boarding their flights, not knowing until the last moment whether they're on a plane or an FDT. Some do indeed board FDTs and die in aviation accidents- rarely, but they do.

The article however says that they shouldn't. Since there is maximal uncertainty at the point where a flight is boarded (you can't know whether it's a plane or an FDT until the very last moment) you shouldn't be boarding. You shouldn't fly. At all. Because there's a tiny chance you might die.

Is that a better analogy?

Re: There's No Fire Alarm for Artificial General Intelligence

#184

Post by YeGoblynQueenne »

saurabh20n wrote: post #137
randomsearch wrote: post #132

Can someone please explain what has happened in ML or AI that makes AGI closer? Whilst some practical results (image processing) have been impressive, the underlying conceptual frameworks have not really changed for 20 or 30 years. We're mostly seeing quantitive improvements (size of data, GPGPU), not qualitative insights. ML in general is just applied statistics. That's not going to get you to AGI. Deep Learning is…

Agree. Deep learning does not bring us closer to AGI. It might get us closer to other proxies of "mechanical intelligence" that will be very productive. I now believe we are 3 years from building an AI that writes Python well enough to build itself, based on some experiments I did recently: http://sparkz.org/ai/program-synthesis/2017/10/12/self-hosti... Most technical people will understand the difference between pro…

>> I now believe we are 3 years from building an AI that writes Python well enough to build itself, based on some experiments I did recently

Why 3 years? Can you elaborate on the timeline? What should happen in 1 year, what in 2, what in 3 etc?

Re: There's No Fire Alarm for Artificial General Intelligence

#185

Post by fourfaces »

We don't need a fire alarm for AGI. The problem is not AGI. Machines will be motivated to do exactly what we tell them to do. It's called classical and operant conditioning. The problem is not AGI for the same reason that the problem is not knives, nuclear power, dynamite or gunpowder. The problem is us. The problem has always been us.

Those who are running around screaming about the danger of AGI and why it should be regulated by the government before it is even here, are just scared that someone else may gain control of it before they do. This is too bad because anybody who is smart enough to figure out AGI is much smarter than they are.

Re: There's No Fire Alarm for Artificial General Intelligence

#186

Post by pizza »

yorwba wrote: post #38
pizza wrote: post #25

Earlier quoted context omitted.

OK but here's a little gedankenexperiment: suppose I had a machine that could simultaneously simulate 2 consciousnesses (either parent/child configuration or sibling), (bear with me) import recordings from conscious beings, and allowed one consciousness to query the other. Then it could know if someone were conscious or not. (if I am getting this right) those would be the minimal criteria for being able to (contentio…

> Then it could know if someone were conscious or not. How? You say "query", but what would the query look like?

subsumption followed by introspection

Re: There's No Fire Alarm for Artificial General Intelligence

#187

Post by ThomPete »

fourfaces wrote: post #175
ThomPete wrote: post #72

Humans evolved from immaterial matter to conscious carriers of information. The real question isn't whether AGI is possible but whether humans are the fittest carrier of information for our DNA and that seems to be technology in some shape or form helped by things like deep learning. My bet is always on evolution. And now that technology can learn it's IMO only a matter of time before we will experience another Cambr…

>Humans evolved from immaterial matter to conscious carriers of information. You know this how? Where is the science behind it?

What do you mean where is the science behind it? It's literally the science we know (physics, chemistry, biology evolution). From primordial soup to humans. Not sure what you are putting into question here?

Re: There's No Fire Alarm for Artificial General Intelligence

#188

Post by sullyj3 »

etiam wrote: post #93

As far as I'm concerned this whole discussion is severely hampered by failing to differentiate between intelligence and agency. Almost all of the bugaboo about runaway superhuman organisms comes down not to machines learning and reasoning about the world but to the effective high-level objective function controlling the actions of an autonomous system. Not making the distinction obscures important things. For one thi…

> without any specifics of what works for an implementation

As I understand it, if you assume that your agent is rational in certain basic ways, for instance that it has ordered, rather than circular preferences, or it can't be Dutch booked/money pumped, it can modelled as having a utility function.

Note that this is different from assuming that an explicit utility function will be programmed in, rather the basic level of rationality implies it into existence.

Once you know that an agent has a utility function, you can use that to do a fair amount of reasoning about its reasoning.

Better (and less briefly) explained here: https://intelligence.org/2016/12/28/ai-alignment-why-its-har...

Re: There's No Fire Alarm for Artificial General Intelligence

#189

Post by sullyj3 »

johansch wrote: post #145
gooseus wrote: post #140

Earlier quoted context omitted.

I don't think I am... if a task is possible, but takes 100x the lifetime of the universe to accomplish, is it actually "possible"? Or is accomplishing that task the same as "magic"? It I send a message using a one-time pad, the other person knows what I sent, you can ask questions and see that communication is actually happening... so if we're not using magic to communicate, it must be possible to communicate in the…

That doesn't really make much sense to me, I'm afraid. Please bear with me, but my first guess is that you have invented a coping mechanism that allows you to deal with conflicting information in your mind. Are you by any chance religious? (I figure I should be allowed to ask such a normally speaking quite loaded question because of my previous statements, up the thread.)

Your first guess? You don't get to dismiss peoples' arguments as superstition just because you don't understand them.

Re: There's No Fire Alarm for Artificial General Intelligence

#190

Post by sullyj3 »

YeGoblynQueenne wrote: post #181

I don't get this article. It keeps making the point that it's very hard to predict the future, even for specialists, then it uses this to argue that we should be preparing for AGI right now, precisely because we don't know if and when it will happen. Well, if you have no way to tell whether something is going to happen, or not, you don't prepare for it- because you can't justify spending the resources to prepare. Or…

I don't think the point of the article is to make a complete argument for acting now, but rather to counter a particular objection to acting now. MIRI has a fair amount of writing, which I would say in totality constitutes an argument for beginning work on AI safety ASAP. I don't think it's fair to expect them to reiterate the entire thing in every article they put out. That's what their FAQ is for.
Post reply on HN