Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What you wanted to know about AI

fastml.com

Post reply on HN
Print view View original

111–120 of 165 posts

Re: What you wanted to know about AI

#111

Post by AndrewKemendo »

cousin_it wrote: post #105
AndrewKemendo wrote: post #89

Earlier quoted context omitted.

We know that human brains don't run on magic, therefore you can build a machine that can do the same thing. I mean that is what I think, and most of the community thinks, but we haven't proven that empirically - nor could we, its unprovable. But you don't have any good arguments for feeling safe. Correct. Notice that I am not saying that we are safe. I personally think we aren't safe [1], but that is a different disc…

Whether AGI is "tragically underfunded" or "luckily underfunded" is a matter of perspective. If the DoD announced a Manhattan-scale project to create an AI for defense, I'd be more worried than I am now. At the current levels of funding, smaller groups have at least some chance of making progress on safety research. It's also encouraging that Google is taking the problem somewhat seriously, and has instituted an "AI…

Definitely agreed, except for the DoD Manhattan project part - I would love to see that.

Re: What you wanted to know about AI

#112

Post by cousin_it »

eli_gottlieb wrote: post #98
cousin_it wrote: post #73

Earlier quoted context omitted.

Probability of AGI is close to 100%, conditional on mankind surviving long enough. We know that human brains don't run on magic, therefore you can build a machine that can do the same thing. (Turing computability doesn't matter, if the brain relies on non-Turing-computable physical effects then a machine can rely on those as well.) The current situation where unmodified humans are the smartest creatures around is uns…

>Saying that we don't have to worry because smarter than human intelligences will have a better understanding of morality is also missing the point. All currently known mathematical formalisms for decision-making allow you to plug in arbitrary utility functions, including "immoral" ones. Thinking that future decision-making formalisms will support only "moral" utility functions is extreme wishful thinking. I'm going…

Yeah, I agree that reinforcement learning is probably a bad approach to FAI. Most of our toy models involve utility functions encoded directly into the AI, not reinforcement.

That said, it's indeed very hard to directly specify a utility function involving paperclips. If our universe were a Game of Life universe and we knew exactly which configuration corresponds to a paperclip, I'd be able to do that right now. But since we don't know the true laws of physics, the "hard way" involves encoding some kind of Solomonoff prior over all possible physical universes, and a rule for recognizing paperclips in each of them. That's kind of a tall order.

There's a shortcut that involves encoding a precise mathematical description of a human mind into an AI, and passing the buck by saying "the AI must maximize whatever function the human mind would output, given enough time to think". That would actually be straightforward to implement, if we had a description of a human mind that we were willing to trust. Unfortunately, the naive form of that approach immediately fails due to acausal blackmail. There's no obvious fix, but some folks are trying to devise non-obvious fixes, and I suppose it can be made to work in the long run.

Of course, after you can formulate a utility function in terms of a precise description of a human mind, the next step is reformulating it in terms of the outputs of an actually existing person, say some webcam videos and written instructions. You instruct the AI to infer the simplest program that would generate these outputs (using something like the Solomonoff prior again), and then proceed as in the previous step. That part has its own problems, but I expect that it can also be made to work.

The whole approach is kind of a long shot, but I hope that I've given a sense that the problem could be solvable by careful human effort on the scale of years. There are other approaches as well.

Re: What you wanted to know about AI

#113

Post by lsz9 »

blixt wrote: post #69

I would agree with this article that every independent solution today does not have the slightest chance to be iteratively improved to some kind of real intelligence. Even the "generic" game playing AI is very limited outside the scope of a pixel buffer and some digital output signals. However, I don't understand how the arguments correlate to "there is no need to worry about AI". The real worry should come from the…

No doubt increased funding and interests will increase the odds of stumbling upon "cheap" strong AI - a faster-than-brute-force solution. But consider the only strong intelligence we know: say intelligence only relied on the brain (which might well not be the case). This organ consumes a large fraction of the energy available to the individual. I'm not saying evolution produces optimal solutions but I guess its fair to say that a vastly superior algorithm would have had a good chance of concurring the world by now. So imitating the biological processes of the brain in silicon would be rather costly no matter the gain. And that would in the end (at best) be a one-in-seven-billion individual ie. no magic powers. I guess funding will flow towards practical systems that augument some consumer who will in the end provide a return on the investment. Just saying we are not short on intelligence and its not likely to be cheap to run massive scale super-AI in silicon.

Re: What you wanted to know about AI

#114

Post by maaku »

csallen wrote: post #76
maaku wrote: post #61

Earlier quoted context omitted.

I'm more in your camp than MIRI's but... Could you argue against the position directly? You kinda took a swerve in the middle there. Bostrom, Yudkowsky, and Barrat's positions are basically "superintelligence is possible, it will eventually happen in some way, and when it does ..." It's not within the scope of the quoted works to elaborate a technical roadmap or provide firm dates for the arrival of superhuman machin…

I'm not worried about the immediate future at all. We definitely haven't reached the point where hastily-implemented regulations will do more good than harm. That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.

> That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.

Actually I am almost completely unpersuaded by the arguments of Bostrom, Yudkowsky, et al, at least in their presented strong form. Superintelligent AI is not a magical black box with infinite compute capacity, the two assumptions which really underlie the scary existential risk scenarios if you look closely. It is not at all clear to me that there will not exist mechanisms by which superintelligences can be contained or their proposed actions properly vetted. I have a number of ideas for this myself.

In the weak form it is true that for most of the history of AI its proponents have more or less ignored the issue of machine morality. There were people in the 90's and earlier who basically advocated for building superintelligent AI and then sitting back and letting it do its thing to enact a positive singularity. That indeed would have been crazy stupid. Kudos to Yudkowsky and others for pointing out that intelligence and morality (mostly?) orthogonal. But it's a huge unjustified leap from "machines don't necessarily share our values" to "the mere existence of a superintelligent amoral machine will inevitably result in the destruction of humanity." (not an actual quote)

Re: What you wanted to know about AI

#115

Post by foobarqux »

eli_gottlieb wrote: post #104
foobarqux wrote: post #33

> The fears of machine superintelligence are based on the belief that true AI is just around a corner. No, the fear of AGI is that the time between when we first start to see promising progess in AI to the point where AI is out of our control would be so short that we would not be able to react to prevent it.

>the time between when we first start to see promising progess in AI to the point where AI is out of our control would be so short that we would not be able to react to prevent it. That seems pretty absurd to me. A human-level AGI is going to require a sizable amount of time and data to train, so that it knows something rather than nothing. In fact, getting to the point where self-improvement becomes viable will take…

What's absurd is to have a strong conviction about inviolable restrictions on systems that haven't been imagined yet. The examples you cite aren't even compelling on their face.

For instance if you include whole-brain emulations in AGI (and we should because the consequences are essentially the same) it would take zero time to train. But even without whole-brain emulations a child learns rapidly from its environment alone. An AGI could conceivably do the same but faster, and moreover could take advantage of access to the Internet.

I don't know what information theory has to do with any of that.

Re: What you wanted to know about AI

#116

Post by csallen »

darkmighty wrote: post #94
csallen wrote: post #76

Earlier quoted context omitted.

I'm not worried about the immediate future at all. We definitely haven't reached the point where hastily-implemented regulations will do more good than harm. That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.

Maybe it's precisely because the it can't be argued against that it's not much use? There's no way we could tell it will ever happen in the moment, no way to tell how intelligent we can make computers, and so on. You can't reliably argue for or against. So what's the point? I mean, you may want to form a group and discuss those scenarios, but it's not excusable to spread fear mongering in the media with no substance…

I disagree that it can't be argued against. It's a set of logical steps, any one of which can be critiqued:

1 - There's no real dispute that computers will eventually be as "powerful" as brains. In fact, it's likely that they will one day be far more powerful.

2 - Assuming the brain evolved naturally, there's no reason to assume humans won't eventually duplicate the software once we have the hardware. It's really only a matter of when.

3 - An AGI with equal/more power than the human brain will, eventually, be able change its own code and improve upon itself.

4 - Given the characteristics of intelligence as it has been observed thus far in nature, even small increases in IQ lead to massive improvements in capability that are difficult for lesser intelligences to comprehend. A sufficiently advanced AGI would not only be highly capable, but would quite possibly be too smart for us to predict how it might behave. This is a dangerous combination.

Further complicating things, we might hit step 2 on accident, without realizing that we hit it. Or some group might accomplish step 2 in secret.

What I'd like to see someone do is argue that the chance of these things happening so small and/or the consequences so minuscule that it's not worth worrying about or planning for.

Re: What you wanted to know about AI

#117

Post by maaku »

AndrewKemendo wrote: post #85
maaku wrote: post #61

Earlier quoted context omitted.

I'm more in your camp than MIRI's but... Could you argue against the position directly? You kinda took a swerve in the middle there. Bostrom, Yudkowsky, and Barrat's positions are basically "superintelligence is possible, it will eventually happen in some way, and when it does ..." It's not within the scope of the quoted works to elaborate a technical roadmap or provide firm dates for the arrival of superhuman machin…

Could you argue against the position directly? No I can't because there isn't a way to argue it. I can't argue that it would be safe because I have nothing to point to that says it will be or even could be. We simply do not know enough about how it would be built to reason on it. It's not within the scope of the quoted works to elaborate a technical roadmap or provide firm dates for the arrival of superhuman machine…

Well, color me disappointed. I think there are many arguments against the viewpoints of MIRI and Bostrom, both discrediting the perceived risk of super-intelligences and the the idea that we could build a "provably friendly" alternative.

But I don't think it's at all a fair criticism to say that you need a technical roadmap to engage in that debate. I would rather say that they need to be willing to discuss specific architectural details in order to even sit at the table, sure. The field of "AI safety" needs more engineering and less philosophy. But that doesn't require unity around a single roadmap, or any concept of deliverable dates.

Re: What you wanted to know about AI

#118

Post by maaku »

AndrewKemendo wrote: post #108
darkmighty wrote: post #94

Earlier quoted context omitted.

Maybe it's precisely because the it can't be argued against that it's not much use? There's no way we could tell it will ever happen in the moment, no way to tell how intelligent we can make computers, and so on. You can't reliably argue for or against. So what's the point? I mean, you may want to form a group and discuss those scenarios, but it's not excusable to spread fear mongering in the media with no substance…

We don't really need to start writing traffic rules for flying cars. This is a great metaphor and one I wish I would have thought up sooner.

It's a terrible metaphor, or at least it argues against your own position. If flying cars were in fact something on the horizon, we would need discussions about air spaces, commute lanes, and traffic rules. Otherwise lethal mid-air collisions would be much more probable, resulting in significant bystander and property damage as these wreckage fall out of the sky.

And, no surprise, we're seeing exactly this discussion going on right now about drones.

Re: What you wanted to know about AI

#119

Post by csallen »

maaku wrote: post #114
csallen wrote: post #76

Earlier quoted context omitted.

I'm not worried about the immediate future at all. We definitely haven't reached the point where hastily-implemented regulations will do more good than harm. That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.

> That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again. Actually I am almost completely unpersuaded by the arguments of Bostrom, Yudkowsky, et al, at least in their presented strong form. Superintelligent AI is not a magical black box with infinite compute capacity, the two…

You're right: AGI might not be as powerful as some predict it can be. And there might be a way to contain it if it is. And we might be able to enforce said containment strategy on researches worldwide. And the AGI might be benevolent even if it is super powerful and uncontainable.

The authors above are unpersuasive in arguing that these things are inevitable, but they are persuasive in arguing that these things are possible. And if they are possible, then what matters to me is their likelihood. Based on current estimates, are we talking about a 50-50 chance here? 80-20? 99-1? 99.999-0.001?

If we're talking about the future of the human race, these chances matter. But instead it seems everyone is just waving their hands at the what-ifs simply because they aren't 100%.

Re: What you wanted to know about AI

#120

Post by gradys »

hartator wrote: post #109
eli_gottlieb wrote: post #92

Earlier quoted context omitted.

>And no, machine learning/deep learning is not that shift; it is just another flavor of the same statistics that everybody already uses. How do you think the human mind runs? Something other than statistics?

Human minds don't run on statistics. Some can be explained by statistical modeling but we don't really understand how an organic brain works.

Your first sentences needs some more support. We know that some of it can be explained by statistical modelling, and we don't yet know how to explain the rest of it. Why does that make you sure that it doesn't run on statistics?
Post reply on HN