Machine intelligence, part 2 • Hacker News Board

Re: Machine intelligence, part 2

#121

Post by AndrewKemendo » Tue, Mar 03, 2015, 3:22 AM UTC

DennisP wrote: post #118

AndrewKemendo wrote: post #74

Earlier quoted context omitted.

Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather: I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and…

Researching friendly AI seems far more useful than government regulation. But it sounds like you and Ben are acknowledging that there is a potential problem here, which we should spend resources trying to prevent. That's a view quite different from the glib dismissals I've seen from some quarters. As Ben said in your link, "once we have AGI systems that are massively smarter than people, they are going to do what the…

But it sounds like you and Ben are acknowledging that there is a potential problem here, which we should spend resources trying to prevent.

No one who has put any serious thought into AGI denies the possibility of an existential threat from it. The fact is though that we have absolutely no clue what chain of events would lead to that. In fact we have (almost) no clue what chain of events will lead to AGI in the first place.

The best way to know what will come from AGI is to start building it and seeing where the controls and capabilities are going. I personally think the AGI community needs to put more effort into a roadmap, which is done at every AGI conference to some degree. The problem though is that it's just too early and there just aren't enough researchers and funds to have any inkling about what is (are) going to be the best approaches to AGI. I could go all day and caveat everything, but the bottom line is, just start building because we are so far from it absolutely anything helps.

"once we have AGI systems that are massively smarter than people, they are going to do what they want to do, or what other powerful forces in the universe want them to do, but not necessarily what we want them to do." That's not the most reassuring comment ever.

Right. My personal proposition is:

AGI:Humans::Humans:Ants

Ben seems to share this approach. So to think we could control it after it's fifth iteration or so is silly. We also share the transhumanist philosophy which I think is still pretty fringe even within the tech community so I try not to go into that too much because it's just a whole ball of wax. I do think though that we need to have some of these conversations about what is next if human work is not necessary and later what we do if humanity itself is not at the top of the intelligence chain.

Re: Machine intelligence, part 2

#122

Post by zep15 » Tue, Mar 03, 2015, 3:49 AM UTC

I find it odd that AI risk has become such a hot topic lately. For one, people are getting concerned about SMI at a time when research toward it is totally stalled---and I say that as someone who believes SMI is possible. Stuff like deep learning, as impressive as the demos are, is not an answer to how to get SMI, and I think ML experts would be the first to admit that!

On top of that, nothing about the AI risk dialogue is new. Here's John McCarthy [1] writing in 1969:

> [Creating strong AI by simulating evolution] would seem to be a dangerous procedure, for a program that was intelligent in a way its designer did not understand might get out of control.

Here's someone thinking about AI risk 46 years ago! The ideas put forward recently by Sam Altman and others are ideas that have occurred to many smart people many times, and they haven't really gone anywhere (e.g., at no point between 1969 and now has regulation been enacted). I wish people would ask themselves why that is before making so much noise about the topic. The only people influenced by that noise are laypeople, and the message they're getting is "AI research = reckless", which is a very counterproductive message to be sending.

[1] McCarthy, John, and Patrick Hayes. Some philosophical problems from the standpoint of artificial intelligence. USA: Stanford University, 1968.

Re: Machine intelligence, part 2

#123

Post by lukeprog » Tue, Mar 03, 2015, 4:11 AM UTC

AndrewKemendo wrote: post #74

lukeprog wrote: post #55

Earlier quoted context omitted.

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include: * Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook * Tom Dietterich, AAAI President * Eric…

Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather: I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and…

I know for a fact that at least 7 of the top AI people that I listed above think intelligence explosion / superintelligence is plausible and that it's not at all clear that will go well for humans. I'm pretty sure all of them think AGI will be feasible one day and will be fairly hard to make reliably safe, at the very least to the degree that current safety-critical systems require a bunch of extra work to make safe (beyond what you'd do if your AI system has no safety implications, e.g. detecting cats on YouTube).

I can't say which people believe which things, because most of them communicated with me in confidence, except for Stuart Russell and Shane Legg because they've made public comments about it about their worries about intelligence explosion:

http://futureoflife.org/PDF/stuart_russell.pdf

http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from...

Re: Machine intelligence, part 2

#124

Post by AndrewKemendo » Tue, Mar 03, 2015, 4:21 AM UTC

lukeprog wrote: post #123

AndrewKemendo wrote: post #74

Earlier quoted context omitted.

Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather: I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and…

I know for a fact that at least 7 of the top AI people that I listed above think intelligence explosion / superintelligence is plausible and that it's not at all clear that will go well for humans. I'm pretty sure all of them think AGI will be feasible one day and will be fairly hard to make reliably safe, at the very least to the degree that current safety-critical systems require a bunch of extra work to make safe…

There is a big difference in acknowledging the potential of a threat which I agree that everyone does, and advocating for an intentional slowing (or rather preventing accelerating) development or alternatively advocacy of regulation until we get "Friendly AI" figured out like MIRI does.

A world of difference actually.

Re: Machine intelligence, part 2

#125

Post by [deleted] » Tue, Mar 03, 2015, 4:52 AM UTC

leot wrote: post #72

antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

While Altman and Musk don't spend all their time on it, I'd hardly call them "lay people". Altman studied AI at Stanford, and Musk knows a fair bit more than the average person about computer science and technology. The DeepMind people are also pretty bright on this topic, and they were concerned enough to set up an "ethics board". As someone with multiple degrees on related topics, I was pretty skeptical of the conc…

[deleted]

Re: Machine intelligence, part 2

#126

Post by dkarapetyan » Tue, Mar 03, 2015, 5:02 AM UTC

ggreer wrote: post #70

dkarapetyan wrote: post #9

Why is this all of a sudden such a hot topic? We're nowhere near anything resembling general intelligence even at the level of a toddler. Barring any kind of hardware innovations we will not get there any time soon either with just software.

> We're nowhere near anything resembling general intelligence even at the level of a toddler. Barring any kind of hardware innovations we will not get there any time soon either with just software. Should people have discussed the control and proliferation of nuclear weapons decades before their invention? It seems to me that even a century beforehand, the conversation would have been productive. If anything, we got…

Experts have been predicting human level intelligence 30-50 years out for a while now. I don't see anything in the current line of research that will change that situation.

My measure of intelligence is creative problem solving in a mathematical discipline like theoretical physics, algebraic topology, combinatorics, etc. All the current AI research is doing is building better and better pattern matching engines. That's all very good but talking about sophisticated pattern matching pieces of code as if they were anything more seems very silly to me.

But I don't think looking at things this way is valuable in either case. Hamming has a great set of lectures on what it would mean for machines to think and in the grand scheme of things I think the question is meaningless. The real question is what can people and thinking machines accomplish together.

Re: Machine intelligence, part 2

#127

Post by rl3 » Tue, Mar 03, 2015, 5:19 AM UTC

AndrewKemendo wrote: post #120

jeffreyrogers wrote: post #97

Earlier quoted context omitted.

I'm glad to see that people like you are replying to some of the misinformation that occurs in threads like these. It's a shame that on HN the people with the least knowledge are the most likely to cast their opinions in broad sweeping statements grounded in hyperbole rather than fact, particularly when the topics of AI and ML come up. In fact a number of the signatories that Luke cites are quoted as saying on no unc…

At the risk of falling into "ad hominem" territory, it is disheartening that Altman (Or Muehlhauser or most of the MIRI people for that matter) gets any play at all on this topic. The fact that Altman thanks Luke in his blog for review just gives you a sense of the "blind leading the blind" here. I think however that says more about the lack of awareness on the part of the readers than anything. That combination is a…

>At the risk of falling into "ad hominem" territory, ...

I wouldn't worry, you and the parent already have solid footing in "appeal to authority" territory.

Re: Machine intelligence, part 2

#128

Post by api » Tue, Mar 03, 2015, 5:45 AM UTC

_delirium wrote: post #39

jbhatab wrote: post #35

Earlier quoted context omitted.

While I understand where you are coming from, one remarkable breakthrough can change that. That does sound absurd, but we would also look like fools as a humanity for being aware of this potential problem and at least not looking into it. Technology has been exploding at a remarkable rate, I'm fine with things slowing a little for the sake of the next 1000 years of humanity.

Sure, but that's true in essentially all kinds of scientific research. I personally think physics research is more likely to produce an existential threat than AI research is. Fusion bombs are already pretty close to a world-ending discovery, possibly already across the line. And that is probably not the worst physicists have to offer in the future, either.

Curious for a reference on fusion bombs. If you are referring to laser inertial, that takes buildings full of lasers.

Re: Machine intelligence, part 2

#129

Post by nzp » Tue, Mar 03, 2015, 5:58 AM UTC

rl3 wrote: post #27

As much as I agree that AGI is a real concern that we should start taking seriously, regulation isn't the answer. At least not yet. Even setting aside all of the considerable drawbacks inherent to regulatory infrastructure, the end result is that, unless the regulations themselves were secret, anyone could discern precisely what the most unsafe areas of research are. The last thing you want is hobbyists able to effec…

> A better approach might be for governments to utilize their vast resources [...] in a safe and moral fashion.

Serious question—you’re joking, right? Based on history and sober common sense, do you really think what governments in general are doing (and have always done) can be described as safe and moral in anything but the most cynical way?

Re: Machine intelligence, part 2

#130

Post by nzp » Tue, Mar 03, 2015, 6:18 AM UTC

davmre wrote: post #56

fchollet wrote: post #41

Here is a short recap of what the people who understand where machine intelligence is now, and where it is going, think of the whole "evil superintelligent AI" blogging trend we're seeing these days. These are some of the people who created the field. In particular LeCun and Bengio are in part responsible for the recent renewed interested in it ("Deep Learning"). Yann LeCun: "Some people have asked what would prevent…

Expert opinion is not unified on this. E.g., Stuart Russell: "A highly capable decision maker – especially one connected through the Internet to all the world's information and billions of screens and most of our infrastructure – can have an irreversible impact on humanity ... We need to build intelligence that is provably aligned with human values ... This issue is an intrinsic part of AI, much as containment is an…

What you quote here is in no way in disagreement with what fchollet quoted. I strictly agree with this. No one is denying the need to not create a raving super-intelligent psychopath. The problem with this fear mongering is that it's not based on any fact, namely, implying that AGI (or SMI, whatever) would by default (naturally even!) be hostile or negligently indifferent to biological life. It could very well be that it would naturally tend to be very friendly, or at least benevolently indifferent. Why would such an entity try to destroy humans?

Not only that, the proposal is to get governments, the US government especially—an organization which in total has not been waging war for only a few decades during it's entire existence—in explicit and tight control of this powerful technology. Doesn't quite seem to be the best idea around, frankly.