Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine intelligence, part 2

blog.samaltman.com

Post reply on HN
Print view View original

71–80 of 157 posts

Re: Machine intelligence, part 2

#71

Post by lazaroclapp »

lukeprog wrote: post #55
antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include: * Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook * Tom Dietterich, AAAI President * Eric…

To be fair "AI safety" is very much an issue even without human-level or post-human AI. I mean, there are far more "mundane" problems such as what happens when a self-driving car gets in a collision? or what do we do after factory workers, service workers and truck drivers are replaced by (dumber-than-human) machines for those jobs?

Re: Machine intelligence, part 2

#72

Post by leot »

antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

While Altman and Musk don't spend all their time on it, I'd hardly call them "lay people". Altman studied AI at Stanford, and Musk knows a fair bit more than the average person about computer science and technology. The DeepMind people are also pretty bright on this topic, and they were concerned enough to set up an "ethics board".

As someone with multiple degrees on related topics, I was pretty skeptical of the concerns as well. That is, until I read Nick Bostrom's book "Superintelligence": the fact is that the consequences of SMI are so potentially disastrous that even if the odds were 1 in a 100 it would make sense for us to be very careful. But the probability of SMI happening within the next 100 years is likely far more than that.

Don't want regulation? Then come up with a proposal that has a better chance of keeping things under control and simultaneously utterly obviates any incremental benefit caused by imposition of a regulatory framework.

It's one thing to distrust authority, especially when said authority wasn't earned. In the case of Musk, Altman, Bostrom, (and many others) ignoring the thoughtfulness they have brought to this issue seems extremely reckless.

Re: Machine intelligence, part 2

#73

Post by mgpc »

I think @sama's plan isn't very likely to work. Once strong AI is possible, there's no regulatory structure that will keep the genie in the bottle for a meaningful amount of time.

The only path I see with some prospect of success is to limit the total amount of computation available on Earth. (Basically like Vinge's slow zone). If we could engineer a pause in Moore's Law at just the right point, it would buy us time. Maybe we should put a drastic tax on computation at some stage.

Re: Machine intelligence, part 2

#74

Post by AndrewKemendo »

lukeprog wrote: post #55
antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include: * Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook * Tom Dietterich, AAAI President * Eric…

Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather:

I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and beneficial.

The letter helps give visibility and to that end better avenues for funding for AGI projects.

[1]http://multiverseaccordingtoben.blogspot.hk/

Re: Machine intelligence, part 2

#75

Post by DennisP »

jeremyrwelch wrote: post #21

I fear anthropomorphizing machine intelligence and attempting to simply "regulate" it like other human affairs will cause more harm than good.

I agree that regulation is probably worse than useless. But the people who are most worried are doing the opposite of anthropomorphizing. A prominent argument is that there are a very large number of possible AI motivations, which needn't have anything in common with the motivations of humans, and most of them are probably incompatible with human survival.

"The AI does not love you or hate you, but you are made out of atoms it can use for something else."

Re: Machine intelligence, part 2

#76

Post by jal278 »

fiatmoney wrote: post #49
sama wrote: post #22

Earlier quoted context omitted.

>This has basically the highest insanity * prominence of speaker product I've ever seen. Is that two compliments in one? :) I am debating releasing a sort of part 3, which actually could be called "insanity", but you should save the criticism for that one.

Maybe in that one you could explain what about your experience with machine learning algorithms and pharma style highly regulated industries leads you to believe that turning machine learning algorithm development (not even "deployment", mind you, since in the FOOM scenario developing a self improving capability is tantamount to deploying it) into a highly regulated industry wouldn't be an unmitigated disaster.

And maybe you could describe why regulating AI research must be an unmitigated disaster? If over time it proves that certain areas of research provide a clear existential threat to our species, why is it irrational to then attempt to slow down certain fields of research relative to others?

Re: Machine intelligence, part 2

#78

Post by jeremyrwelch »

DennisP wrote: post #75
jeremyrwelch wrote: post #21

I fear anthropomorphizing machine intelligence and attempting to simply "regulate" it like other human affairs will cause more harm than good.

I agree that regulation is probably worse than useless. But the people who are most worried are doing the opposite of anthropomorphizing. A prominent argument is that there are a very large number of possible AI motivations, which needn't have anything in common with the motivations of humans, and most of them are probably incompatible with human survival. "The AI does not love you or hate you, but you are made out o…

This is exactly what I am referring to actually. Re-read my point, but apply it to your own example. This prominent argument assumes that we would be able to understand the possible AI motivations, which is a form of anthropomorphizing. We must acknowledge that the reasons it exists and what it may want could be impossible for us to understand or process. The interesting question then becomes how to act if understanding its wants is impossible?

Re: Machine intelligence, part 2

#79

Post by vonnik »

Contributor to Deeplearning4j here.

I deeply disagree with Sam’s assessment of the dangers of SMI and his prescriptions to regulate it.

1) In our lifetimes, humanity is much more likely to damage itself through anthropogenic climate change and nuclear disasters both deliberate and accidental than it is vulnerable to the development of SMI. If anything, machine intelligence will probably allow us to approach those other significant problems more effectively.

2) For regulatory limits on the progress and use of technology to be effective, the rules must be global, and the technology must be detectable. Neither of those conditions is likely to be fulfilled in this situation. Any law that requires the agreement of and enforcement by every nation in the world fails before it is even formulated. Humanity has shown very little capacity to agree on a global scale. The proliferation of nuclear arms and the persistence of human trafficking are two cases in point.

3) Unlike nuclear testing, the development of SMI would not be detectable with Geiger counters. It does not produce explosions that are seismologically measurable half a planet away.

While the US should incentivize the “right” research into machine intelligence through financing and other means, onerous regulations and absurd government bodies will simply shift research offshore to other countries looking for an edge. (Machine learning research does not require large facilities. All you need is a keyboard and the cloud.) That offshoring would be disastrous for the United States, because our competitive advantage in the world economy is technological innovation.

At the same time, regulation by slow-moving committees will simply drive military/intelligence research underground. I doubt the three-letter agencies will acquiesce to a law that puts them at a disadvantage to their peers.

In other words, if SMI is possible, then it is inevitable. It will be put to many uses, like all technology. If we are technological liberals at all, then we must trust that the positive affects of machine intelligence, on balance, will outweigh the negative ones. We should not be the ones sowing fear.

Re: Machine intelligence, part 2

#80

Post by jeremyrwelch »

DennisP wrote: post #75
jeremyrwelch wrote: post #21

I fear anthropomorphizing machine intelligence and attempting to simply "regulate" it like other human affairs will cause more harm than good.

I agree that regulation is probably worse than useless. But the people who are most worried are doing the opposite of anthropomorphizing. A prominent argument is that there are a very large number of possible AI motivations, which needn't have anything in common with the motivations of humans, and most of them are probably incompatible with human survival. "The AI does not love you or hate you, but you are made out o…

And a related question – if we can't understand its motivations or its why, what can we actually understand about it?
Post reply on HN