Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine intelligence, part 2

blog.samaltman.com

Post reply on HN
Print view View original

91–100 of 157 posts

Re: Machine intelligence, part 2

#91

Post by msutherl »

jkramar wrote: post #82
msutherl wrote: post #61

Earlier quoted context omitted.

That letter is not about "superhuman machine intelligence", it is about "ensuring that AI remains robust and beneficial." Among the threats cited are: autonomous vehicles, machines that need to make ethical decisions, autonomous weapons, surveillance, and issues of verification, validity, and control. Words like "singularity", "superhuman", and "AGI" do not appear.

Superintelligence and intelligence explosion are mentioned as valuable things to investigate.

Ah, turns out you're right:

Finally, research on the possibility of superintelligent machines or rapid, sustained self-improvement (“intelligence explosion”) has been highlighted by past and current projects on the future of AI as potentially valuable to the project of maintaining reliable control in the long term. The the AAAI 2008–09 Presidential Panel on Long-Term AI Futures’ “Subgroup on Pace, Concerns, and Control” stated that:

"There was overall skepticism about the prospect of an intelligence explosion... Nevertheless, there was a shared sense that additional research would be valuable on methods for understanding and verifying the range of behaviors of complex computational systems to minimize unexpected outcomes. Some panelists recommended that more research needs to be done to better define “intelligence explosion,” and also to better formulate different classes of such accelerating intelligences. Technical work would likely lead to enhanced understanding of the likelihood of such phenomena, and the nature, risks, and overall outcomes associated with different conceived variants."

Stanford’s One-Hundred Year Study of Artificial Intelligence includes “Loss of Control of AI systems” as an area of study, specifically highlighting concerns over the possibility that:

"we could one day lose control of AI systems via the rise of superintelligences that do not act in accordance with human wishes – and that such powerful systems would threaten humanity. Are such dystopic outcomes possible? If so, how might these situations arise? ...What kind of investments in research should be made to better understand and to address the possibility of the rise of a dangerous superintelligence or the occurrence of an 'intelligence explosion'?"

Research in this area could include any of the long-term research priorities listed above, as well as theoretical and forecasting work on intelligence explosion and superintelligence, and could extend or critique existing approaches begun by groups such as the Machine Intelligence Research Institute.

Re: Machine intelligence, part 2

#93

Post by sanxiyn »

vonnik wrote: post #79

Contributor to Deeplearning4j here. I deeply disagree with Sam’s assessment of the dangers of SMI and his prescriptions to regulate it. 1) In our lifetimes, humanity is much more likely to damage itself through anthropogenic climate change and nuclear disasters both deliberate and accidental than it is vulnerable to the development of SMI. If anything, machine intelligence will probably allow us to approach those oth…

> Humanity has shown very little capacity to agree on a global scale. The proliferation of nuclear arms and the persistence of human trafficking are two cases in point.

This is very true. Therefore you think regulations on nuclear arms and human trafficking are worthless?

Re: Machine intelligence, part 2

#95

Post by pgodzin »

mgpc wrote: post #73

I think @sama's plan isn't very likely to work. Once strong AI is possible, there's no regulatory structure that will keep the genie in the bottle for a meaningful amount of time. The only path I see with some prospect of success is to limit the total amount of computation available on Earth. (Basically like Vinge's slow zone). If we could engineer a pause in Moore's Law at just the right point, it would buy us time.…

Are you suggesting that's what we should do in the face of this potential threat?

Re: Machine intelligence, part 2

#96

Post by jkramar »

vonnik wrote: post #79

Contributor to Deeplearning4j here. I deeply disagree with Sam’s assessment of the dangers of SMI and his prescriptions to regulate it. 1) In our lifetimes, humanity is much more likely to damage itself through anthropogenic climate change and nuclear disasters both deliberate and accidental than it is vulnerable to the development of SMI. If anything, machine intelligence will probably allow us to approach those oth…

Partial measures do matter; in fact, currently most of the advances are coming from the US. And research can be regulated. Probably noone will catch an independent researcher working alone and not releasing anything, but progress tends to come from teams of experts who have come through elite labs/universities. Furthermore having significant data&hardware seems to be helpful for making progress.

It doesn't require mass surveillance for something like this to work. I'm not saying there are clear positive next steps, just that blanket dismissals on the basis that perfect enforcement won't be reached are silly.

We shouldn't actually trust that positive effects of every technology outweigh negative ones; that seems obviously wrong (think about weapons tech for example).

Re: Machine intelligence, part 2

#97

Post by jeffreyrogers »

AndrewKemendo wrote: post #74
lukeprog wrote: post #55

Earlier quoted context omitted.

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include: * Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook * Tom Dietterich, AAAI President * Eric…

Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather: I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and…

I'm glad to see that people like you are replying to some of the misinformation that occurs in threads like these. It's a shame that on HN the people with the least knowledge are the most likely to cast their opinions in broad sweeping statements grounded in hyperbole rather than fact, particularly when the topics of AI and ML come up.

In fact a number of the signatories that Luke cites are quoted as saying on no uncertain terms that fear of AI is widely out of proportion to its capabilities now and for the foreseeable future. The most prominent I'm aware of saying thing to this effect are Yoshua Bengio, Geoff Hinton, and Yann LeCun.

Re: Machine intelligence, part 2

#98

Post by jimrandomh »

vonnik wrote: post #79

Contributor to Deeplearning4j here. I deeply disagree with Sam’s assessment of the dangers of SMI and his prescriptions to regulate it. 1) In our lifetimes, humanity is much more likely to damage itself through anthropogenic climate change and nuclear disasters both deliberate and accidental than it is vulnerable to the development of SMI. If anything, machine intelligence will probably allow us to approach those oth…

The technologies leading towards AGI/SMI are advancing. The technologies needed to make it safe are also advancing. The goal is not to ban SMIs - it's to ensure that it's safe. Think of it as a race, between research into capabilities into safety and control measures. If the safety and control research is sufficiently advanced when SMI appears, then it can solve humanity's other problems for us. If that safety research isn't sufficiently advanced, then we're in very serious trouble.

This is the idea behind differential technological advancement: favor technologies that help us make safe AGIs that will help us, over technologies that are too risky. Right now, safety-oriented AGI research is severely under-funded, so it's hard to tell the difference. Hence the need for monitoring and research. And while we certainly hope that safety will get on track to win the race, if it looks like it isn't, a regulatory body and regulatory framework may provide a saving throw.

Re: Machine intelligence, part 2

#99

Post by vonnik »

sanxiyn wrote: post #93
vonnik wrote: post #79

Contributor to Deeplearning4j here. I deeply disagree with Sam’s assessment of the dangers of SMI and his prescriptions to regulate it. 1) In our lifetimes, humanity is much more likely to damage itself through anthropogenic climate change and nuclear disasters both deliberate and accidental than it is vulnerable to the development of SMI. If anything, machine intelligence will probably allow us to approach those oth…

> Humanity has shown very little capacity to agree on a global scale. The proliferation of nuclear arms and the persistence of human trafficking are two cases in point. This is very true. Therefore you think regulations on nuclear arms and human trafficking are worthless?

Worthless is a strong word.

When you consider the number of unstable nation-states that have nuclear arms, I would say that non-proliferation has been very ineffective. Likewise human trafficking laws. I agree with their intent, but they are not working well.

What I am arguing here is that the nation that regulates first is at a disadvantage. I do not want the US to be at a disadvantage, and I do not believe that this problem is one the law can solve.

Re: Machine intelligence, part 2

#100

Post by RoboTeddy »

vonnik wrote: post #79

Contributor to Deeplearning4j here. I deeply disagree with Sam’s assessment of the dangers of SMI and his prescriptions to regulate it. 1) In our lifetimes, humanity is much more likely to damage itself through anthropogenic climate change and nuclear disasters both deliberate and accidental than it is vulnerable to the development of SMI. If anything, machine intelligence will probably allow us to approach those oth…

> we trust that the positive effects, on balance, will outweigh the negative ones.

So far, the positive effects from technology definitely seem like they've outweighed the negative ones. Technology has raised our standards of living, and we've managed to work around many potential downsides.

Most technologies to date have a fairly limited per-use radius of impact. Guns for example, are local: bullets don't travel more than a couple miles. Mishaps with guns (e.g. shooting sprees) can be terrible, but they have a small radius of impact, so they're not the end of the world.

As technologies become more powerful, it's reasonable to assume that maximum radius of impact will also increase. If a radius of impact is the size of the earth (or larger), we have to worry very much about accidents, even if we can expect accidents to be rare.

AI might be one of the first technologies humanity comes up with that has a global per-use effect radius. Extreme caution is warranted.

Post reply on HN