Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There’s No Fire Alarm for Artificial General Intelligence (2017)

intelligence.org

Post reply on HN
Print view View original

121–130 of 130 posts

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#121

Post by nopinsight »

taberiand wrote: post #117
nopinsight wrote: post #73

Earlier quoted context omitted.

From the article, would you say we shouldn’t prepare for anything if we suspect that a horde of advanced alien space ships may be landing on earth in a few decades (and possibly sooner)?

My point is in thirty years, due to humankinds inability to work together on a common goal of self-preservation and equality, we'll most likely simply be further down the path of the probable collapse of civilization. If we knew aliens were coming, we'd probably be best off hoping for their benevolence and mercy and preparing as best we could to deserve it.

I respectfully disagree. You may wish to check out “Enlightenment Now” by Stephen Pinker for a more optimistic perspective on human progress.

Aliens may not share our values. What we perceive as good may not be so for them, nor might they care if we are “good”.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#123

Post by narag »

umvi wrote: post #112
narag wrote: post #106

Earlier quoted context omitted.

In the name is not the same as having profits.

You can also make crappy decisions that increase profit in the short term but burn out your employees and run the company into the ground in the long term.

I didn't say "short term profits".

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#124

Post by memexy »

nopinsight wrote: post #120
memexy wrote: post #110

Earlier quoted context omitted.

I do not share your perspective and fears then. The point isn't that a computer can beat Magnus, the point is that computers helped him reach his full potential and the same technology when expanded through AI can help others in the same way. I don't fear any AGI takeover because people already are AGI and life is pretty good being among them.

AIs, being digital, have some clear advantages over people. Very rapid replication, enormous communication bandwidth, potential to expand capacity to global scale, for example. A human-level AI with these features common among ordinary software will already be super-human. People have deep, often implicitly shared values. Most care for human and their own lives, for example. Given limited capacity of a person, they n…

> A human-level AI ...

What evidence do you have for this claim? What are examples of research programs that claim to be working towards this goal and why are their claims to be believed (other than being good marketing material for scaring people).

I already mentioned mathematical abilities (https://news.ycombinator.com/item?id=23413003) and any general intelligence at a level of a human must be able to prove theorems without brute force search. I see no evidence that this is possible or will ever be possible with the current statistical methods and neural network architectures. And if we generalize a little bit then any general intelligence will be able to not only prove theorems in complex analysis but in all domains of mathematics and again I see no evidence that this is possible with existing techniques and methods. When an AI research lab presents evidence for any of their products being able to derive and prove something like Cauchy's Residue Theorem then I will have reason to believe artificial intelligence can reach human levels of intelligence.

My pessimism is not about AI being beneficial, my pessimism is about folks claiming human level intelligence is possible and that it will be malevolent. My view of AI is the same as Demis Hassabis' view because AI is just a tool and a tool can't be malevolent:

> "I think about AI as a very powerful tool. What I'm most excited about is applying those tools to science and accelerating breakthroughs" [0]

--

[0]: https://www.techrepublic.com/article/google-deepmind-founder...

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#125

Post by memexy »

pgt wrote: post #52
memexy wrote: post #41

Earlier quoted context omitted.

Proving theorems. I will nominate Cauchy's Residue Theorem. It has already been formalized so a neural network or any generally intelligent agent should be able to do the same: https://www.cl.cam.ac.uk/~wl302/publications/itp16.pdf . Complex analysis in general is one of the more fun parts of math and I doubt it's getting automated any time soon. It also has a lot of application in signal processing ( https://www.quo…

Depending on who's holding the hammer, I do fear the hammer.

Why would you fear the hammer and not the person holding the hammer.

Addressing the problem of a person coming at you with a hammer is a much more pertinent issue than worrying about self-aware and malevolent hammers.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#126

Post by mD5pPxMcS6fVWKE »

General artificial intelligence is nowhere on the horizon, but in it's lesser form - deep learning - it is already happening and already hurting a lot of people. The problem is that its development is concentrated at the hands of a handful of companies and they use it to their competitive advantage. Smaller firms just can't compete. You need 1) algorithms - these are more or less public 2) a lot of hardware - this is very expensive 3) a lot of (labeled) data - this is just not available. You can't get access to the training set that Google collects when billions of people solve reCaptchas, or the NSA listening to billions of conversations, or the Amazon looking at what people buy and what people search.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#127

Post by armitron »

memexy wrote: post #111
Smaug123 wrote: post #107

Earlier quoted context omitted.

There is nothing beyond a human with an abacus as far as computation is concerned, and yet computers can do so much more. "Turing complete, therefore nothing can do any better" is true only in the least meaningful sense: "given infinite time and effort, I can do anything with it". In reality we don't have infinite time and effort. You seem to believe that "figuring some way out of performing the given task" is a thin…

> It can augment and help people improve their planning horizons which in my book is always a good thing. Why do I need protection from something that helps me become a better decision maker and planner? Every computational tool has made me a better person. I want that kind of capability as widely spread as possible so everyone can reach their full potential like Magnus Carlson. More generally, whatever capabilities…

Humans have no moral or ethical concerns that stop them exterminating life forms they deem inferior. You don’t think it's plausible superior AGI would view humans as vermin?

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#128

Post by nopinsight »

memexy wrote: post #124
nopinsight wrote: post #120

Earlier quoted context omitted.

AIs, being digital, have some clear advantages over people. Very rapid replication, enormous communication bandwidth, potential to expand capacity to global scale, for example. A human-level AI with these features common among ordinary software will already be super-human. People have deep, often implicitly shared values. Most care for human and their own lives, for example. Given limited capacity of a person, they n…

> A human-level AI ... What evidence do you have for this claim? What are examples of research programs that claim to be working towards this goal and why are their claims to be believed (other than being good marketing material for scaring people). I already mentioned mathematical abilities ( https://news.ycombinator.com/item?id=23413003 ) and any general intelligence at a level of a human must be able to prove theo…

By the time a single AI system can prove those theorems and learn to perform well on very unrelated tasks, it could be too late to think about AI Safety.

AI Safety researchers and I are not arguing that AGI will certainly arrive in a specified amount of time, it just that we can’t be sure.

There are a significant number of AI researchers, however, who believe it might get developed within a few decades.

OpenAI, for example, is aiming for it; DeepMind as well. Both have research programs on AI Safety.

Do you have any other objections to the reasoning in the OP (no fire alarm..)? Subjective implausibility is not a good one. Your argument rests on using only “existing techniques”. When thousands of brilliant minds are working in the field and hundreds of good papers are being published every year, how can we be sure there won’t be a novel technique that can perform outside current limitations within a few decades?

At least two top computer vision researchers I talked with a couple years ago said they didn’t believe what their groups could do just 5 years before they did it.

See this thread for more on the rationale for preparation: https://news.ycombinator.com/item?id=23414197

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#129

Post by amcknight »

memexy wrote: post #41
jbay808 wrote: post #15

There are a lot of people in this thread who seem to be very confident and skeptical of the author's position. Maybe this would be a good opportunity to try the exercise he challenges in the article? Please post your nomination for what’s the least impressive accomplishment that you are very confident cannot be done in the next two years. We can check back in 2022 and see how we did!

Proving theorems. I will nominate Cauchy's Residue Theorem. It has already been formalized so a neural network or any generally intelligent agent should be able to do the same: https://www.cl.cam.ac.uk/~wl302/publications/itp16.pdf . Complex analysis in general is one of the more fun parts of math and I doubt it's getting automated any time soon. It also has a lot of application in signal processing ( https://www.quo…

You've basically amounted to saying autonomous programs are impossible. The problem is when the hammer can run around on its own hammering every nail shaped object it can find.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#130

Post by amcknight »

someguyorother wrote: post #82
nopinsight wrote: post #68

Our civilization and ability to dominate over other animals largely exist because of human intelligence. An AGI could be a great partner toward unprecedented prosperity, but an AGI not fully aligned with human values may also spur a major catastrophe. Two promising approaches toward AI Safety I have encountered: * Make sure the AI is not too certain of itself and will always defer to humans as the final judge. https:…

You're going to have to implement a value system into the AGI if it is a fully general AGI. Otherwise, it will just use its exceptional intelligence to defeat its own sensors, take control of its objective function, and essentially wirehead itself. So an AGI without a value system is (mostly) harmless. If the AGI has a defined value system, it's a lot clearer what that value system is. No "whoops gonna destroy the wo…

An AGI with a small goal that it can wirehead to max out is not harmless by default. It can always add 9s to its confidence that it isn't being tricked and then secure itself while doing so.
Post reply on HN