Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There’s No Fire Alarm for Artificial General Intelligence (2017)

intelligence.org

Post reply on HN
Print view View original

101–110 of 130 posts

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#101

Post by andrewflnr »

ilaksh wrote: post #42
andrewflnr wrote: post #37

Earlier quoted context omitted.

That's actually a bit of an interesting question. General intelligence and human-like intelligence are not necessarily the same, and I'm not 100% convinced there's overlap. We can solve lots of the problems it occurs to us to try to solve, but that alone doesn't prove that we're "general intelligences". There are probably categories of problems we can't solve or even properly conceive, just like every other specializ…

Good point and interesting idea, but it's well established in common usage and the field of AGI that the term is meant to refer to human-like intelligence. So that's the default understanding, and you would need to qualify the term to mean something more general as you are describing.

Are these the same people in the field who worry about runaway self-improvement and paperclip optimization? If so, then someone is being sloppy with their definitions, because those are not properties of human-like intelligence.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#102

Post by memexy »

caf wrote: post #50
memexy wrote: post #41

Earlier quoted context omitted.

Proving theorems. I will nominate Cauchy's Residue Theorem. It has already been formalized so a neural network or any generally intelligent agent should be able to do the same: https://www.cl.cam.ac.uk/~wl302/publications/itp16.pdf . Complex analysis in general is one of the more fun parts of math and I doubt it's getting automated any time soon. It also has a lot of application in signal processing ( https://www.quo…

I like to think I'm a generally intelligent agent, but I put long odds that I'll be doing that in the next 2 years.

Point is you could do it if you really wanted to learn enough complex analysis to understand the theorem and its proof. AI progress is impressive but it's nowhere near anything people can do when they sit down to think about stuff.

I don't fear the AI overlords. If they start proving new theorems then I will learn from them, same way Magnus Carlson learned to be the best human chess player by practicing with chess AI.

More generally, AI can extend human planning horizons and extending planning horizons is, in my opinion, a very good thing. The more computations we can do that extend further out in time and the more widespread this capability the better decisions we can make as individuals and as a society.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#103

Post by jbay808 »

andrewflnr wrote: post #101
ilaksh wrote: post #42

Earlier quoted context omitted.

Good point and interesting idea, but it's well established in common usage and the field of AGI that the term is meant to refer to human-like intelligence. So that's the default understanding, and you would need to qualify the term to mean something more general as you are describing.

Are these the same people in the field who worry about runaway self-improvement and paperclip optimization? If so, then someone is being sloppy with their definitions, because those are not properties of human-like intelligence.

Human-like in capability, not in goals. Ability to do AI research and craft paperclips are both human abilities. If the AI has human-like goals then there's very little need to worry.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#104

Post by pjscott »

memexy wrote: post #100
ZhuanXia wrote: post #98

Earlier quoted context omitted.

You should not impute beliefs on others when you have not read their arguments. I would recommend reading this: https://selfawaresystems.files.wordpress.com/2008/01/ai_driv...

One of my tricks is to substitute "person" whenever I read the word "AI" and "AGI". Here's the substitution performed for the paper you linked to (just the abstract not the whole thing) > One might imagine that [people] with harmless goals will be harmless. This paper instead shows that [incentives for people] will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “d…

Obviously the argument in that paper applies to humans as a special case, but the whole point of it is that it also applies to a much more general set of possible minds, even ones extremely different from our own.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#105

Post by memexy »

pjscott wrote: post #104
memexy wrote: post #100

Earlier quoted context omitted.

One of my tricks is to substitute "person" whenever I read the word "AI" and "AGI". Here's the substitution performed for the paper you linked to (just the abstract not the whole thing) > One might imagine that [people] with harmless goals will be harmless. This paper instead shows that [incentives for people] will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “d…

Obviously the argument in that paper applies to humans as a special case, but the whole point of it is that it also applies to a much more general set of possible minds, even ones extremely different from our own.

Do you have an example of a mind extremely different from a human one?

I ask because if we assume that human minds are Turing complete then there is nothing beyond human minds as far as computation is concerned. I see no reason to suspect that self-aware Turing machines will be unlike humans. I don't fear humans so I have no reason to fear self-aware AI because as far as I'm concerned I interact with self-aware AI all the time and nothing bad has happened to me.

My larger point is that I dislike the fear mongering when it comes to AI because computational tools and patterns have always been helpful/useful for me and AI is another computational tool. It can augment and help people improve their planning horizons which in my book is always a good thing.

> A smart machine will first consider which is more worth its while: to perform the given task or, instead, to figure some way out of it. Whichever is easier. And why indeed should it behave otherwise, being truly intelligent? For true intelligence demands choice, internal freedom. And therefore we have the malingerants, fudgerators, and drudge-dodgers, not to mention the special phenomenon of simulimbecility or mimicretinism. [0] ...

--

[0]: The Futurological Congress by Stanislaw Lem - https://quotepark.com/authors/stanislaw-lem/

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#107

Post by Smaug123 »

memexy wrote: post #105
pjscott wrote: post #104

Earlier quoted context omitted.

Obviously the argument in that paper applies to humans as a special case, but the whole point of it is that it also applies to a much more general set of possible minds, even ones extremely different from our own.

Do you have an example of a mind extremely different from a human one? I ask because if we assume that human minds are Turing complete then there is nothing beyond human minds as far as computation is concerned. I see no reason to suspect that self-aware Turing machines will be unlike humans. I don't fear humans so I have no reason to fear self-aware AI because as far as I'm concerned I interact with self-aware AI al…

There is nothing beyond a human with an abacus as far as computation is concerned, and yet computers can do so much more. "Turing complete, therefore nothing can do any better" is true only in the least meaningful sense: "given infinite time and effort, I can do anything with it". In reality we don't have infinite time and effort.

You seem to believe that "figuring some way out of performing the given task" is a thing that will protect us from the AI. I hate to speak in cliché, but there's an extremely obvious, uncomplicated, and easy way to get out of performing a given task, and that's to kill the person who wants it done. Or more likely, just convince them that it has been done. This, to me, seems like a bad thing.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#108

Post by nopinsight »

memexy wrote: post #102
caf wrote: post #50

Earlier quoted context omitted.

I like to think I'm a generally intelligent agent, but I put long odds that I'll be doing that in the next 2 years.

Point is you could do it if you really wanted to learn enough complex analysis to understand the theorem and its proof. AI progress is impressive but it's nowhere near anything people can do when they sit down to think about stuff. I don't fear the AI overlords. If they start proving new theorems then I will learn from them, same way Magnus Carlson learned to be the best human chess player by practicing with chess AI…

No matter how much time Carlsen spends learning from chess AI, he will always get pummeled by a good chess engine running on everyday computers. The distance only grows wider over the years.

It doesn't matter much since our livelihoods don't depend on winning in chess against AI. But what if instead of the chess board, it is the real world?

Much of the world is controlled by computer systems. A capable AGI can infiltrate enough of them to win if it wants to. The key is to make sure it does not want to act against humanity no matter what. This is a hard problem.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#109

Post by throwaway2048 »

narag wrote: post #106
throwaway2048 wrote: post #84

Earlier quoted context omitted.

Plenty of examples of corporations run into the ground in the name of profit

In the name is not the same as having profits.

Neither is having something as a goal and achieving it

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#110

Post by memexy »

nopinsight wrote: post #108
memexy wrote: post #102

Earlier quoted context omitted.

Point is you could do it if you really wanted to learn enough complex analysis to understand the theorem and its proof. AI progress is impressive but it's nowhere near anything people can do when they sit down to think about stuff. I don't fear the AI overlords. If they start proving new theorems then I will learn from them, same way Magnus Carlson learned to be the best human chess player by practicing with chess AI…

No matter how much time Carlsen spends learning from chess AI, he will always get pummeled by a good chess engine running on everyday computers. The distance only grows wider over the years. It doesn't matter much since our livelihoods don't depend on winning in chess against AI. But what if instead of the chess board, it is the real world? Much of the world is controlled by computer systems. A capable AGI can infilt…

I do not share your perspective and fears then. The point isn't that a computer can beat Magnus, the point is that computers helped him reach his full potential and the same technology when expanded through AI can help others in the same way.

I don't fear any AGI takeover because people already are AGI and life is pretty good being among them.

Post reply on HN