Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

281–290 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#281

Post by rspeer »

ikeboy wrote: post #134

If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe? Also, >AI alarmists believe in something called the Orthogonality Thesis. This says that even very complex beings can have simple motivations, like the paper-clip maximizer. Uh, no. The point of the paper clip maximizer i…

> That's not really a premise. A better version would be "a likely path to super-intelligence will be a self-improving agent".

If I understand your claim, it's not very relevant. You seem to be saying that, given that superintelligence happens, the probability that it will have happened via a self-improving agent is high.

That doesn't refute the claim that "superintelligence probably will not happen".

> You have a point that we shouldn't be ignoring other more important things for this, but to argue against increasing spending from 8 figures to 9 figures you need better arguments.

Why don't we spend that money on non-vague problems that actually face us now? For example, unbiased AI and explainable AI are topics of research that would help us right now and don't require wacky far-future predictions.

Re: Superintelligence: The Idea That Eats Smart People

#282

Post by paulbaumgart »

TeMPOraL wrote: post #256
paulbaumgart wrote: post #161

Earlier quoted context omitted.

That even if you see diminishing returns to what you can accomplish with greater intelligence, a small advantage will still compound into a large one over the theoretically infinite lifespan of an AI.

s/theoretically/practically/, and I doubt it changes the conclusion that much (in practice).

I'll settle for "effectively." Deal?

Re: Superintelligence: The Idea That Eats Smart People

#283

Post by fjdlwlv »

SerLava wrote: post #60
md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

I don't understand why people have such a weird problem reconciling the brain with the mind. It IS all physical. It's also an unimaginably complex fucking shitload of tiny physical objects working incredibly quickly. If your brain was big enough to see the parts working like little gears of a clock, it would probably be planet sized, or something like that. Huge. I would EVEN say it doesn't raise any interesting phil…

> I don't understand why people have such a weird problem reconciling the brain with the mind.

You do understand, you said it yourself! Because they have no free will to not have the problem!

I can't blame you though. You had no choice.

BTW,you are the first person I know who admits being a Compatibilist. I don't understand how you can rationally take that position.

Re: Superintelligence: The Idea That Eats Smart People

#284

Post by rntz »

maxerickson wrote: post #275
rntz wrote: post #252

Earlier quoted context omitted.

1. Plenty of academics write books. 2. Comparing a paper and a book for length is obviously unfair. Bostrom has also written papers: https://en.wikipedia.org/wiki/Nick_Bostrom#Journal_articles_... 3. "Concrete theory" is vague. Is it a stand-in for "I won't accept any argument by a philosopher, only physicists need apply"?

I'm not intentionally trying to snub philosophy. With concrete theory, the point I was reaching for is that when you look to measure impact you probably want to point back to a compact articulation of an idea. The book comparison was probably a cheap shot (on the other side of it, Einstein didn't need popular interest/approval for his ideas to matter; I think that is a positive). I think as much as anything the compa…

Sure, that's fair. I think Bostrom is no Einstein. But I maintain that he's no crank, either. There's a lot of space in the world for people who are neither.

Re: Superintelligence: The Idea That Eats Smart People

#285

Post by TeMPOraL »

mrob wrote: post #57

Wooly Definitions: The only definition that matters is ability to make correct predictions about the future. All other things called "intelligence" are a consequence of this. This definition is formalized with AIXI, which is uncomputable, but computable approximations exist. Hawking's Cat, Einstein's Cat: Scams require some intelligence from the victim. The victim needs to mistakenly believe they're doing something s…

> Hawking's Cat, Einstein's Cat: Scams require some intelligence from the victim. The victim needs to mistakenly believe they're doing something smart.

Only if your scam is too complicated for the victim :). Cats are pretty easy to scam - you just have to stick to simple things. Cats respond predictably to food they like. As a cat person, I find this trick to be more than enough for all my needs ;).

This nitpick aside, I second all your points.

> If this turns out to be needed, why can't increasing intelligence result in increased ability to simulate an environment suitable for raising superintelligences?

Indeed, we could feed the required stimuli to an AI faster than real-time. VR is a thing already, and so is the "fast-forward" button.

Re: Superintelligence: The Idea That Eats Smart People

#286

Post by TeMPOraL »

paulbaumgart wrote: post #282
TeMPOraL wrote: post #256

Earlier quoted context omitted.

s/theoretically/practically/, and I doubt it changes the conclusion that much (in practice).

I'll settle for "effectively." Deal?

Deal.

My point being, there's plenty of time before coming heat death starts to cause problems :).

Re: Superintelligence: The Idea That Eats Smart People

#287

Post by fjdlwlv »

Afforess wrote: post #159

> Observe that in these scenarios the AIs are evil by default, just like a plant on an alien planet would probably be poisonous by default. I believe this is a core misunderstanding. Bostrom never says that a superintelligent AI is evil by default. Bostrom argues the AI will be orthogonal, it's goals will be underspecified in such a way that leads it to destroy humanity. The paperclip optimizer AI doesn't want to kil…

The emergent system of human society already does this (read The Wealth of Nations) and you gave some examples yourself,so why (aside from amusement) should we care about dangerous Superintelligence when we already have so many out of control processes heading towards destroying our habitats, societies, and selves?

Re: Superintelligence: The Idea That Eats Smart People

#288

Post by Eliezer »

idlewords wrote: post #62
mrfusion wrote: post #28

I've always wondered if the problems an intelligence solves are exponentially hard so even if we build a super intelligence it wouldn't be all that much smarter than we are. For example compare how many more cities in the traveling salesman problem a super computer can solve vs your grandmas pc. It's more but surprisingly not all that many more. What do you think of that idea?

I think this basic concept of intractability, which programmers are very familiar with, hasn't penetrated far enough into AI world. Bostrom and Yudkowsky in particular seem happy to hand-wave past computational complexity.

I'm getting kind of sick of people who haven't read 2% of the stuff imagining what they think we've never talked about over the last 16 years.

https://intelligence.org/files/IEM.pdf

Post reply on HN