Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

341–350 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#341

Post by wyager »

tvural wrote: post #326
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

People are worried about AI risk because ensuring that the strong AI you build to do X will do X without doing something catastrophic to humanity instead is a very hard problem, and people who have not thought much about this problem tend to vastly underestimate how hard it is. Whatever goals the AI has, it will certainly be better at achieving them if it can stay alive. And it will be more likely to stay alive if th…

> if you are given a strong AI randomly selected from the space of all strong AIs

Why would this ever apply? We're building them, not picking them out of a hat.

Re: Superintelligence: The Idea That Eats Smart People

#342

Post by SerLava »

fjdlwlv wrote: post #283
SerLava wrote: post #60

Earlier quoted context omitted.

I don't understand why people have such a weird problem reconciling the brain with the mind. It IS all physical. It's also an unimaginably complex fucking shitload of tiny physical objects working incredibly quickly. If your brain was big enough to see the parts working like little gears of a clock, it would probably be planet sized, or something like that. Huge. I would EVEN say it doesn't raise any interesting phil…

> I don't understand why people have such a weird problem reconciling the brain with the mind. You do understand, you said it yourself! Because they have no free will to not have the problem! I can't blame you though. You had no choice. BTW,you are the first person I know who admits being a Compatibilist. I don't understand how you can rationally take that position.

>BTW,you are the first person I know who admits being a Compatibilist.

Not really - I'm determinist, but I just know the difference between "Things are technically predetermined" and "I'm being forced to do something, like at gunpoint." As in, it's not demoralizing because that makes no sense.

I like it when my brain makes a decision, and I don't care that my brain isn't pulling decisions magically from a mysterious dimension full of floaty ghosts.

Re: Superintelligence: The Idea That Eats Smart People

#343

Post by aaachilless »

tptacek wrote: post #330
aaachilless wrote: post #100

Earlier quoted context omitted.

What if you're the first human on an island full of baby tigers? I think most AI alarmists would argue that this analogy is vastly more appropriate.

The idea here is that the person has to kill all the baby tigers, right? Because otherwise the end state is the same as the island full of adult tigers.

I was thinking that if you were _dropped_ onto an island full of tigers you'd have no chance, but that you could figure out a way to survive if you had some time to figure out a plan. Maybe you could find a way to coexist with the tigers. Become one of the pack, y'know?

Re: Superintelligence: The Idea That Eats Smart People

#345

Post by jmagoon »

kobayashi wrote: post #279
mundo wrote: post #151

Earlier quoted context omitted.

People will stop downvoting this if you edit it to add a specific example or three.

Too late to edit, so I'll post just a few examples here: >The only way out of this mess is to design a moral fixed point, so that even through thousands and thousands of cycles of self-improvement the AI's value system remains stable, and its values are things like 'help people', 'don't kill anybody', 'listen to what people want'. Bostrom absolutely did not say that the only way to inhibit a cataclysmic future for hu…

Well, the thing is there is no such thing as 'machine intelligence', so it's all just an assumption on top of an assumption about a thing we don't have a very good grasp of yet.

You're essentially saying that the author is wrong for saying the philosopher's stone can't transmute 100 bars of iron to 100 bars of gold, because a philosopher's stone could absolutely do that type of thing, because that's what philosopher's stones do.

To walk down arguing the merits of this position, why must a machine intelligence 'not share any of that complexity' of a human intelligence? What suggests that intelligence is able to arise absent of complexity? Isn't the only current example of machine intelligence we currently have a property of feeding massive amounts of complex information into a program that gradually adjusts itself to its newly discovered outside world? Or are you suggesting that you could feed singular types of information to something that would then be classified as intelligent?

Re: Superintelligence: The Idea That Eats Smart People

#346

Post by AaronFriel »

wyager wrote: post #341
tvural wrote: post #326

Earlier quoted context omitted.

People are worried about AI risk because ensuring that the strong AI you build to do X will do X without doing something catastrophic to humanity instead is a very hard problem, and people who have not thought much about this problem tend to vastly underestimate how hard it is. Whatever goals the AI has, it will certainly be better at achieving them if it can stay alive. And it will be more likely to stay alive if th…

> if you are given a strong AI randomly selected from the space of all strong AIs Why would this ever apply? We're building them, not picking them out of a hat.

I think the current state of deep neural networking designs and research funding pouring into generalizing the simple neural nets we're working with now suggests that we are, in fact, pulling them out of a hat.

Right now we're just discarding all the ones that are defective, at a stupendously high rate as we train neural nets.

I can't speak to what method would generate the first strong AI, but I suspect the overall process - if not the details - will be similar. Training, discarding, training, discarding, testing, and so on. And the first truly strong AI will likely just be the first random assemblage of parts that passes those tests.

Re: Superintelligence: The Idea That Eats Smart People

#347

Post by SerLava »

lucker wrote: post #254
SerLava wrote: post #60

Earlier quoted context omitted.

I don't understand why people have such a weird problem reconciling the brain with the mind. It IS all physical. It's also an unimaginably complex fucking shitload of tiny physical objects working incredibly quickly. If your brain was big enough to see the parts working like little gears of a clock, it would probably be planet sized, or something like that. Huge. I would EVEN say it doesn't raise any interesting phil…

I don't think there's a problem reconciling the brain with the mind, if by 'mind' one means problem-solving ability. The problem is in reconciling the brain with consciousness . There is, as far as I know, no theory that explains how consciousness can arise from matter.

I don't know that they're really different. It's just a first-person frame of reference.

When something else is conscious, or something else has a mind, that's the same thing.

Re: Superintelligence: The Idea That Eats Smart People

#348

Post by red75prime »

jimmcslim wrote: post #12

'In particular, there's no physical law that puts a cap on intelligence at the level of human beings.' Maybe not, but there are definitely very physical laws governing everything else, that a superintelligent being's ambitions would run into. A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible. More relevant, a superintelligent b…

Regarding enslavement with robots. Superintelligent means that if it is not feasible to enslave humanity with robots, then SAI will find feasible way: human collaborators, genetically engineered fungi, backing of puppet president candidate, fabricating external threat and convincing people that only it can handle it, and so on and so forth.

Re: Superintelligence: The Idea That Eats Smart People

#350

Post by [deleted] »

tvural wrote: post #326
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

People are worried about AI risk because ensuring that the strong AI you build to do X will do X without doing something catastrophic to humanity instead is a very hard problem, and people who have not thought much about this problem tend to vastly underestimate how hard it is. Whatever goals the AI has, it will certainly be better at achieving them if it can stay alive. And it will be more likely to stay alive if th…

[deleted]
Post reply on HN