Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc. – Ilya Sutskever

twitter.com

Post reply on HN
Print view View original

21–30 of 34 posts

Re: Safe Superintelligence Inc. – Ilya Sutskever

#21

Post by peddling-brink »

philwelch wrote: post #8
swyx wrote: post #4

Reactions: 1. good for him. there was obviously infinity money to back him in achieving his goals 2. i have the same problem with this as i did the original version of anthropic - you cannot achieve "unilateral ai safety". either the danger is real and humanity is not safe until all labs are locked down, or the danger is not real and theres nothing really to discuss. Anthropic already tried the "we are openai but saf…

> i have the same problem with this as i did the original version of anthropic - you cannot achieve "unilateral ai safety". either the danger is real and humanity is not safe until all labs are locked down, or the danger is not real and theres nothing really to discuss. It’s not clear to me that this would be true. Remember that there’s a lot of vagueness about the specific AI risk model, aside from the general idea…

I’m not sure good and evil are well enough defined to hope for this. A machine super intelligence would be unknowable to us. Gorillas vs Homo sapiens.

Re: Safe Superintelligence Inc. – Ilya Sutskever

#22

Post by philwelch »

peddling-brink wrote: post #21
philwelch wrote: post #8

Earlier quoted context omitted.

> i have the same problem with this as i did the original version of anthropic - you cannot achieve "unilateral ai safety". either the danger is real and humanity is not safe until all labs are locked down, or the danger is not real and theres nothing really to discuss. It’s not clear to me that this would be true. Remember that there’s a lot of vagueness about the specific AI risk model, aside from the general idea…

I’m not sure good and evil are well enough defined to hope for this. A machine super intelligence would be unknowable to us. Gorillas vs Homo sapiens.

From the gorilla’s perspective, humans who preserve gorilla habitats are preferable to humans who hunt gorillas for sport. But gorillas have tribes of their own[1] and if gorillas could invent humans, some of them might invent humans who hunt the enemy tribe while helping theirs. So if you were a gorilla from the rival tribe and you had nuanced ethics of your own, you might even try to invent humans that try to stop the enemy humans without necessarily exterminating the enemy tribe.

[1] I don’t know if gorillas specifically have been observed engaging in tribal warfare, but chimpanzees have; the Gombe Chimpanzee War for instance.

Re: Safe Superintelligence Inc. – Ilya Sutskever

#26

Post by ok_dad »

dustfinger wrote: post #23

> Superintelligence is within reach. Don't we need to achieve general artificial intelligence first? Or is this "super" intelligence just that? To me, super intelligence would be beyond our own and therefore beyond general AI as well.

Once you have AGI, you can dedicate more cycles to it and the theory is (as far as I can tell) that you’ll get an exponential increase in “intelligence” which is basically just “thinking faster”. Personally, I don’t buy that fully, I think there are some problems that can only be solved by any intelligence at the speed of time in the real world. Certainly more cycles to dedicate to mathematics would advance that field more rapidly, but for a lot of biological puzzles you probably can’t accelerate time like this.

Re: Safe Superintelligence Inc. – Ilya Sutskever

#27

Post by dustfinger »

ok_dad wrote: post #26
dustfinger wrote: post #23

> Superintelligence is within reach. Don't we need to achieve general artificial intelligence first? Or is this "super" intelligence just that? To me, super intelligence would be beyond our own and therefore beyond general AI as well.

Once you have AGI, you can dedicate more cycles to it and the theory is (as far as I can tell) that you’ll get an exponential increase in “intelligence” which is basically just “thinking faster”. Personally, I don’t buy that fully, I think there are some problems that can only be solved by any intelligence at the speed of time in the real world. Certainly more cycles to dedicate to mathematics would advance that fiel…

> Once you have AGI, you can...

So if Superintelligence depends on AGI, then we can't say:

> Superintelligence is within reach.

until we can say AGI is within reach. Is there any substantive evidence supporting the revised claim? Do we even have a reliable way to evaluate if an AI is a AGI? The standard answer was the Turing Test, but in recent years there have been multiple claims that it has been broken by non-AGI [1].

To me, AGI still feels a long ways off.

[1]:

- https://www.nature.com/articles/d41586-023-02361-7

- https://www.washingtonpost.com/technology/2022/06/17/google-...

Re: Safe Superintelligence Inc. – Ilya Sutskever

#28

Post by ok_dad »

dustfinger wrote: post #27
ok_dad wrote: post #26

Earlier quoted context omitted.

Once you have AGI, you can dedicate more cycles to it and the theory is (as far as I can tell) that you’ll get an exponential increase in “intelligence” which is basically just “thinking faster”. Personally, I don’t buy that fully, I think there are some problems that can only be solved by any intelligence at the speed of time in the real world. Certainly more cycles to dedicate to mathematics would advance that fiel…

> Once you have AGI, you can... So if Superintelligence depends on AGI, then we can't say: > Superintelligence is within reach. until we can say AGI is within reach. Is there any substantive evidence supporting the revised claim? Do we even have a reliable way to evaluate if an AI is a AGI? The standard answer was the Turing Test, but in recent years there have been multiple claims that it has been broken by non-AGI…

Yea I’m saying the same thing, you need AGI to get whatever “superintelligence” is. That’s mostly my opinion, but I wouldn’t call it SI without AGI. Personally, I’m no AI expert, but we’re very far from AGI unless one of these experts knows things I don’t about current research. LLMs are interesting, same for other large models like vision, but they don’t impress me with their abilities yet. I’m be impressed when they can replace someone other than content farmers.

Re: Safe Superintelligence Inc. – Ilya Sutskever

#29

Post by bykhun »

015a wrote: post #15
swyx wrote: post #4

Reactions: 1. good for him. there was obviously infinity money to back him in achieving his goals 2. i have the same problem with this as i did the original version of anthropic - you cannot achieve "unilateral ai safety". either the danger is real and humanity is not safe until all labs are locked down, or the danger is not real and theres nothing really to discuss. Anthropic already tried the "we are openai but saf…

Yeah, I think that 2nd point is really critical; I'm pretty plugged into this space, and its really not obvious to me what "safety" even means in the context of a superintelligence. Is it just, e.g., "filtering the AI so it never tells users how to build nukes"? Is it more of a firewall to stop sentient process/VM/computer escape [1]? Its a redundant question with no answer because we don't even know what the threat…

> I'd also argue that the biggest threat from ASI is what I've heard Roman Yampolskiy label as "ikigai-doom"; that AI could become so much better than humans at all the things humans do, that even in the best case humanity is left with no purpose

I bet status games will stay. Robots may be sexual partners, CEOs and therapists, but they'd never take on status roles in our society – only utility roles.

Same as we do Olympics, even though machines are much better at throwing and lifting than us – we do it to win approval of others

Post reply on HN