I'm yet to find a convincing definition for super intelligence. "Intelligence that surpasses human brain" is such a half hearted definition that I don't know what to make of it.
Safe Superintelligence Inc. – Ilya Sutskever
31–34 of 34 posts
Re: Safe Superintelligence Inc. – Ilya Sutskever
#32I’m glad someone is focusing on safety but, to me, opening an office in Tel Aviv given the genocide happening in Gaza and the mandatory conscription of Israeli citizens does not seem like a good fit.
finally someone pointed it out.
Re: Safe Superintelligence Inc. – Ilya Sutskever
#33Earlier quoted context omitted.
Once you have AGI, you can dedicate more cycles to it and the theory is (as far as I can tell) that you’ll get an exponential increase in “intelligence” which is basically just “thinking faster”. Personally, I don’t buy that fully, I think there are some problems that can only be solved by any intelligence at the speed of time in the real world. Certainly more cycles to dedicate to mathematics would advance that fiel…
> Once you have AGI, you can... So if Superintelligence depends on AGI, then we can't say: > Superintelligence is within reach. until we can say AGI is within reach. Is there any substantive evidence supporting the revised claim? Do we even have a reliable way to evaluate if an AI is a AGI? The standard answer was the Turing Test, but in recent years there have been multiple claims that it has been broken by non-AGI…
> So if Superintelligence depends on AGI,
Not sure, is superintelligence generic, alive. Does it have motivation, goals? Sounds like a big calculator which he personally will control. Technically it's possible. LLMs aren't the end of AI evolution.
> To me, AGI still feels a long ways off.
It depends on how you define it. Just a couple of years back current LLMs' level was like from sci-fi. But here we are and progress is accelerating. I wonder how many labs and individuals are working on AGI trying to avoid any attention. It feels like any time now. May be something subhuman first.
Re: Safe Superintelligence Inc. – Ilya Sutskever
#34Earlier quoted context omitted.
Yeah, I think that 2nd point is really critical; I'm pretty plugged into this space, and its really not obvious to me what "safety" even means in the context of a superintelligence. Is it just, e.g., "filtering the AI so it never tells users how to build nukes"? Is it more of a firewall to stop sentient process/VM/computer escape [1]? Its a redundant question with no answer because we don't even know what the threat…
> I'd also argue that the biggest threat from ASI is what I've heard Roman Yampolskiy label as "ikigai-doom"; that AI could become so much better than humans at all the things humans do, that even in the best case humanity is left with no purpose I bet status games will stay. Robots may be sexual partners, CEOs and therapists, but they'd never take on status roles in our society – only utility roles. Same as we do Ol…
Effectively what you've just said is: You can still do all these pursuits, as long as you're literally the best in the world. That's not really helpful; that's still ikigai-doom.
The point is: If your passion is animation, you can probably find work doing that today. It might suck, certainly more than in times past, and its hard, and that's an intrinsic part of passion. When AI can do that, maybe the best animators in the world will still find work at some bespoke studio making "the authentic hand-made stuff", like a farmers market, but AI may make everything else. And it may be, not just because AI could produce it "better" (it may or may not), but definitely because AI will produce it cheaper. Capitalism doesn't generally care about quality.
And the same goes for many careers: the ironic thing is that AI probably won't come for the plumbers and janitors very quickly, and there seems to be some kind of weird correlation between the jobs people find high passion in, and the jobs AI is likely to replace. In effect, we're evolving into a world where humans are relegated to manual labor, while AI handles everything else, poorly, but humans can't actually make a living doing that other stuff because AI does it so much cheaper and good enough.
But sure; there will still be millionaire status celebs.