What I want to know about Illya Sutskever is whether he's related to the great Yiddish poet, Avrom Sutzkever?
Safe Superintelligence Inc.
311–320 of 1001 posts
Re: Safe Superintelligence Inc.
#312This is a great opportunity to present my own company which is also working on developing not just a super intelligence but an ultra genius intelligence with a patented and trademarked architecture called the panoptic computronium cathedral™. We are so focused on development that we didn't even bother setting up an announcement page because it would have taken time away from the most important technical problem of our time and every nanosecond counts when working on such an important task. My days are structured around writing code and developing the necessary practices and rituals for the coming technological god which will be implemented with mathematics on GPUs. If anyone wants to work on the development of this god then I will post a job announcement at some point and spell out the requirements for what it takes to work at my company.
Re: Safe Superintelligence Inc.
#313I get what Ilya is trying to do, and I'm not against it. But I think safety is a reputation you earn. Having "Safe" in a company name is like having "Democratic" in a country name.
Re: Safe Superintelligence Inc.
#314Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?
I'll bite... "a"
Re: Safe Superintelligence Inc.
#315Earlier quoted context omitted.
Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.
'Fearing a rise of killer robots is like worrying about overpopulation on Mars.' - Andrew Ng
Sentient killer robots is not the risk most AI researchers are worried about. The risk is what happens as corporations give AI ever larger power over significant infrastructure and marketing decisions.
Facebook is an example of AI in it's current form already doing massive societal damage. It's algorithms optimize for "success metrics" with minimal regard for consequences. What happens when these algorithms are significantly more self modifying? What if a marketing campaign realizes a societal movement threatens it's success? Are we prepared to weather a propaganda campaign that understands our impulses better than we ever could?
Re: Safe Superintelligence Inc.
#316This won't age well.
I disagree. Life is short. It's fun to be a little hyperbolic once in a while.
Re: Safe Superintelligence Inc.
#317And now we have our answer. sama said that Ilya was going "to start something that was personally important to him." Since that thing is apparently AI safety, we can assume that that is not important to OpenAI.
This only makes sense if OpenAI just doesn't believe AGI is a near-term-enough possibility to merit their laser focus right now, when compared to investing in R&D that will make money from GPT in a shorter time horizon (2-3 years).
I suppose you could say OpenAI is being irresponsible in adopting that position, but...come on guys, that's pretty cynical to think that a company AND THE MAJORITY OF ITS EMPLOYEES would all ignore world-ending potential just to make some cash.
So in the end, this is not necessarily a bad thing. This has just revealed that the boring truth was the real situation all along: that OpenAI is walking the fine line between making rational business decisions in light of the far-off time horizon of AGI, and continuing to claim AGI is soon as part of their marketing efforts.
Companies in the end are predictable!
Re: Safe Superintelligence Inc.
#318Earlier quoted context omitted.
Since no one knows how to build an AGI, hard to say. But you might imagine that more restricted goals could end up being easier to accomplish. A "safe" AGI is more focused on doing something useful than figuring out how to take over the world and murder all the humans.
Hinton's point does make sense though. Even if you focus an AGI on producing more cars for example, it will quickly realize that if it has more power and resources it can make more cars.
Assuming AGI works like a braindead consulting firm, maybe. But if it worked like existing statistical tooling (which it does, today, because for an actual data scientist and not aunt cathy prompting bing, using ml is no different than using any other statistics when you are writing your python or R scripts up), you could probably generate some fancy charts that show some distributions of cars produced under different scenarios with fixed resource or power limits.
In a sense this is what is already done and why ai hasn't really made the inroads people think it will even if you can ask google questions now. For the data scientists, the black magicians of the ai age, this spell is no more powerful than other spells, many of which (including ml) were created by powerful magicians from the early 1900s.
Re: Safe Superintelligence Inc.
#319Earlier quoted context omitted.
I think the idea is that a safe super intelligence would help solve those problems. I am skeptical because the vast majority are social coordination problems, and I don’t see how a machine intelligence no matter how smart can help with that.
By any means necessary I presume. If Russian propaganda helped get Trump elected, AI propaganda could help social coordination by influencing public perception of issues and microtargeting down to the individual level to get people on board.
could
but it's owners might have a vested interest in influencing public perceptions to PREVENT positive social outcomes and favor the owners financial interests.
(seems rather more likely, given who will/would own such a machine)
Re: Safe Superintelligence Inc.
#320I'm just glad Google didn't start with DoNoEvil Inc.
StabilityAI and OpenAI ruined it.