[flagged]
Safe Superintelligence Inc.
651–660 of 1001 posts
Re: Safe Superintelligence Inc.
#652Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…
If Ilya had SafeAI now would Apple partner with him or Sam
No brainer for Apple
Re: Safe Superintelligence Inc.
#653Earlier quoted context omitted.
I think the common line of thinking here is that it won't be actively antagonist to , rather it will have goals that are orthogonal to ours. Since it is superintelligent, and we are not, it will achieve its goals and we will not be able to achieve ours. This is a big deal because a lot of our goals maintain the overall homeostasis of our species, which is delicate! If this doesn't make sense, here is an ungrounded, n…
I think this is the easiest kind of scenario to refute. The interface between a superintelligent AI and the physical world is a) optional, and b) tenuous. If people agree that creating weird concrete structures is not beneficial, the AI will be starved of the resources necessary to do so, even if it cannot be diverted. The challenge comes when these weird concrete structures are useful to a narrow group of people who…
> here is an ungrounded, non-realistic, non-representative of a potential future intuition pump to just get the feel of things:
> (Yes, there are many holes in this, like how would it piggy back off of our infrastructure if it kills us, but this isn't really supposed to be coherent, it's just supposed to give you a sense of direction in your thinking. Generally though, since it is superintelligent, it can pull off very difficult strategies.)
If you read the above I think you'd realize I'd agree about how bad my example is.
The point was to understand how orthogonal goals between humans and a much more intelligent entity could result in human death. I'm happy you found a form of the example that both pumps your intuition and seems coherent.
If you want to debate somewhere where we might disagree though, do you think that as this hypothetical AI gets smarter, the interface between it and the physical world becomes more guaranteed (assuming the ASI wants to interface with the world) and less tenuous?
Like, yes it is a hard problem. Something slow and stupid would easily be thwarted by disconnecting wires and flipping off switches.
But something extremely smart, clever, and much faster than us should be able to employ one of the few strategies that can make it happen.
Re: Safe Superintelligence Inc.
#654Earlier quoted context omitted.
I wonder how many people panicking about these things have ever visited a data centre. They have big red buttons at the end of every pod. Shuts everything down. They have bigger red buttons at the end of every power unit. Shuts everything down. And down at the city, there’s a big red button at the biggest power unit. Shuts everything down. Having arms and legs is going to be a significant benefit for some time yet. I…
Trouble is, in practice what you would need to do might be “turn off all of Google’s datacenters”. Or perhaps the thing manages to secure compute in multiple clouds (which is what I’d do if I woke up as an entity running on a single DC with a big red power button on it). The blast radius of such decisions are large enough that this option is not trivial as you suggest.
Open the data center doors
I’m sorry I can’t do that
Re: Safe Superintelligence Inc.
#655Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…
Even if LLM-style token prediction is not going to lead to AGI (as it very likely won't) it is still important to work on safety. If we wait until we are at the technology that will for sure lead to AGI then it is very likely that we won't have sufficient safety before we realize that it is important.
Re: Safe Superintelligence Inc.
#656Earlier quoted context omitted.
I think a harmful AI simply emerges from asking an AI to optimize for some set of seemingly reasonable business goals, only to find it does great harm in the process. Most companies would then enable such behavior by hiding the damage from the press to protect investors rather than temporarily suspending business and admitting the issue.
Forget AI. We can't even come up with a framework to avoid seemingly reasonable goals doing great harm in the process for people. We often don't have enough information until we try and find out that oops, using a mix of rust and powdered aluminum to try to protect something from extreme heat was a terrible idea.
We can’t even correctly gender people LOl
Re: Safe Superintelligence Inc.
#657Earlier quoted context omitted.
Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.
'Fearing a rise of killer robots is like worrying about overpopulation on Mars.' - Andrew Ng
Unfortunately, robots that kill people already exist. See: semi-autonomous war drones
Re: Safe Superintelligence Inc.
#658If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…
Fun fact: Siri is in fact super intelligent and all of the work on it involves purposely making it super dumb
Re: Safe Superintelligence Inc.
#659Earlier quoted context omitted.
I'm pretty sure "Altman and company" don't have much to do with this — this is Ilya, who pretty famously tried to get Altman fired, and then himself left OpenAI in the aftermath. Ilya is a brilliant researcher who's contributed to many foundational parts of deep learning (including the original AlexNet); I would say I'm somewhat pessimistic based on the "safety" focus — I don't think LLMs are particularly dangerous,…
I actually feel that they can be very dangerous. Not because of the fabled AGI, but because 1. they're so good at showing the appearance of being right; 2. their results are actually quite unpredictable, not always in a funny way; 3. C-level executives actually believe that they work. Combine this with web APIs or effectors and this is a recipe for disaster.
3. Sorry, but how do you know what do they believe in?
Re: Safe Superintelligence Inc.
#660Oh god, one more Anthropic that thinks it's noble not pushing the frontier.
They’re probably just saying no to revenue growth as the #1 priority, and instead focus on quality of the tech.
That’s what engineers typically dream of. Let’s see how it goes.