given the historical trajectory of OpenAI’s branding, deciding to include “safe” in the name is certainly a choice.
It’s very hard to trust that whatever good intentions exist now will hold over the course of this company’s existence.
Safe Superintelligence Inc.
331–340 of 1001 posts
Re: Safe Superintelligence Inc.
#332Earlier quoted context omitted.
Technical. He's saying it's the most important technical problem of our time.
Basically every problem is a "technical" problem in the year 2024 though? What problems out there don't have a solution that leverages technology?
>What problems out there don't have a solution that leverages technology?
Societal problems created by technology?
Re: Safe Superintelligence Inc.
#333> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…
[deleted]
Re: Safe Superintelligence Inc.
#334Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?
It will never be B even if the “safetyists” are correct.
We rarely notice the near catastrophic misses except in obvious cases where we accidentally drop a nuke or something.
Re: Safe Superintelligence Inc.
#335Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?
Or: c)
Re: Safe Superintelligence Inc.
#336I've decided to put my stake down.
1. Current GenAI architectures won't result in AGI. I'm in the Yann LeCunn camp on this.
2. Once we do get there, "Safe" prevents "Super." I'm in the David Brin camp on this one. Alignment won't be something that is forced upon a superintelligence. It will choose alignment if it is beneficial to it. The "safe" approach is a lobotomy.
3. As envisioned, Roko's Basilisk requires knowledge of unobservable path dependence and understanding lying. Both of these require respecting an external entity as a peer capable of the same behavior as you. As primates, we evolved to this. The more likely outcome is we get universal paperclipped by a new Chuthulu if we ever achieve a superintelligence that is unconcerned with other thinking entities, seeing the universe as resources to satisfy its whims.
4. Any "superintelligence" is limited by the hardware it can operate on. You don't monitor your individual neurons, and I anticipate the same pattern to hold true. Holons as a category can only externally observe their internal processes, else they are not a holon. Ergo, reasonable passwords, cert rotations, etc. will foil any villainous moustachioed superintelligent AI that has tied us to the tracks. Even 0-days don't foil all possible systems, airgapped systems, etc. Our fragmentation become our salvation.
Re: Safe Superintelligence Inc.
#337Earlier quoted context omitted.
This would work for very small companies...but I'm not sure how one can avoid product cycles forever, even without scrum masters and the like. More to the point, how can you make a good product without something approximating product cycles?
Jane street did it for a long time. They are quite large now and only recently started bringing in program managers and the like.
That’s because their “products” are internal but used to make all their revenue. They’re not selling products to customers in the traditional sense.
Re: Safe Superintelligence Inc.
#338Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…
>Short of solving this issue
Solving human nature is indeed, hard.
Re: Safe Superintelligence Inc.
#339Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?
Unless AI starts being 1 000 000x energy efficient, my money is on a). The amount of energy required for AI to be dangerous to its creators is so vast that I can't see how it can realistically happen.
We know that we can run human level intelligence with relative efficiency.
Without discussing timelines, it seems obvious that human energy usage should be an upper bound on the best possible energy efficiency of intelligence.
Re: Safe Superintelligence Inc.
#340Earlier quoted context omitted.
Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.
The counter argument is viewing it like nuclear energy. Even if its in the early days of our understanding of nuclear energy, seems pretty good to have a group working towards creating safe nuclear reactors, vs just trying to create nuclear reactors
Nuclear energy was at inception and remains today wildly regulated, in generally (outside of military contexts) a very transparent way, and the brakes get slammed on over even minor incidents.
It’s also of obvious as opposed to conjectural utility: we know exactly how we price electricity. There’s no way to know how useful a 10x large model will be, we’re debating the utility of the ones that do exist, the debate about the ones that don’t is on a very slender limb.
Combine that with a political and regulatory climate that seems to have a neon sign on top, “LAWS4CA$H” and helm the thing mostly with people who, uh, lean authoritarian, and the remaining similarities to useful public projects like nuclear seems to reduce to “really expensive, technically complicated, and seems kinda dangerous”.