All the safety freaks should join this and leave openai alone
Safe Superintelligence Inc.
121–130 of 1001 posts
Re: Safe Superintelligence Inc.
#122Earlier quoted context omitted.
The kind of AI researchers being discussed here likely make an order of magnitude more than run of the mill "software engineers".
You're comparing top names with run of the mill engineers maybe, which isn't fair. And maybe you need to discover talent rather than buy talent from the previous generation.
[deleted]
Re: Safe Superintelligence Inc.
#123Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…
> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…
Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.
Re: Safe Superintelligence Inc.
#124Incredible website design, I hope they keep the theme. With so many AI startups going with advanced WebGL/ThreeJS wacky overwhelming animated website designs, the simplicity here is a stark contrast.
Probably Daniel Gross picked it up from Nat Friedman?
1. Nat Friedman has this site: https://nat.org/
2. They made this together: https://nfdg.com/
3. And then this: https://andromeda.ai/
4. Now we have https://ssi.inc/
If you look at the (little) CSS in all of the above sites you'll see there's what seems to be a copy/paste block. The Nat and SSI sites even have the same "typo" indentation.
Re: Safe Superintelligence Inc.
#125> Building safe superintelligence (SSI) is the most important technical problem of our time.
Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really saying that there is a nice total-ordering of problems by importance to the world, and that the one you're interested happens also to be at the top?
Re: Safe Superintelligence Inc.
#126Interesting choice of name. It’s like safe-super-weapon.
Re: Safe Superintelligence Inc.
#127I am not on the bleeding edge of this stuff. I wonder though: How could a safe super intelligence out compete an unrestricted one? Assuming another company exists (maybe OpenAI) that is tackling the same goal without spending the cycles on safety, what chance do they have to compete?
Since no one knows how to build an AGI, hard to say. But you might imagine that more restricted goals could end up being easier to accomplish. A "safe" AGI is more focused on doing something useful than figuring out how to take over the world and murder all the humans.
Re: Safe Superintelligence Inc.
#128> Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures. Can someone explain how their singular focus means they won't have product cycles or management overhead?
Don't hire anyone who is a certified scrum master or has an MBA and you tend to be able to get a lot done.
Re: Safe Superintelligence Inc.
#129This makes sense. Ilya can probably raise practically unlimited money on his name alone at this point. I'm not sure I agree with the "no product until we succeed" direction. I think real world feedback from deployed products is going to be important in developing superintelligence. I doubt that it will drop out of the blue from an ivory tower. But I could be wrong. I definitely agree that superintelligence is within…
I have a strong intuition that chat logs are actually the most useful kind of data. They contain many LLM outputs followed by implicit or explicit feedback, from humans, from the real world, and from code execution. Scaling this feedback to 180M users and 1 trillion interactive tokens per month like OpenAI is a big deal.
Except LLMs are a distraction from AGI
Re: Safe Superintelligence Inc.
#130At this point, all the computing power is concentrated among various companies such as Google, Facebook, Microsoft, Amazon, Tesla, etc. It seems to me it would be much safer and more intelligent to create a massive model and distribute the benefits among everyone. Why not use a P2P approach?
Backprop is neither commutative nor associative.
What do you mean? There's a bunch of proof-of-concepts such as Hydra, peer-nnet, Learnae, and so on.