Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

81–90 of 1001 posts

Re: Safe Superintelligence Inc.

#81
post #73

Earlier quoted context omitted.

Galaxy Brain: TransparentAI

Maybe I'm just old and grumpy, but I can't help shake that the real most dangerous thing about AGI/ASI is centralization of its power (if it is ever possibly achieved). Everyone just fiend-ing for their version of it.

You're not old or grumpy, you're just stating the quiet part out loud. It's the same game, but now with 100% more AI.

Re: Safe Superintelligence Inc.

#82
post #43

At this point, all the computing power is concentrated among various companies such as Google, Facebook, Microsoft, Amazon, Tesla, etc. It seems to me it would be much safer and more intelligent to create a massive model and distribute the benefits among everyone. Why not use a P2P approach?

In my area, internet and energy are insanely expensive and that means I'm not at all willing to share my precious bandwidth or compute just to subsidize someone generating Rule 34 porn of their favorite anime character.

I don't seed torrents for the same reason. If I lived in South Korea or somewhere that bandwidth was dirt cheap, then maybe.

Re: Safe Superintelligence Inc.

#84

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

Academic compensation is different than what you’d find elsewhere on Hacker News. Likewise, academic performance is evaluated differently than what you’d expect as a software engineer. Ultimately, everyone cares about scientific impact so academic compensation relies on name and recognition far more than money. Personally, I care about the performance of the researchers (i.e., their publications), the institution’s l…

But wouldn't the focus on "safety first" sort of preclude them from giving their researchers the unfettered right to publish their work however and whenever they see fit? Isn't the idea to basically try to solve the problems in secret and only release things when they have high confidence in the safety properties?

If I were a researcher, I think I'd care more about ensuring that I get credit for any important theoretical discoveries I make. This is something that LeCun is constantly stressing and I think people underestimate this drive. Of course, there might be enough researchers today who are sufficiently scared of bad AI safety outcomes that they're willing to subordinate their own ego and professional drive to the "greater good" of society (at least in their own mind).

Re: Safe Superintelligence Inc.

#85
I know it is a difficult subject but whichever country gets access to this superintelligence will certainly use it for "safety" reasons. Sutskever has lived in israel and now has a team there , but israel doesnt strike me as a state that can be trusted with the safety of the world. (many of the AI business leaders are of jewish descent, but not sure if they have half their team there).

US on the other hand is a known quantity when it comes to policing the world.

Ultimately the only safe AI is going to be the open one, and it will probably have a stupid name

Re: Safe Superintelligence Inc.

#86

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

They will be able to pay their researchers the same way every other startup in the space is doing it – by raising an absurd amount of money.

Re: Safe Superintelligence Inc.

#87
If everyone is creating AGI, another AI company will just create another AGI. There is no such thing as SAFE AGI.

I feel like this 'safe' word is another word for censorship. Like google search results have become censored.

Re: Safe Superintelligence Inc.

#88
post #78
post #25

based on the naming conventions established by OpenAI and StabilityAI, this may be the most dangerous AI company yet

Being powerful is like being a lady. If you have to tell people you are, you aren’t. – Margaret Thatcher

Or: "Any man who must say, "I am the King", is no true king." - Tywin Lannister

Re: Safe Superintelligence Inc.

#89

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

> compensation packages that truly gifted AI researchers can make now

I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incredible work but getting paid less, academics making very little, and even the occasional "hobbyist" making nothing and churning out great work while hiding behind an anime girl avatar.

OpenAI clearly has some talented people, but there's also a bunch of the typical "TC optimization" crowd in there these days. The fact that so many were willing to resign with sama if necessary appears largely because they were more concerned with losing their nice compensation packages than any of their obsession with doing top tier research.

Re: Safe Superintelligence Inc.

#90

Earlier quoted context omitted.

Last I checked the researcher salaries haven't even reached software engineer levels.

The kind of AI researchers being discussed here likely make an order of magnitude more than run of the mill "software engineers".

Unless you know something I don’t, that’s not the case. It also makes sense, engineers are far more portable and scarcity isn’t an issue (many ML PhDs find engineering positions).
Post reply on HN