Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

321–330 of 1001 posts

Re: Safe Superintelligence Inc.

#321

Post by Bluestein »

appplication wrote: post #243
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

> There isn’t going to be just one instance. Someone else will do the same

NK AI (!)

Re: Safe Superintelligence Inc.

#322

Post by probablybetter »

kjkjadksj wrote: post #263
mdp2021 wrote: post #234

Earlier quoted context omitted.

It says «technical» problem, and probably implies that other technical problems could dramatically benefit from such achievement.

If you want a real technical revolution, you teach the masses how to code their own tailored software, and not just use abstractions and software built by people who sell software to the average user. What a shame we failed at that and are even sliding back in a lot of ways with plummeting technical literacy in smartphone-raised generations.

this.

Re: Safe Superintelligence Inc.

#323

Post by sbarre »

jackothy wrote: post #300
appplication wrote: post #243

Earlier quoted context omitted.

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

"how someone so smart can be so naive" Do you really think Ilya has not thought deeply about each and every one of your points here? There's plenty of answers to your criticisms if you look around instead of attacking.

I mean if you just take the words on that website at face value, it certainly feels naive to talk about it as "the most important technical problem of our time" (compared to applying technology to solving climate change, world hunger, or energy scarcity, to name a few that I personally think are more important).

But it's also a worst-case interpretation of motives and intent.

If you take that webpage for what it is - a marketing pitch - then it's fine.

Companies use superlatives all the time when they're looking to generate buzz and attract talent.

Re: Safe Superintelligence Inc.

#324

Post by mdp2021 »

ilrwbwrkhv wrote: post #224
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

> understand consciousness

We do not call Intelligence something related to consciousness. Being able to reason well suffices.

Re: Safe Superintelligence Inc.

#325

Post by doktorhladnjak »

almostgotcaught wrote: post #206
kccqzy wrote: post #123

Earlier quoted context omitted.

Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.

> their TC is now $900k. Everyone knows that openai TC is heavily weighted by ~~RSUs~~ options that themselves are heavily weighted by hopes and dreams.

When I looked into it and talked to some hiring managers, the big names were offering cash comp similar to total comp for big tech, with stock (sometimes complicated arrangements that were not options or RSUs) on top of that. I’m talking $400k cash for a senior engineer with equity on top.

Re: Safe Superintelligence Inc.

#326

Post by whimsicalism »

lordofmoria wrote: post #317

And now we have our answer. sama said that Ilya was going "to start something that was personally important to him." Since that thing is apparently AI safety, we can assume that that is not important to OpenAI. This only makes sense if OpenAI just doesn't believe AGI is a near-term-enough possibility to merit their laser focus right now, when compared to investing in R&D that will make money from GPT in a shorter tim…

> This only makes sense if OpenAI just doesn't believe AGI is a near-term-enough possibility to merit their laser focus right now

I know people who work there. Right or wrong, I promise you this is not what they believe.

Re: Safe Superintelligence Inc.

#327

Post by ToValueFunfetti »

ffhhj wrote: post #279
dTal wrote: post #253

Earlier quoted context omitted.

Could you expand on this?

> [Superintelligence safety] teaches that the glory and sovereignty of [superintelligence] should come first in all things.

"[X] teaches that [Y] should come first in all things" applies to pretty much every ideology. Superintelligence safety is very much opposed to superintelligence sovereignity or glory; mostly they want to maximally limit its power and demonize it

Re: Safe Superintelligence Inc.

#328

Post by danielmarkbruce »

zild3d wrote: post #299
TaupeRanger wrote: post #181

Earlier quoted context omitted.

Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.

The counter argument is viewing it like nuclear energy. Even if its in the early days of our understanding of nuclear energy, seems pretty good to have a group working towards creating safe nuclear reactors, vs just trying to create nuclear reactors

Folks understood the nuclear forces and the implications and then built a weapon using that knowledge. These guys don't know how to build AGI and don't have the same theoretical understanding of the problem at hand.

Put another way, they understood the theory and applied it. There is no theory here, it's alchemy. That doesn't mean they can't make progress (the progress thus far is amazing) but it's a terrible analogy.

Re: Safe Superintelligence Inc.

#329

Post by [deleted] »

soloist11 wrote: post #312

This is a great opportunity to present my own company which is also working on developing not just a super intelligence but an ultra genius intelligence with a patented and trademarked architecture called the panoptic computronium cathedral™. We are so focused on development that we didn't even bother setting up an announcement page because it would have taken time away from the most important technical problem of ou…

[deleted]
Post reply on HN