Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: Safe Superintelligence Inc.

#271

Post by philwelch »

dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

Love to see the traditional middlebrow dismissal as the top comment. Never change, HN.

> Are you really saying that there is a nice total-ordering of problems by importance to the world, and that the one you're interested happens also to be at the top?

It might be the case that the reason Ilya is “interested in” this problem (to the degree of dedicating almost his entire career to it) is exactly because he believes it’s the most important.

Re: Safe Superintelligence Inc.

#272

Post by wwweston »

appplication wrote: post #243
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

There’s no safe intelligence, so there’s no safe superintelligence. If you want safer superintelligence, you figure out how to augment the safest intelligence.

Re: Safe Superintelligence Inc.

#273

Post by Zacharias030 »

brcmthrowaway wrote: post #218
nuz wrote: post #211

Quite impressive how many AI companies Daniel Gross has had a hand in lately. Carmack, this, lots of other promising companies. I expect him to be quite a big player once some of these pays off in 10 years or so.

What's Carmack?

John Carmack, https://en.m.wikipedia.org/wiki/John_Carmack

Re: Safe Superintelligence Inc.

#274

Post by myth_drannon »

seydor wrote: post #85

I know it is a difficult subject but whichever country gets access to this superintelligence will certainly use it for "safety" reasons. Sutskever has lived in israel and now has a team there , but israel doesnt strike me as a state that can be trusted with the safety of the world. (many of the AI business leaders are of jewish descent, but not sure if they have half their team there). US on the other hand is a known…

Nvidia has a very large presence in Israel. They just acquired another startup there (Run:AI). If Nvidia, the most valuable company in the world and the most important AI company actively increases their presence there, so should others.

Israel has the largest concentration of AI startups in the world. It's just a no brainer to start an AI company there.

But since you brought up your favourite topic of discussion - Yahood, I will remind you that Jews have a fairly long tradition of working with AIs and the safety of it, dating for thousands of years. Look for Golem or Golem of Prague - https://en.wikipedia.org/wiki/Golem

Re: Safe Superintelligence Inc.

#275

Post by TeMPOraL »

Starlevel004 wrote: post #198
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

This all makes more sense when you realise it's Calvinism for programmers.

I think I heard that one before. Nuclear weapons are the Armageddon of nerds. Climate change is the Flood of the nerds. And so on.

Re: Safe Superintelligence Inc.

#276

Post by crazygringo »

TideAd wrote: post #254
TaupeRanger wrote: post #181

Earlier quoted context omitted.

Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.

So, this is actually an aspect of superintelligence that makes it way more dangerous than most people think. That we have no way to know if any given alignment technique works for the N+1 generation of AIs. It cuts down our ability to react, whenever the first superintelligence is created, if we can only start solving the problem after it's already created .

Fortunately, whenever you create a superintelligence, you obviously have a choice as to whether you confine it to inside a computer or whether you immediately hook it up to mobile robots with arms and fine finger control. One of these is obviously the far wiser choice.

As long as you can just turn it off by cutting the power, and you're not trying to put it inside of self-powered self-replicating robots, it doesn't seem like anything to worry about particularly.

A physical on/off switch is a pretty powerful safeguard.

(And even if you want to start talking about AI-powered weapons, that still requires humans to manufacture explosives etc. We're already seeing what drone technology is doing in Ukraine, and it isn't leading to any kind of massive advantage -- more than anything, it's contributing to the stalemate.)

Re: Safe Superintelligence Inc.

#277

Post by benreesman »

TaupeRanger wrote: post #181
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.

InstructGPT is basically click through rate optimization. The underlying models are in fact very impressive and very capable for a computer program, but they’re then subject to training and tuning with the explicit loss function of manipulating what human scorers click on, in a web browser or the like.

Is it any surprise that there’s no seeming upper bound on how crazy otherwise sane people act in the company of such? It’s like if TikTok had a scholarly air and arbitrary credibility.

Re: Safe Superintelligence Inc.

#278

Post by [deleted] »

jetrink wrote: post #187
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

To a technoutopian, scientific advances, and AI in particular, will one day solve all other human problems, create heaven on earth, and may even grant us eternal life. It's the most important problem in the same way that Christ's second coming is important in the Christian religion.

[deleted]

Re: Safe Superintelligence Inc.

#279

Post by ffhhj »

dTal wrote: post #253
Starlevel004 wrote: post #198

Earlier quoted context omitted.

This all makes more sense when you realise it's Calvinism for programmers.

Could you expand on this?

> [Superintelligence safety] teaches that the glory and sovereignty of [superintelligence] should come first in all things.
Post reply on HN