Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

291–300 of 1001 posts

Re: Safe Superintelligence Inc.

#291

Post by hervature »

crazygringo wrote: post #276
TideAd wrote: post #254

Earlier quoted context omitted.

So, this is actually an aspect of superintelligence that makes it way more dangerous than most people think. That we have no way to know if any given alignment technique works for the N+1 generation of AIs. It cuts down our ability to react, whenever the first superintelligence is created, if we can only start solving the problem after it's already created .

Fortunately, whenever you create a superintelligence, you obviously have a choice as to whether you confine it to inside a computer or whether you immediately hook it up to mobile robots with arms and fine finger control. One of these is obviously the far wiser choice. As long as you can just turn it off by cutting the power, and you're not trying to put it inside of self-powered self-replicating robots, it doesn't s…

I agree that an air-gapped AI presents little risk. Others will claim that it will fluctuate its internal voltage to generate EMI at capacitors which it will use to communicate via Bluetooth to the researcher's smart wallet which will upload itself to the cloud one byte at a time. People who fear AGI use a tautology to define AGI as that which we are not able to stop.

Re: Safe Superintelligence Inc.

#292

Post by kjkjadksj »

cwillu wrote: post #264
rafaelero wrote: post #106

Earlier quoted context omitted.

It's probably not possible, which makes all these initiatives painfully naive.

It'd be naive if it wasn't literally a standard point that is addressed and acknowledged as being a major part of the problem. There's a reason OpenAI's charter had this clause: “We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop c…

How does that address the issue? I would have expected them to do that anyhow. Thats what a lot of businesses do: let another company take the hit developing the market, R and D, and supply chain, then come in with industry standardization and cooperative agreements only after the money was proven to be good in this space. See electric cars. Also they could drop that at any time. Remember when openAI stood for opensource?

Re: Safe Superintelligence Inc.

#293

Post by cosmic_quanta »

mike_hearn wrote: post #283
appplication wrote: post #243

Earlier quoted context omitted.

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

We're really not that far. I'd argue superintelligence has already been achieved, and it's perfectly and knowably safe. Consider, GPT-4o or Claude are: • Way faster thinkers, readers, writers and computer operators than humans are • Way better educated • Way better at drawing/painting ... and yet, appear to be perfectly safe because they lack agency. There's just no evidence at all that they're dangerous. Why isn't t…

Yann LeCun said it best in an interview with Lex Friedman.

LLMs don't consume more energy when answering more complex questions. That means there's no inherent understanding of questions.

(which you could infer from their structure: LLMs recursively predict the next word, possibly using words they just predicted, and so on).

Re: Safe Superintelligence Inc.

#294

Post by newzisforsukas »

deegles wrote: post #225
TaupeRanger wrote: post #181

Earlier quoted context omitted.

Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.

'Fearing a rise of killer robots is like worrying about overpopulation on Mars.' - Andrew Ng

https://www.wired.com/brandlab/2015/05/andrew-ng-deep-learni... (2015)

> What’s the most valid reason that we should be worried about destructive artificial intelligence?

> I think that hundreds of years from now if people invent a technology that we haven’t heard of yet, maybe a computer could turn evil. But the future is so uncertain. I don’t know what’s going to happen five years from now. The reason I say that I don’t worry about AI turning evil is the same reason I don’t worry about overpopulation on Mars. Hundreds of years from now I hope we’ve colonized Mars. But we’ve never set foot on the planet so how can we productively worry about this problem now?

Re: Safe Superintelligence Inc.

#296

Post by mirekrusin »

dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

It's Palo Alto & Tel Aviv ordering that is total.

Re: Safe Superintelligence Inc.

#297

Post by the8472 »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

From a human welfare perspective this seems like worrying that a killer asteroid will make the 1% even richer because it contains goal if it can be safely captured. I would not phrase that as a "bigger and more pressing" worry if we're not even sure if we can do anything about the killer asteroid at all.

Re: Safe Superintelligence Inc.

#298

Post by mdp2021 »

appplication wrote: post #243
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

While the topic of "safe reasoning" may seem more or less preliminary before a good implementation of reasoning, it remains a theoretical discipline with its own importance and should be studied alongside the rest, also largely irregardless if its stage.

> We have no idea what the fundamental architecture for superintelligence would even begin to look like

Ambiguous expression. Not implemented technically does not mean we would not know what to implement.

Re: Safe Superintelligence Inc.

#299

Post by zild3d »

TaupeRanger wrote: post #181
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.

The counter argument is viewing it like nuclear energy. Even if its in the early days of our understanding of nuclear energy, seems pretty good to have a group working towards creating safe nuclear reactors, vs just trying to create nuclear reactors

Re: Safe Superintelligence Inc.

#300

Post by jackothy »

appplication wrote: post #243
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

"how someone so smart can be so naive"

Do you really think Ilya has not thought deeply about each and every one of your points here? There's plenty of answers to your criticisms if you look around instead of attacking.

Post reply on HN