Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

471–480 of 1001 posts

Re: Safe Superintelligence Inc.

#471

Post by Xenoamorphous »

tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

I wonder if this is an Ian Malcolm in Jurassic Park situation, i.e. “your scientists were so preoccupied with whether they could they didn t stop to think if they should”.

Maybe the only way to avoid an unsafe superintelligence is to not create a superintelligence at all.

Re: Safe Superintelligence Inc.

#472

Post by m3kw9 »

I bet they will not be the first to get super intelligence or that they will devolve back in to move fast and make money to survive and deprioritize safety, but still say safety. All companies knows this, they know the value of safety(because they themselves doesn’t want to die) and that to continue development, they need money.

Re: Safe Superintelligence Inc.

#473

Post by chrisldgk »

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book.

But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able to be creative outside of deriving things humans have already made before. Granted, they‘re really good at doing that, but not much else.

To me, this is such a transparent attention grab (and, by extension, money grab by being overvalued by investors and shareholders) by Altman and company, that I’m just baffled people are still going with it.

Re: Safe Superintelligence Inc.

#474

Post by daniel_reetz »

delichon wrote: post #430
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

If superintelligence can be achieved, I'm pessimistic that a team committed to doing it safely can get there faster than other teams without the safety. They may be wearing leg shackles in a foot race with the biggest corporations, governments and everyone else. For the sufficiently power hungry, safety is not a moat.

Exactly. Regulation and safety only affect law abiding entities. This is precisely why it's a "genie out of the bottle" situation -- those who would do the worst with it are uninhibited.

Re: Safe Superintelligence Inc.

#475

Post by insane_dreamer »

ilrwbwrkhv wrote: post #224
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

I don't think the AI has to be "sentient" in order to be a threat.

Re: Safe Superintelligence Inc.

#476

Post by foobiekr »

chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

It's no mystery, AI has attracted tons of grifters trying to cash out before the bubble pops, and investors aren't really good at filtering.

Re: Safe Superintelligence Inc.

#477

Post by spacecadet »

chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

Egos man. Egos.

Re: Safe Superintelligence Inc.

#478

Post by Nasrudith »

kmacdough wrote: post #397
worldsayshi wrote: post #364

Earlier quoted context omitted.

Yeah this feels close to the issue. Seems more likely that a harmful super intelligence emerges from an organisation that wants it to behave in that way than it inventing and hiding motivations until it has escaped.

I think a harmful AI simply emerges from asking an AI to optimize for some set of seemingly reasonable business goals, only to find it does great harm in the process. Most companies would then enable such behavior by hiding the damage from the press to protect investors rather than temporarily suspending business and admitting the issue.

Forget AI. We can't even come up with a framework to avoid seemingly reasonable goals doing great harm in the process for people. We often don't have enough information until we try and find out that oops, using a mix of rust and powdered aluminum to try to protect something from extreme heat was a terrible idea.

Re: Safe Superintelligence Inc.

#480

Post by jrflowers »

satvikpendem wrote: post #462
creato wrote: post #459

Earlier quoted context omitted.

"Government doesn't work. We just need to make a new government that is much more effective and far reaching in controlling people's behavior."

That's not what they said though. Seems to me more of a libertarian ideal than making a new government.

Reinventing government and calling it a private corporation is one of the main activities that libertarians engage in
Post reply on HN