Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

461–470 of 1001 posts

Re: Safe Superintelligence Inc.

#461

Post by nilkn »

seanmcdirmid wrote: post #360
kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

The safe business won’t hold very long if someone can gain a short term business advantage with unsafe AI. Eventually government has to step in with a legal and enforcement framework to prevent greed from ruining things.

It's possible that safety will eventually become the business advantage, just like privacy can be a business advantage today but wasn't taken so seriously 10-15 years ago by the general public.

This is not even that far-fetched. A safe AI that you can trust should be far more useful and economically valuable than an unsafe AI that you cannot trust. AI systems today aren't powerful enough for the difference to really matter yet, because present AI systems are mostly not yet acting as fully autonomous agents having a tangible impact on the world around them.

Re: Safe Superintelligence Inc.

#462

Post by satvikpendem »

creato wrote: post #459
__MatrixMan__ wrote: post #445

Earlier quoted context omitted.

Government is controlled by the highest bidder. I think we should be prepared to do this ourselves by refusing to accept money made by unsafe businesses, even if it means saying goodbye to the convenience of fungible money.

"Government doesn't work. We just need to make a new government that is much more effective and far reaching in controlling people's behavior."

That's not what they said though. Seems to me more of a libertarian ideal than making a new government.

Re: Safe Superintelligence Inc.

#463

Post by m3kw9 »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

There will always be a factor of time in terms of able to utilize super intelligence to do your bidding and there is a big spectrum of things that can be achieved it it always starts small. The imagination is lazy when thinking about all the steps and inbetween + scenarios. In the time that super intelligence is found and used, there will be competing near super intelligences, as all forms of cutting edge models are likely commercial at first because that is where most scientific activities are at. Things very unlikely will go Skynet all of a sudden at first because humans at the control are not that stupid otherwise nuclear war would have us all killed by now and it’s been 50 years since invention

Re: Safe Superintelligence Inc.

#464

Post by OtherShrezzing »

zeknife wrote: post #448
ben_w wrote: post #443

Earlier quoted context omitted.

A human-level AI can do anything that a human can do (modulo did you put it into a robot body, but lots of different groups are already doing that with current LLMs). Therefore, please imagine the most amoral, power-hungry, successful sociopath you've ever heard of. Doesn't matter if you're thinking of a famous dictator, or a religious leader, or someone who never got in the news and you had the misfortune to meet in…

Anything except tasks that require having direct control of a physical body. Until fully functional androids are developed, there is a lot a human-level AI can't do.

I think there's usually a difference between human-level and super-intelligent in these conversations. You can reasonably assume (some day) a superintelligence is going to

1) understand how to improve itself & undertake novel research

2) understand how to deceive humans

3) understand how to undermine digital environments

If an entity with these three traits were sufficiently motivated, they could pose a material risk to humans, even without a physical body.

Re: Safe Superintelligence Inc.

#465

Post by satvikpendem »

kmacdough wrote: post #397
worldsayshi wrote: post #364

Earlier quoted context omitted.

Yeah this feels close to the issue. Seems more likely that a harmful super intelligence emerges from an organisation that wants it to behave in that way than it inventing and hiding motivations until it has escaped.

I think a harmful AI simply emerges from asking an AI to optimize for some set of seemingly reasonable business goals, only to find it does great harm in the process. Most companies would then enable such behavior by hiding the damage from the press to protect investors rather than temporarily suspending business and admitting the issue.

This is well known via the paperclip maximization problem.

Re: Safe Superintelligence Inc.

#466

Post by satvikpendem »

tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

You should read the book Superintelligence by Nick Bostrom as this is exactly what he discusses.

Re: Safe Superintelligence Inc.

#467

Post by m3kw9 »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

If robots (hardware/self assembling factories/ resource gathering etc) are not involved this isnt likely a problem. You will know when these things form and will be crystal clear, but just having the model won’t do much when hardware is what really kills right now

Re: Safe Superintelligence Inc.

#468

Post by z7 »

aristofun wrote: post #388

What a waste of an intelligence. Pursuing artificial goal to solve a non existent problem to profit off meaningless hype around it. World would have been better off if he made a decent alternative to k8s or invested his skills into curing cancer or at least protecting world from totalitarian governments and dangerous ideologies (if he wants to belong to vague generic cause). You know, real problems, like the ones peo…

Nice absolute certainty you have there.

Re: Safe Superintelligence Inc.

#469

Post by seanmcdirmid »

__MatrixMan__ wrote: post #445
seanmcdirmid wrote: post #360

Earlier quoted context omitted.

The safe business won’t hold very long if someone can gain a short term business advantage with unsafe AI. Eventually government has to step in with a legal and enforcement framework to prevent greed from ruining things.

Government is controlled by the highest bidder. I think we should be prepared to do this ourselves by refusing to accept money made by unsafe businesses, even if it means saying goodbye to the convenience of fungible money.

Replace government with collective society assurance that no one cheats so we aren’t all doomed. Otherwise, someone will do it, and we all will have to bear the consequences.

If only enough individuals are willing to buy these services, then again we all will bear the consequences. There is no way out of this where libertarian ideals can be used to come to a safe result. What makes this even a more wicked problem is that decisions made in other countries will affect us all as well, we can’t isolate ourselves from AI policies made in China for example.

Re: Safe Superintelligence Inc.

#470

Post by mochomocha »

__MatrixMan__ wrote: post #445
seanmcdirmid wrote: post #360

Earlier quoted context omitted.

The safe business won’t hold very long if someone can gain a short term business advantage with unsafe AI. Eventually government has to step in with a legal and enforcement framework to prevent greed from ruining things.

Government is controlled by the highest bidder. I think we should be prepared to do this ourselves by refusing to accept money made by unsafe businesses, even if it means saying goodbye to the convenience of fungible money.

> Government is controlled by the highest bidder.

While this might be true for the governments you have personally experienced, this is far from being an aphorism.

Post reply on HN