Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

791–800 of 1001 posts

Re: Safe Superintelligence Inc.

#791

Post by dchichkov »

dreamcompiler wrote: post #756
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

Well, copilots do precisely that, no?

Or you talking folding literal laundry, in which case this is more of a robotics problem, not the ASI, right?

You don't need ASI to fold laundry, you do need to achieve reliable, safe and cost efficient robotics deployments. These are different problems.

Re: Safe Superintelligence Inc.

#792

Post by sgregnt »

benreesman wrote: post #753
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

It’s just clearly military R&D at this point. And it’s not even a little bit controversial that cutting edge military R&D is classified in general and to an extreme in wartime. The new thing is the lie that it’s a consumer offering. What’s new is giving the helm to shady failed social network founders with no accountability. These people aren’t retired generals with combat experience. They aren’t tenured professors a…

To me it is not clear at all, can you please elaborate why you make such a strong claim?

Re: Safe Superintelligence Inc.

#793

Post by bbor »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

Yup, well said. I think it's important to remember sometimes that Skynet was some sort of all-powerful military program -- maybe we should just, y'know, not do that part? Not even to win a war? That's the hope...

More generally/academically, you've pointed out that this covers only half of the violence problem, and I'd argue there's actually a whole other dimension at play bringing the total number of problem areas to four, of which this just the first:

  ## I.1. Subservient Violence
  ## I.2. Subservient Misinformation
  ## I.3. Autonomous Violence
  ## I.4. Autonomous Misinformation
But I think it's a lot harder to recruit for an AI alignment company than it is to recruit for an AI safety company.

Re: Safe Superintelligence Inc.

#794

Post by sgregnt »

buckinkb wrote: post #776
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

We should be less concerned about super intelligence and more about the immediate threat of job loss. An AI doesn’t need to be Skynet to wreak massive havoc on society. Replacing 20% of jobs in a very short period of time could spark global unrest resulting in WW3

The thing is, the replaced 20% people can always revert to having economy i.e., business among themselves, unless of cause they themselves prefer (cheaper) buissiness with AI. But then this just means they are better off in the first place from this change.

It is a bit like claiming that third world low productivity countries are suffering because there are countries with much much higher productivity. Well, they can continue to do low productivity business but increase it a bit using things like phones developed by high productivity country elsewhere.

Re: Safe Superintelligence Inc.

#795

Post by bbor »

ridiculous_leke wrote: post #790
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

> The "cold war" nuclear arms race, which brought the world to the brink of (at least partial) annihilation, is a good recent example. The same era saw big achievements like first human in space, eradication of smallpox, peaceful nuclear exploration etc. It's good to be a skeptic but history does favor the optimists for the most part.

Holy hell please knock on wood, this is the kinda comment that gets put in a museum in 10,000 years on The Beginning of the End of The Age of Hubris. We've avoided side effects from our new weapons for 80 years -- that does not exactly make me super confident it won't happen again!

In general, I think drawing conclusions about "history" from the past couple hundred years is tough. And unless you take a VERY long view, I don't see how one could describe the vast majority of the past as a win for the optimists. I guess suffering is relative, but good god was there a lot of suffering before modern medicine.

If anyone's feeling like we've made it through to the other side of the nuclear threat, "Mission Accomplished"-style, I highly recommend A Canticle for Lebowitz. It won a hugo award, and it's a short read best done with little research beforehand.

Re: Safe Superintelligence Inc.

#796

Post by sgregnt »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

How about this possibility: The good guys will be one step ahead, they will have more resources the bad guys will risk imprisonment if they misapply super intelligence. And this will be discovered and protected from by even better super intelligence.

Re: Safe Superintelligence Inc.

#797

Post by bradleykingz »

dreamcompiler wrote: post #756
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

The way I see it, this is simply a repetition of history.

El dorado, the fountain of youth, turning dirt to gold, the holy grail and now... superintelligence.

Re: Safe Superintelligence Inc.

#798

Post by worldsayshi »

Nasrudith wrote: post #478
kmacdough wrote: post #397

Earlier quoted context omitted.

I think a harmful AI simply emerges from asking an AI to optimize for some set of seemingly reasonable business goals, only to find it does great harm in the process. Most companies would then enable such behavior by hiding the damage from the press to protect investors rather than temporarily suspending business and admitting the issue.

Forget AI. We can't even come up with a framework to avoid seemingly reasonable goals doing great harm in the process for people. We often don't have enough information until we try and find out that oops, using a mix of rust and powdered aluminum to try to protect something from extreme heat was a terrible idea.

> We can't even come up with a framework to avoid seemingly reasonable goals doing great harm in the process for people.

I mean it's not like we're trying all that much in a practical sense right?

Whatever happened to charter cities?

Re: Safe Superintelligence Inc.

#799

Post by rldjbpin »

as much hate as the ceo is getting, the posturing here implies that openai branding and their marketing has been a team effort.

while i wish ssi great success, it sounds like another ai startup selling decades away dream.

Re: Safe Superintelligence Inc.

#800

Post by insane_dreamer »

ridiculous_leke wrote: post #790
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

> The "cold war" nuclear arms race, which brought the world to the brink of (at least partial) annihilation, is a good recent example. The same era saw big achievements like first human in space, eradication of smallpox, peaceful nuclear exploration etc. It's good to be a skeptic but history does favor the optimists for the most part.

Were any of these big achievements side effects of creating nuclear weapons? If not, then they're not relevant to the issue.

I'm not saying nothing else good happened in the past 70 years, but rather that the invention of atomic weapons has permanently placed humanity in a position in which it had never been before: the possibility of wiping out much of the planet, averted only thanks to treaties, Stanislav Petrov[0], and likely other cool heads.

[0] https://en.wikipedia.org/wiki/Stanislav_Petrov

Post reply on HN