Safe Superintelligence Inc. • Hacker News Board

Re: Safe Superintelligence Inc.

#721

Post by Tempest1981 » Thu, Jun 20, 2024, 1:37 AM UTC

jen729w wrote: post #580

Sharlin wrote: post #441

Earlier quoted context omitted.

> You can make a safe AI today, but what happens when the next person is managing things? The point of safe superintelligence , and presumably the goal of SSI Inc., is that there won't be a next (biological) person managing things afterwards. At least none who could do anything to build a competing unsafe SAI. We're not talking about the banal definition of "safety" here. If the first superintelligence has any reason…

I wonder how many people panicking about these things have ever visited a data centre. They have big red buttons at the end of every pod. Shuts everything down. They have bigger red buttons at the end of every power unit. Shuts everything down. And down at the city, there’s a big red button at the biggest power unit. Shuts everything down. Having arms and legs is going to be a significant benefit for some time yet. I…

Might be running on a botnet of CoPilot PCs

Re: Safe Superintelligence Inc.

#722

Post by varelse » Thu, Jun 20, 2024, 1:41 AM UTC

kccqzy wrote: post #123

PheonixPharts wrote: post #89

Earlier quoted context omitted.

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.

[dead]

Re: Safe Superintelligence Inc.

#723

Post by AYBABTME » Thu, Jun 20, 2024, 1:42 AM UTC

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

A counter argument is that nuclear arms brought unprecedented worldwide peace. If it's to be used as an analogy for AI, we should consider that the outcome isn't clear cut and lies in the eye of the beholder.

Re: Safe Superintelligence Inc.

#724

Post by no_exit » Thu, Jun 20, 2024, 1:44 AM UTC

tasoeur wrote: post #544

nuz wrote: post #211

Quite impressive how many AI companies Daniel Gross has had a hand in lately. Carmack, this, lots of other promising companies. I expect him to be quite a big player once some of these pays off in 10 years or so.

Good for him honestly, but I’m not approaching a company with Daniel Gross in leadership…, working with him back at Apple after their company was acquired for Siri improvements was just terrible.

A member of the Lee Kuan Yew fanclub milieu is a dickhead? I'm surprised.

Re: Safe Superintelligence Inc.

#725

Post by AYBABTME » Thu, Jun 20, 2024, 1:45 AM UTC

hackerlight wrote: post #708

richrichie wrote: post #700

Earlier quoted context omitted.

US is no angel and it cannot be the only one which wins the race. We have hard evidence of how monopoly power gets abused in the case of the US e.g. as the sole nuclear power, it used nukes on civilians. We need every one to win this race to keep things on balance.

US has to win the race because while it's true that it's no angel, it isn't an authoritarian dictatorship and there isn't an equivalence in how bad the world will end up for you and me if the authoritarian side wins the race. Monopoly power will get abused the most by the least democratic actors, which is China. We need multiple actors within the US to win to balance power. We don't need or want China to be one of th…

If anyone doubts this. Recent (<100y) leaders of China and Russia internally displaced and caused the death of large % of their population, for essentially fanciful ideological reasons.

Re: Safe Superintelligence Inc.

#726

Post by gmuslera » Thu, Jun 20, 2024, 1:51 AM UTC

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

We don't know if that superintelligence will be safe or not. But as long as we are in the mix, the combination is unsafe. At the very least, because it will expand the inequality. But probably there are deeper reasons, things that make that combination of words an absurd. Or it will be abused, or the reason that it is not is that it wasn't so unsafe after all.

Re: Safe Superintelligence Inc.

#727

Post by richrichie » Thu, Jun 20, 2024, 1:59 AM UTC

hackerlight wrote: post #708

richrichie wrote: post #700

Earlier quoted context omitted.

US is no angel and it cannot be the only one which wins the race. We have hard evidence of how monopoly power gets abused in the case of the US e.g. as the sole nuclear power, it used nukes on civilians. We need every one to win this race to keep things on balance.

US has to win the race because while it's true that it's no angel, it isn't an authoritarian dictatorship and there isn't an equivalence in how bad the world will end up for you and me if the authoritarian side wins the race. Monopoly power will get abused the most by the least democratic actors, which is China. We need multiple actors within the US to win to balance power. We don't need or want China to be one of th…

I am puzzled how it is OK to kill people of other countries, but not your own. US, China and Russia have all indulged in wanton mass killing of people. So that's an even keel for me.

The nuking of civilians and the abuse of supremacy post cold war show that the US cannot trusted to act morally and ethically in the absence of comparable adversaries. Possession of nukes by Russia and China clearly kept the US military adventures somewhat in check.

Re: Safe Superintelligence Inc.

#728

Post by richrichie » Thu, Jun 20, 2024, 2:02 AM UTC

insane_dreamer wrote: post #475

ilrwbwrkhv wrote: post #224

Earlier quoted context omitted.

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

I don't think the AI has to be "sentient" in order to be a threat.

Just bad software can be existential threat if it is behind sensitive systems. A neural network is bad software for critical systems.

Re: Safe Superintelligence Inc.

#729

Post by imchillyb » Thu, Jun 20, 2024, 2:04 AM UTC

Safe Superintelligence is impossible.

Superintelligence will be monitored run and updated by humans.

Humans cannot safeguard anything for long.

SCIFs are supposed to be the safest most secured facilities on earth. Yet, how many 'leaks' have there been in the last 2 decades?

Security is an impossible task. Safety is an impossible task. Sisyphus has more chance of completing his mission than either of those things have of being reality.

Re: Safe Superintelligence Inc.

#730

Post by rqtwteye » Thu, Jun 20, 2024, 2:30 AM UTC

I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege.