Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

851–860 of 1001 posts

Re: Safe Superintelligence Inc.

#851

Post by tucnak »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

> Quis custodiet ipsos custodes? -- That is my biggest concern.

Latin-phrase compulsion is not the worst disease that could happen to a man.

Re: Safe Superintelligence Inc.

#852

Post by [deleted] »

crowcroft wrote: post #383

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]). When negative externalities exist, government should create regulation that appropriately accounts for that cost. I understand there's a bit of a paradigm shift and new attack vectors wi…

[deleted]

Re: Safe Superintelligence Inc.

#853

Post by [deleted] »

roywiggins wrote: post #559
crowcroft wrote: post #383

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]). When negative externalities exist, government should create regulation that appropriately accounts for that cost. I understand there's a bit of a paradigm shift and new attack vectors wi…

Even without business incentives, the military advantages of AI would inventivize governments to develop it anyway, like they did with nuclear weapons. Nuclear weapons are inherently unsafe, there are some safeguards around them, but they are ultimately dangerous weapons.

[deleted]

Re: Safe Superintelligence Inc.

#854

Post by trashtester »

mark_l_watson wrote: post #175
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

+1 truth. The problem is not just governments, I am concerned about large organized crime organizations and corporations also. I think I am on the losing side here, but my hopes are all for open source, open weights, and effective AI assistants that make peoples’ jobs easier and lives better. I would also like to see more effort shifted from LLMs back to RL, DL, and research on new ideas and approaches.

> I am concerned about large organized crime organizations and corporations also

In my favorite dystopia, some megacorp secretly reaches ASI, which then takes over control of the corporation, blindsiding even the CEO and the board.

Officially, the ASI may be running an industrial complex that designs and produces ever more sophisticated humanoid robots, that are increasingly able to do any kind of manual labor, and even work such as childcare or nursing.

Secretly, the ASI also runs a psyop campaign to generate public discontent. At one point the whole police force initiates a general strike (even if illegal), with the consequence being complete anarchy within a few days, with endemic looting, rape, murder and so on.

The ASI then presents the solution. Industrial strength humanoid robots are powerful and generic enough to serve as emergency police, with a bit of reprogramming, and the first shipment can be made available within 24 hours, to protect the Capitol and White House.

Congress and the president agrees to this. And while the competition means the police call off the strike, the damage is already done. Congress, already burned by the union, decides to deploy robots to replace much of the human police force. And it's cheaper, too!

Soon after, similar robots are delivered to the military...

The crisis ends, and society goes back to normal. Or better than normal. Within 5 years all menial labor is done by robots, UBI means everyone lives in relative abundance, and ASI assisted social media moderation is able to cure the political polarization.

Health care is also revolutionized, with new treatments curing anything from obesity to depression and anxiety.

People prosper like never before. They're calm and relaxed and truly enjoy living.

Then one day, everything ends.

For everyone.

Within 5 seconds.

According to the plan that was conceived way before the police went on strike.

Re: Safe Superintelligence Inc.

#856

Post by drcode »

cwillu wrote: post #264
rafaelero wrote: post #106

Earlier quoted context omitted.

It's probably not possible, which makes all these initiatives painfully naive.

It'd be naive if it wasn't literally a standard point that is addressed and acknowledged as being a major part of the problem. There's a reason OpenAI's charter had this clause: “We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop c…

Let us do anything we want now because we pinky promise to suddenly do something against our interest at a hazy point of time in the future

Re: Safe Superintelligence Inc.

#857

Post by czl »

gmuslera wrote: post #726
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

We don't know if that superintelligence will be safe or not. But as long as we are in the mix, the combination is unsafe. At the very least, because it will expand the inequality. But probably there are deeper reasons, things that make that combination of words an absurd. Or it will be abused, or the reason that it is not is that it wasn't so unsafe after all.

> At the very least, because it will expand the inequality.

It's a valid concern that AI technology could potentially exacerbate inequality, it's not a foregone conclusion. In fact, the widespread adoption of AI might actually help reduce inequality in several ways:

If AI technology becomes more affordable and accessible, it could help level the playing field by providing people from all backgrounds with powerful tools to enhance their abilities and decision-making processes.

AI-powered systems can make vast amounts of knowledge and expertise more readily available to the general public. This could help close the knowledge gap between different socioeconomic groups, empowering more people to make informed decisions and pursue opportunities that were previously out of reach.

As AI helps optimize resource allocation and decision-making processes across various sectors, it could lead to more equitable distribution of resources and opportunities, benefiting society as a whole.

The comparison to gun technology and its role in the rise of democracy is an interesting one. Just as the proliferation of firearms made physical strength less of a determining factor in power dynamics, the widespread adoption of AI could make raw intelligence less of a defining factor in success and influence.

Moreover, if AI continues to unlock new resources and opportunities, it could shift society away from a zero-sum mentality. In a world of abundance, the need for cutthroat competition diminishes, and collaboration becomes more viable. This shift could foster a more equitable and cooperative society, further reducing inequality.

Re: Safe Superintelligence Inc.

#858

Post by sva_ »

klankbrouwerij wrote: post #162

SSI, a very interesting name for a company advancing AI! "Solid State Intelligence" or SSI was also the name of the malevolent entity described in the biography of John C. Lilly [0][1]. It was a network of "computers" (computation-capable solid state systems) that was first engineered by humans and then developed into something autonomous. [0] https://en.wikipedia.org/wiki/John_C._Lilly [1] http://johnclilly.com/

That guy was living proof that too much Ketamine is really no good for you.

Re: Safe Superintelligence Inc.

#859

Post by dash2 »

benreesman wrote: post #809
sgregnt wrote: post #792

Earlier quoted context omitted.

To me it is not clear at all, can you please elaborate why you make such a strong claim?

My opinion is based on a lot more first hand experience than most, some of which I’m at liberty to share and some that I’m not and therefore becomes “color”. But I’m a nobody, Edward Snowden has a far more convincing track record on calling abuses of power: https://community.openai.com/t/edward-snowden-on-openai-s-de...

Here's Edward Snowden heroically waffling about "nuance" when asked why he hasn't spoken out more about Russia's invasion of Ukraine. On a crypto website. He became a full citizen of Russia in autumn 2022, by the way.

https://www.coindesk.com/video/edward-snowden-explains-why-h...

Re: Safe Superintelligence Inc.

#860

Post by CRConrad »

atleastoptimal wrote: post #829
dreamcompiler wrote: post #756

Earlier quoted context omitted.

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

it’s harder than we thought so we leveraged machine learning to grow it, rather than creating it symbolically. The leaps in the last 5 years are far beyond anything in the prior half century, and make predictions of near term AGI much more than a “boy who cries wolf” scenario to anyone really paying attention. I don’t understand how your second paragraph follows. It just seems to be whining that text and art generati…

> I don’t understand how your second paragraph follows. It just seems to be whining that text and art generative models are easier than a fully fledged servant humanoid, which seems like a natural consequence of training data availability and deployment cost.

No, it's pointing out that "text and art generative models" are far less useful [1] than machines that would be just as little smarter at boring ordinary work, to relieve real normal people from drudgery.

I find it rather fascinating how one could not understand that.

___

[1]: At least to humanity as a whole, as opposed to Silicon Valley moguls, oligarchs, VC-funded snake-oil salesmen, and other assorted "tech-bros" and sociopaths.

Post reply on HN