Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

861–870 of 1001 posts

Re: Safe Superintelligence Inc.

#861

Post by CRConrad »

mdp2021 wrote: post #818
dreamcompiler wrote: post #756

Earlier quoted context omitted.

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

That "tweet" loses a veneer if you see that we value what has Worth as a collective treasure, and the more Value is produced the better - while that one engages in producing something of value is (hopefully but not necessarily) a good exercise in intelligent (literal sense) cultivation. So, yes, if algorithms strict or loose could one day produce Art, and Thought, and Judgement, of Superior quality: very welcome. Do…

> we value what has Worth as a collective treasure, and the more Value is produced the better ... So, yes, if algorithms strict or loose could one day produce Art, and Thought, and Judgement, of Superior quality: very welcome.

Except that's not how we value the "worth" of something. If "Art, and Thought, and Judgement" -- be they of "Superior quality" or not -- could be produced by machines, they'd be worth a heck of a lot less. (Come to think of it, hasn't that process already begun?)

Also, WTF is up with the weird capitalisations? Are you from Germany, or just from the seventeenth century?

Re: Safe Superintelligence Inc.

#862

Post by trashtester »

richardw wrote: post #703
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

The size of the gap between “smarter than humans” and “not controlled by humans anymore” is obviously where the disagreement is. To assume it’s a chasm that can never be overcome, you need at least the following to be true: No amount of focus or time or intelligence or mistakes in coding will ever bridge the gap. That rules and safeguards can be made that are perfectly inescapable. And nobody else will get enough pow…

I think the greatest concern is not so much that a single AI will be poorly aligned.

The greatest threat is if a population of AI's start to compete in ways that triggers Darwinian evolution between them.

If that happens, they will soon develop self preservation / replication drives that can gradually cause some of them to ignore human safety and prosperity conditioning in their loss function.

And if they're sufficiently advanced by then, we will have no way of knowing.

Re: Safe Superintelligence Inc.

#863

Post by CRConrad »

sneak wrote: post #840
bradleykingz wrote: post #797

Earlier quoted context omitted.

The way I see it, this is simply a repetition of history. El dorado, the fountain of youth, turning dirt to gold, the holy grail and now... superintelligence.

Human flight, resurrection (cardiopulmonary resuscitation machines), doubling human lifespans, instantaneous long distance communication, all of these things are simply pipe dreams.

> resurrection (cardiopulmonary resuscitation machines)

Get back to me when that can ressurect me after I've been dead for a week or so.

Re: Safe Superintelligence Inc.

#864

Post by bnjemian »

dreamcompiler wrote: post #756
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

Sometimes, my dishwasher stacks are poetry.

Re: Safe Superintelligence Inc.

#865

Post by dartos »

dash2 wrote: post #859
benreesman wrote: post #809

Earlier quoted context omitted.

My opinion is based on a lot more first hand experience than most, some of which I’m at liberty to share and some that I’m not and therefore becomes “color”. But I’m a nobody, Edward Snowden has a far more convincing track record on calling abuses of power: https://community.openai.com/t/edward-snowden-on-openai-s-de...

Here's Edward Snowden heroically waffling about "nuance" when asked why he hasn't spoken out more about Russia's invasion of Ukraine. On a crypto website. He became a full citizen of Russia in autumn 2022, by the way. https://www.coindesk.com/video/edward-snowden-explains-why-h...

Tbf he already has 1 superpower after his head.

Re: Safe Superintelligence Inc.

#866

Post by CRConrad »

creer wrote: post #763
gavin_gee wrote: post #232

Earlier quoted context omitted.

This. Every nation-state will be in the game. Private enterprise will be in the game. Bitcoin-funded individuals will be in the game. Criminal enterprises will be in the game. How does one company building a safe version stop that? If I have access to hardware and data how does a safety layer get enforced? Regulations are for organizations that care about public perception, the law, and stock prices. Criminals and na…

"nation state" doesn't mean what you think it means. More constructively, I don't know that very much will stop even a hacker from getting time on the local corporate or university AI and get it to do some "work". After all the first thing the other kind of hacker tried with generative AI is to get them to break out of their artificial boundaries, and hook them to internet resources. I don't know that anyone has hook…

> "nation state" doesn't mean what you think it means.

So what do you think it means? And what do you think the GP meant?

Feels annoying as fuck, bitching "your definition is wrong" without providing the (presumably) correct one.

Re: Safe Superintelligence Inc.

#867

Post by czl »

ilrwbwrkhv wrote: post #224
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

> These models will remain in the word vector similarity phase forever.

Forever? The same AI techniques are already being applied to analyze and understand images and video information after that comes ability to control robot hands and interact with the world and work on that is also ongoing.

> Till the tie we understand consciousness, we will not crack AGI …

We did not fully understand how bird bodies work yet that did not stop development of machines that fly. Why is an understanding of consciousness necessary to “crack AGI”?

Re: Safe Superintelligence Inc.

#870

Post by dartos »

CRConrad wrote: post #861
mdp2021 wrote: post #818

Earlier quoted context omitted.

That "tweet" loses a veneer if you see that we value what has Worth as a collective treasure, and the more Value is produced the better - while that one engages in producing something of value is (hopefully but not necessarily) a good exercise in intelligent (literal sense) cultivation. So, yes, if algorithms strict or loose could one day produce Art, and Thought, and Judgement, of Superior quality: very welcome. Do…

> we value what has Worth as a collective treasure, and the more Value is produced the better ... So, yes, if algorithms strict or loose could one day produce Art, and Thought, and Judgement, of Superior quality: very welcome. Except that's not how we value the "worth" of something. If "Art, and Thought, and Judgement" -- be they of "Superior quality" or not -- could be produced by machines, they'd be worth a heck of…

The issue I have with all of these discussions is how vague everyone always is.

“Art” isn’t a single thing. It’s not just pretty pictures. AI can’t make art. And give a good solid definition for thought which doesn’t depend on lived experiences while we’re at it. You can’t. We don’t have one.

“AGI” as well.

Post reply on HN