Safe Superintelligence Inc. • Hacker News Board

Re: Safe Superintelligence Inc.

#841

Post by rstuart4133 » Thu, Jun 20, 2024, 8:50 AM UTC

dreamcompiler wrote: post #756

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

AGI does look like an unsolved problem right now, and a hard one at that. But I think it is wrong to think that it needs an AGI to cause total havoc.

I think my dyslexic namesake Prof Stuart Russell got it right. It humans won't need an AGI to dominate and kill each other. Mosquitoes have killed far more people than war. Ask yourself how long will it take us to develop a neutral network as smart as a mosquito, because that's all it will take.

It seems so simple, as the beastie only has 200,000 neurons. Yet I've been programming for over 4 decades and for most of them it was evident neither I nor any of my contemporaries were remotely capable of emulating it. That's still true if course. Never in my wildest dreams did it occur to me that repeated applications could produce something I couldn't, a mosquito brain. Now that looks imminent.

Now I don't know what to be more scared of. An AGI, or a artificial mosquito swarm run by Pol Pot.

Re: Safe Superintelligence Inc.

#842

Post by sneak » Thu, Jun 20, 2024, 8:52 AM UTC

roywiggins wrote: post #559

crowcroft wrote: post #383

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]). When negative externalities exist, government should create regulation that appropriately accounts for that cost. I understand there's a bit of a paradigm shift and new attack vectors wi…

Even without business incentives, the military advantages of AI would inventivize governments to develop it anyway, like they did with nuclear weapons. Nuclear weapons are inherently unsafe, there are some safeguards around them, but they are ultimately dangerous weapons.

Dangerous weapons are not inherently unsafe.

Take a Glock, for example. It is a deadly weapon, designed for one thing and one thing alone. It is, however, one of the safest machines ever built.

Re: Safe Superintelligence Inc.

#843

Post by sneak » Thu, Jun 20, 2024, 8:54 AM UTC

throwaway4good wrote: post #801

botanical wrote: post #766

Earlier quoted context omitted.

The consequences of actual AI is far reaching compared to a plane crash.

No they are not. Not unless you connect a machine gun to your neural net. Otherwise - we are talking about a chat bot - yes if there is no safety - it will say something racist - or implement a virus for you that you would have had to search 4chan for or something. None of this is any more dangerous than what you can find on the far corners of the internet. And it will not bring the end of the world.

If it is sufficiently intelligent, then it will be able to arrange the hooking up of machine guns just via chat. You seem to fail to note that superintelligence has an inherent value prop. It can use its better understanding of the world to generate value in a near vacuum, and use that value to bribe, coerce, blackmail, and otherwise manipulate whatever humans can read its output.

Imagine a chatbot that can reward people with becoming a billionaire via the stock market if you just do what it says. Imagine a chatbot that can pay your colleagues in solved bitcoin block hashes to kill your children if you don’t do what it says to connect it to more systems. Imagine a superintelligence that can inductively reason about gaps in a missile defense system and dangle that informational carrot to generals in exchange for an unfiltered ethernet port.

There is a great A24 movie called “Ex Machina” that explores this concept.

Superintelligence is inherently dangerous to the human status quo. It may be impossible to develop it “safely” in the sense that most people mean by that. It may also be impossible to avoid developing it. This might be the singularity everyone’s been talking about, just without the outcome that everyone hoped for.

Re: Safe Superintelligence Inc.

#844

Post by czl » Thu, Jun 20, 2024, 8:59 AM UTC

chii wrote: post #711

whimsicalism wrote: post #398

Earlier quoted context omitted.

I think the honest view (and you can scoff at it) is that winning the SI race basically wins you the enforcement race for free

That's why it's called an arms race, and it does not really end in this predictable manner. The party that's about to lose will use any extrajudicial means to reclaim their victory, regardless of the consequences, because their own destruction would be imminent otherwise. This ultimately leads to violence.

> The party that's about to lose will use any extrajudicial means to reclaim their victory,

How will the party about lose know they are about to lose?

> regardless of the consequences, because their own destruction would be imminent otherwise.

Why would AGI solve things using destruction? Consider how the most inteligent among us view our competition with other living beings. Is destruction the goal? So why would an even more intelligent AGI have that goal?

Re: Safe Superintelligence Inc.

#845

Post by arisAlexis » Thu, Jun 20, 2024, 9:08 AM UTC

dreamcompiler wrote: post #756

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

it's not according to expert consensus (top labs, top scientists)

Re: Safe Superintelligence Inc.

#846

Post by czl » Thu, Jun 20, 2024, 9:21 AM UTC

insane_dreamer wrote: post #800

ridiculous_leke wrote: post #790

Earlier quoted context omitted.

> The "cold war" nuclear arms race, which brought the world to the brink of (at least partial) annihilation, is a good recent example. The same era saw big achievements like first human in space, eradication of smallpox, peaceful nuclear exploration etc. It's good to be a skeptic but history does favor the optimists for the most part.

Were any of these big achievements side effects of creating nuclear weapons? If not, then they're not relevant to the issue. I'm not saying nothing else good happened in the past 70 years, but rather that the invention of atomic weapons has permanently placed humanity in a position in which it had never been before: the possibility of wiping out much of the planet, averted only thanks to treaties, Stanislav Petrov[0]…

> Were any of these big achievements side effects of creating nuclear weapons?

The cold aspect of the Cold War was an achievement. Any doubt this was due to creation of nuclear weapons and the threat of their use?

How do you think countries will behave if every country faces being wiped out if it makes war on another country?

To prevent catastrophe I think teaching your citizens to hate other groups (as is done today due to national politics) will become dangerous and mental illness and extremist views will need to be kept in check.

Re: Safe Superintelligence Inc.

#847

Post by mafuy » Thu, Jun 20, 2024, 9:29 AM UTC

jrflowers wrote: post #830

akira2501 wrote: post #706

Earlier quoted context omitted.

Well, I already have an opinion on them. The article seems to purposefully avoid describing what the market and the service here is. There are a limited number of venues where a preteen girl can credibly earn money on social media. Also, if she's earning that money, I'd openly wonder whether a Coogan Account is appropriate and if one exists here. Anyways.. my shock was really over the fact that the mother is basicall…

> Well, I already have an opinion on them. That was the impression I got from your previous post. > There are a limited number of venues where a preteen girl can credibly earn money on social media. … ??? > my shock was really over the fact that the mother is basically saying "I want to sell access to my daughter online but I'm surprised that the biggest spenders are adult men with questionable intentions." This arti…

Noone said they wanted to see pictures. Only to know their nature.

This is a valid request. Insinuations can be inserted anywhere, e.g. to drive clicks, so verification is good practice.

Re: Safe Superintelligence Inc.

#848

Post by yayr » Thu, Jun 20, 2024, 9:34 AM UTC

benreesman wrote: post #753

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

It’s just clearly military R&D at this point. And it’s not even a little bit controversial that cutting edge military R&D is classified in general and to an extreme in wartime. The new thing is the lie that it’s a consumer offering. What’s new is giving the helm to shady failed social network founders with no accountability. These people aren’t retired generals with combat experience. They aren’t tenured professors a…

most technology is dual or multiple use, starting with a rock or knife...

so it is up to the fabric of our society and everyone involved in dealing with the technology, how the rules and boundaries are set.

that there will be military use is obvious. However, it is naive to think one can avoid military use by others by not enabling oneself for it.

Re: Safe Superintelligence Inc.

#849

Post by trashtester » Thu, Jun 20, 2024, 9:35 AM UTC

czl wrote: post #844

chii wrote: post #711

Earlier quoted context omitted.

That's why it's called an arms race, and it does not really end in this predictable manner. The party that's about to lose will use any extrajudicial means to reclaim their victory, regardless of the consequences, because their own destruction would be imminent otherwise. This ultimately leads to violence.

> The party that's about to lose will use any extrajudicial means to reclaim their victory, How will the party about lose know they are about to lose? > regardless of the consequences, because their own destruction would be imminent otherwise. Why would AGI solve things using destruction? Consider how the most inteligent among us view our competition with other living beings. Is destruction the goal? So why would an…

Let's say China realize they're behind in the SI race. They may have achieved AGI, but only barely, while the US may be getting close to ASI takeoff.

Now let's assume they're able to quickly build a large datacenter far underground, complete with a few nuclear reactors and all spare parts, etc, needed. Even a greenhouse (using artificial light) big enough to feed 1000 people.

But they realize that their competitors are about to create ASI at a level that will enable them to completely overrun all of China with self-replicating robots within 100 days.

In a situation, the leadership MAY decide to enter those caves alongside a few soldiers and the best AI researchers, and then simply nuke all US data centers (that are presumably above ground), as well as any other data center that could be a threat, worldwide.

And by doing that, they may (or at least may think) they can buy enough time to win the ASI race, at the cost of a few billion people.

Would they do it? Would we?

Re: Safe Superintelligence Inc.

#850

Post by gary_0 » Thu, Jun 20, 2024, 9:37 AM UTC

sneak wrote: post #840

bradleykingz wrote: post #797

Earlier quoted context omitted.

The way I see it, this is simply a repetition of history. El dorado, the fountain of youth, turning dirt to gold, the holy grail and now... superintelligence.

Human flight, resurrection (cardiopulmonary resuscitation machines), doubling human lifespans, instantaneous long distance communication, all of these things are simply pipe dreams.

Setting foot on the moon, splitting the atom and transmuting elements, curing incurable diseases like genetic blindness and spinal atrophy...

> doubling human lifespans

This is partly a statistical effect of greatly reducing infant mortality (which used to be as bad as 50%) but even that is mind-blowing.