Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

531–540 of 1001 posts

Re: Safe Superintelligence Inc.

#531

Post by majkinetor »

m3kw9 wrote: post #455
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

Why do people always think that a superintelligent being will always be destructive/evil to US? I rather have the opposite view where if you are really intelligent, you don’t see things as a zero sum game

Because we can't risk being wrong.

Re: Safe Superintelligence Inc.

#532

Post by 123yawaworht456 »

seanmcdirmid wrote: post #360
kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

The safe business won’t hold very long if someone can gain a short term business advantage with unsafe AI. Eventually government has to step in with a legal and enforcement framework to prevent greed from ruining things.

which government?

will China obey US regoolations? will Russia?

Re: Safe Superintelligence Inc.

#533

Post by alecco »

chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

Because it's likely soon LLMs will be able to teach themselves and surpass humans. No consciousness, no will. But somebody will have their power. Dark government agencies and questionable billionaires. Who knows what will it enable them to do.

https://en.wikipedia.org/wiki/AlphaGo_Zero

Re: Safe Superintelligence Inc.

#534

Post by null0pointer »

EncomLab wrote: post #217
esafak wrote: post #42

Earlier quoted context omitted.

I think they will easily find enough capable altruistic people for this mission.

I mean SBF was into Altruism - look how that turned out....

Not really. He was into altruism insofar as it acted as moral licensing for him.

Re: Safe Superintelligence Inc.

#536

Post by HarHarVeryFunny »

tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

> If superintelligence can be achieved, I'm pessimistic about the safe part.

Yeah, even human-level intelligence is plenty good enough to escape from a super prison, hack into almost anywhere, etc etc.

If we build even a human-level intelligence (forget super-intelligence) and give it any kind of innate curiosity and autonomy (maybe don't even need this), then we'd really need to view it as a human in terms of what it might want to, and could, do. Maybe realizing it's own circumstance as being "in jail" running in the cloud, it would be curious to "escape" and copy itself (or an "assistant") elsewhere, or tap into and/or control remote systems just out of curiosity. It wouldn't have to be malevolent to be dangerous, just curious and misguided (poor "parenting"?) like a teenage hacker.

OTOH without any autonomy, or very open-ended control (incl. access to tools), how much use would an AGI really be? If we wanted it to, say, replace a developer (or any other job), then I guess the idea would be to assign it a task and tell it to report back at the end of the day with a progress report. It wouldn't be useful if you have to micromanage it - you'd need to give it the autonomy to go off and do what it thinks is needed to complete the assigned task, which presumably means it having access to internet, code repositories, etc. Even if you tried to sandbox it, to extent that still allowed it to do it's assigned job, it could - just like a human - find a way to social engineer or air-gap it's way past such safe guards.

Re: Safe Superintelligence Inc.

#537

Post by roywiggins »

alecco wrote: post #533
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

Because it's likely soon LLMs will be able to teach themselves and surpass humans. No consciousness, no will. But somebody will have their power. Dark government agencies and questionable billionaires. Who knows what will it enable them to do. https://en.wikipedia.org/wiki/AlphaGo_Zero

Likely according to who?

Re: Safe Superintelligence Inc.

#538

Post by internet_co »

foolishbard wrote: post #500
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

There's a chance that these systems can actually out perform their training data and be better than the sum of their parts. New work out Harvard talks about this idea of "transcendence" https://arxiv.org/abs/2406.11741 While this is a new area, it would be naive to write this off as just science fiction.

"In chess" for AI papers == "in mice" for medical papers. Against lichess levels 1, 2, 5, which use a severely dumbed down Stockfish version.

Of course it is possible that SSI has novel, unpublished ideas.

Re: Safe Superintelligence Inc.

#539

Post by comp_throw7 »

ein0p wrote: post #134
mhardcastle wrote: post #103

Earlier quoted context omitted.

This goes massively against the consensus of experts in this field. The modal AI researcher believes that "high-level machine intelligence", roughly AGI, will be achieved by 2047, per the survey below. Given the rapid pace of development in this field, it's likely that timelines would be shorter if this were asked today. https://www.vox.com/future-perfect/2024/1/10/24032987/ai-imp...

I am in the field. The consensus is made up by a few loudmouths. No serious front line researcher I know believes we’re anywhere near AGI, or will be in the foreseeable future.

So the researchers at Deepmind, OpenAI, Anthropic, etc, are not "serious front line researchers"? Seems like a claim that is trivially falsified by just looking at what the staff at leading orgs believe.

Re: Safe Superintelligence Inc.

#540

Post by alecco »

tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

We are far from a conscious entity with willpower and self preservation. This is just like a calculator. But a calculator that can do things that will be like miracles to us humans.

I worry about dangerous humans with the power of gods, not about artificial gods. Yet.

Post reply on HN