Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

511–520 of 1001 posts

Re: Safe Superintelligence Inc.

#511

Post by exe34 »

Nasrudith wrote: post #495
exe34 wrote: post #185

Earlier quoted context omitted.

the aligning means it should do what the board of directors wants, not what's good for society.

Poisoning Socrates was done because it was "good for society". I'm frankly even more suspicious of "good for society" than the average untrustworthy board of directors.

seriously? you're more worried about what your elected officials might legislate than what a board of directors whose job is to make profits go brrr at all costs, including poisoning the environment, exploiting people and avoiding taxes?

Re: Safe Superintelligence Inc.

#512

Post by Der_Einzige »

whimsicalism wrote: post #393
torginus wrote: post #382

Earlier quoted context omitted.

Half the advancements around Stable Diffusion (Controlnet etc.) came from internet randoms wanting better anime waifus

advancements around parameter efficient fine tuning came from internet randoms because big cos don’t care about PEFT

... Sort of?

HF is sort of big now. Stanford is well funded and they did PyReft.

Re: Safe Superintelligence Inc.

#513

Post by null_point »

delichon wrote: post #430
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

If superintelligence can be achieved, I'm pessimistic that a team committed to doing it safely can get there faster than other teams without the safety. They may be wearing leg shackles in a foot race with the biggest corporations, governments and everyone else. For the sufficiently power hungry, safety is not a moat.

I'm on the fence with this because it's plausible that some critical component of achieving superintelligence might be discovered more quickly by teams that, say, have sophisticated mechanistic interpretability incorporated into their systems.

Re: Safe Superintelligence Inc.

#514

Post by ldjkfkdsjnv »

dbish wrote: post #505
ldjkfkdsjnv wrote: post #119

Earlier quoted context omitted.

Are you seriously asking how the most talented AI researcher of the last decade will be able to recruit other researchers? Ilya saw the potential of deep learning way before other machine learning academics.

Sorry, are you attributing all of deep learning research to Ilya? The most talented AI researcher of the last decade?

Not attributing all of it

Re: Safe Superintelligence Inc.

#515

Post by whimsicalism »

Der_Einzige wrote: post #512
whimsicalism wrote: post #393

Earlier quoted context omitted.

advancements around parameter efficient fine tuning came from internet randoms because big cos don’t care about PEFT

... Sort of? HF is sort of big now. Stanford is well funded and they did PyReft.

HF is not very big, Stanford doesn’t have lots of compute.

Neither of these are even remotely big labs like what I’m discussing

Re: Safe Superintelligence Inc.

#516

Post by null_point »

m3kw9 wrote: post #455
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

Why do people always think that a superintelligent being will always be destructive/evil to US? I rather have the opposite view where if you are really intelligent, you don’t see things as a zero sum game

They don't think superintelligence will "always" be destructive to humanity. They believe that we need to ensure that a superintelligence will "never" be destructive to humanity.

Re: Safe Superintelligence Inc.

#517

Post by ben_w »

softg wrote: post #510
ben_w wrote: post #454

Earlier quoted context omitted.

No more so than trying to control a supersonic aircraft when we can't even control pigeons.

I know nothing about physics. If I came across some magic algorithm that occasionally poops out a plane that works 90 percent of the time, would you book a flight in it? Sure, we can improve our understanding of how NNs work but that isn't enough. How are humans supposed to fully understand and control something that is smarter than themselves by definition? I think it's inevitable that at some point that smart thing…

> I know nothing about physics. If I came across some magic algorithm that occasionally poops out a plane that works 90 percent of the time, would you book a flight in it?

With this metaphor you seem to be saying we should, if possible, learn how to control AI? Preferably before anyone endangers their lives due to it? :)

> I think it's inevitable that at some point that smart thing will behave in ways humans don't expect.

Naturally.

The goal, at least for those most worried about this, is to make that surprise be not a… oh, I've just realised a good quote:

""" the kind of problem "most civilizations would encounter just once, and which they tended to encounter rather in the same way a sentence encountered a full stop." """ - https://en.wikipedia.org/wiki/Excession#Outside_Context_Prob...

Not that.

Re: Safe Superintelligence Inc.

#519

Post by majormajor »

foolishbard wrote: post #500
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

There's a chance that these systems can actually out perform their training data and be better than the sum of their parts. New work out Harvard talks about this idea of "transcendence" https://arxiv.org/abs/2406.11741 While this is a new area, it would be naive to write this off as just science fiction.

It would be nice if authors wouldn't use a loaded-as-fuck word like "transcendence" for "the trained model can sometimes achieve better performance than all [chess] players in the dataset" because while certainly that's demonstrating an impressive internalization of the game, it's also something that many humans can also do. The machine, of course, can be scaled in breadth and performance, but... "transcendence"? Are they trying to be mis-interpreted?

Re: Safe Superintelligence Inc.

#520

Post by ben_w »

sroussey wrote: post #508
ben_w wrote: post #454

Earlier quoted context omitted.

No more so than trying to control a supersonic aircraft when we can't even control pigeons.

I can shoot down a pigeon that’s overhead pretty easily, but not so with an overhead supersonic jet.

If that's your standard of "control", then we can definitely "control" human intelligence.
Post reply on HN