Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

631–640 of 1001 posts

Re: Safe Superintelligence Inc.

#631

Post by nradov »

bbor wrote: post #498
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

Well, an entire industry of researchers, which used to be divided, is now uniting around calls to slow development and emphasize safety (like, “dissolve companies” emphasis not “write employee handbooks” emphasis). They’re saying, more-or-less in unison, that GPT3 was an unexpected breakthrough in the Frame Problem, based on Judea Pearl’s prescient predictions. If we agree on that, there are two options: 1. They’ve a…

We don't agree on that. They're just making things up with no real scientific evidence. There are way more than 2 options.

Re: Safe Superintelligence Inc.

#632

Post by theptip »

jen729w wrote: post #580
Sharlin wrote: post #441

Earlier quoted context omitted.

> You can make a safe AI today, but what happens when the next person is managing things? The point of safe superintelligence , and presumably the goal of SSI Inc., is that there won't be a next (biological) person managing things afterwards. At least none who could do anything to build a competing unsafe SAI. We're not talking about the banal definition of "safety" here. If the first superintelligence has any reason…

I wonder how many people panicking about these things have ever visited a data centre. They have big red buttons at the end of every pod. Shuts everything down. They have bigger red buttons at the end of every power unit. Shuts everything down. And down at the city, there’s a big red button at the biggest power unit. Shuts everything down. Having arms and legs is going to be a significant benefit for some time yet. I…

Trouble is, in practice what you would need to do might be “turn off all of Google’s datacenters”. Or perhaps the thing manages to secure compute in multiple clouds (which is what I’d do if I woke up as an entity running on a single DC with a big red power button on it).

The blast radius of such decisions are large enough that this option is not trivial as you suggest.

Re: Safe Superintelligence Inc.

#633

Post by sb77 »

Kerb_ wrote: post #592
majormajor wrote: post #519

Earlier quoted context omitted.

It would be nice if authors wouldn't use a loaded-as-fuck word like "transcendence" for "the trained model can sometimes achieve better performance than all [chess] players in the dataset" because while certainly that's demonstrating an impressive internalization of the game, it's also something that many humans can also do. The machine, of course, can be scaled in breadth and performance, but... "transcendence"? Are…

It transcends the training data, I get the usage intended but it certainly is ripe for misinterpretation

The word for that is "generalizes" or "generalization" and it has existed for a very long time.

Re: Safe Superintelligence Inc.

#634

Post by roywiggins »

marshray wrote: post #579
roywiggins wrote: post #542

Earlier quoted context omitted.

See, this sort of claim I am instantly skeptical of. Nobody has ever caught a human brain producing or storing tokens, and certainly the subjective experience of, say, throwing a ball, doesn't involve symbols of any kind.

> Nobody has ever caught a human brain producing or storing tokens Do you remember learning how to read and write? What are spelling tests? What if "subjective experience" isn't essential, or is even just a distraction, for a great many important tasks?

Entirely possible. Lots of things exhibit complex behavior that probably don't have subjective experience.

My point is just that the evidence for "humans are just token prediction machines and nothing more" is extremely lacking, but there's always someone in these discussions who asserts it like it's obvious.

Re: Safe Superintelligence Inc.

#636

Post by lupire »

foobiekr wrote: post #476
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

It's no mystery, AI has attracted tons of grifters trying to cash out before the bubble pops, and investors aren't really good at filtering.

Why are you assigning moral agency where there may be none? These so called "grifters" are just token predictors writing business plans (prompts) with the highest computed probability of triggering $ + [large number] token pair from venture capital token predictors.

Re: Safe Superintelligence Inc.

#637

Post by roywiggins »

insane_dreamer wrote: post #607
roywiggins wrote: post #559

Earlier quoted context omitted.

Even without business incentives, the military advantages of AI would inventivize governments to develop it anyway, like they did with nuclear weapons. Nuclear weapons are inherently unsafe, there are some safeguards around them, but they are ultimately dangerous weapons.

If someone really wanted to use nukes, they would have been used by now. What has protected us is not technology (in the aftermath of the USSR it wasn't that difficult to steal a nuke), but rather lack of incentives. A bad actor doesn't have much to gain by detonating a nuke (unless they're deranged and want to see people die for the pleasure of it). OK, you could use it as blackmail, which North Korea essentially tr…

Sure, that just makes the military incentives to develop such a thing even stronger. All I mean is that business incentives don't really come into it, as long as there is competition, someone's going to want to build weapons to gain advantage, whether it's a business or a government.

Re: Safe Superintelligence Inc.

#638

Post by zombiwoof »

TaylorAlexander wrote: post #585
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

> why are we talking about things like this? > this is such a transparent attention grab (and, by extension, money grab by being overvalued by investors and shareholders) Ilya believes transformers can be enough to achieve superintelligence (if inefficiently). He is concerned that companies like OpenAI are going to succeed at doing it without investing in safety, and they're going to unleash a demon in the process. I…

Ilya has never said transformers are the end all be all

Re: Safe Superintelligence Inc.

#639

Post by greatpostman »

lazide wrote: post #626
TaylorAlexander wrote: post #585

Earlier quoted context omitted.

> why are we talking about things like this? > this is such a transparent attention grab (and, by extension, money grab by being overvalued by investors and shareholders) Ilya believes transformers can be enough to achieve superintelligence (if inefficiently). He is concerned that companies like OpenAI are going to succeed at doing it without investing in safety, and they're going to unleash a demon in the process. I…

His paycheck is already dependent on people believing this world view. It’s important to not lose sight of that.

Dude he’s probably worth > 1 Billion.
Post reply on HN