Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

621–630 of 1001 posts

Re: Safe Superintelligence Inc.

#621

Post by lupire »

vineyardmike wrote: post #593
reissbaker wrote: post #577

Earlier quoted context omitted.

I'm pretty sure "Altman and company" don't have much to do with this — this is Ilya, who pretty famously tried to get Altman fired, and then himself left OpenAI in the aftermath. Ilya is a brilliant researcher who's contributed to many foundational parts of deep learning (including the original AlexNet); I would say I'm somewhat pessimistic based on the "safety" focus — I don't think LLMs are particularly dangerous,…

> I don't think LLMs are particularly dangerous “Everyone” who works in deep AI tech seems to constantly talk about the dangers. Either they’re aggrandizing themselves and their work, or they’re playing into sci-fi fear for attention or there is something the rest of us aren’t seeing. I’m personally very skeptical there is any real dangers today. If I’m wrong, I’d love to see evidence. Are foundation models before fi…

That's what Safe Super intelligence misses. Superintelligence isn't practically more dangerous. Super stupidity is already here, and bad enough.

Re: Safe Superintelligence Inc.

#622

Post by qeternity »

jen729w wrote: post #580
Sharlin wrote: post #441

Earlier quoted context omitted.

> You can make a safe AI today, but what happens when the next person is managing things? The point of safe superintelligence , and presumably the goal of SSI Inc., is that there won't be a next (biological) person managing things afterwards. At least none who could do anything to build a competing unsafe SAI. We're not talking about the banal definition of "safety" here. If the first superintelligence has any reason…

I wonder how many people panicking about these things have ever visited a data centre. They have big red buttons at the end of every pod. Shuts everything down. They have bigger red buttons at the end of every power unit. Shuts everything down. And down at the city, there’s a big red button at the biggest power unit. Shuts everything down. Having arms and legs is going to be a significant benefit for some time yet. I…

Have you seen all of the autonomous cars, drones and robots we've built?

Re: Safe Superintelligence Inc.

#623

Post by bamboozled »

falcor84 wrote: post #613
jen729w wrote: post #580

Earlier quoted context omitted.

I wonder how many people panicking about these things have ever visited a data centre. They have big red buttons at the end of every pod. Shuts everything down. They have bigger red buttons at the end of every power unit. Shuts everything down. And down at the city, there’s a big red button at the biggest power unit. Shuts everything down. Having arms and legs is going to be a significant benefit for some time yet. I…

It's been more than a decade now since we first saw botnets based on stealing AWS credentials and running arbitrary code on them (e.g. for crypto mining) - once an actual AI starts duplicating itself in this manner, where's the big red button that turns off every single cloud instance in the world?

This is making a lot of assumptions like…a super intelligence can easily clone itself…maybe such an entity would require specific hardware to run ?

Re: Safe Superintelligence Inc.

#624

Post by lazide »

joantune wrote: post #620
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

> But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able to be creative outside of deriving things humans have already made before. Granted, they‘re really good at doing that, but not much else. Remove token, and that's what we humans do. Like, you need to realize that neural networks came to be because someone had the idea…

Except it really, actually, isn’t.

People don’t ‘think’ the same way, even if some part of how humans think seems to be somewhat similar some of the time.

That is an important distinction.

This is the hype cycle.

Re: Safe Superintelligence Inc.

#625

Post by lupire »

Kerb_ wrote: post #592
majormajor wrote: post #519

Earlier quoted context omitted.

It would be nice if authors wouldn't use a loaded-as-fuck word like "transcendence" for "the trained model can sometimes achieve better performance than all [chess] players in the dataset" because while certainly that's demonstrating an impressive internalization of the game, it's also something that many humans can also do. The machine, of course, can be scaled in breadth and performance, but... "transcendence"? Are…

It transcends the training data, I get the usage intended but it certainly is ripe for misinterpretation

That's trivial though, conceptually. Every regression line transcends the training data. We've had that since Wisdom of Crowds.

Re: Safe Superintelligence Inc.

#626

Post by lazide »

TaylorAlexander wrote: post #585
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

> why are we talking about things like this? > this is such a transparent attention grab (and, by extension, money grab by being overvalued by investors and shareholders) Ilya believes transformers can be enough to achieve superintelligence (if inefficiently). He is concerned that companies like OpenAI are going to succeed at doing it without investing in safety, and they're going to unleash a demon in the process. I…

His paycheck is already dependent on people believing this world view. It’s important to not lose sight of that.

Re: Safe Superintelligence Inc.

#627

Post by [deleted] »

insane_dreamer wrote: post #607
roywiggins wrote: post #559

Earlier quoted context omitted.

Even without business incentives, the military advantages of AI would inventivize governments to develop it anyway, like they did with nuclear weapons. Nuclear weapons are inherently unsafe, there are some safeguards around them, but they are ultimately dangerous weapons.

If someone really wanted to use nukes, they would have been used by now. What has protected us is not technology (in the aftermath of the USSR it wasn't that difficult to steal a nuke), but rather lack of incentives. A bad actor doesn't have much to gain by detonating a nuke (unless they're deranged and want to see people die for the pleasure of it). OK, you could use it as blackmail, which North Korea essentially tr…

[deleted]

Re: Safe Superintelligence Inc.

#628

Post by joe_the_user »

chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

I actually do doubt that LLMs will create AGI but when these systems are emulating a variety of human behaviors in a way that isn't directly programmed and is good enough to be useful, it seems foolish to not take notice.

The current crop of systems is a product of the transformers architecture - an innovation that accelerated performance significantly. I put the odds another changing everything but I don't think we can entirely discount the possibility. That no one understands these systems cuts both ways.

Re: Safe Superintelligence Inc.

#629

Post by lazide »

Yoric wrote: post #600
reissbaker wrote: post #577

Earlier quoted context omitted.

I'm pretty sure "Altman and company" don't have much to do with this — this is Ilya, who pretty famously tried to get Altman fired, and then himself left OpenAI in the aftermath. Ilya is a brilliant researcher who's contributed to many foundational parts of deep learning (including the original AlexNet); I would say I'm somewhat pessimistic based on the "safety" focus — I don't think LLMs are particularly dangerous,…

I actually feel that they can be very dangerous. Not because of the fabled AGI, but because 1. they're so good at showing the appearance of being right; 2. their results are actually quite unpredictable, not always in a funny way; 3. C-level executives actually believe that they work. Combine this with web APIs or effectors and this is a recipe for disaster.

The ‘plausible text generator’ element of this is perfect for mass fraud and propaganda.

Re: Safe Superintelligence Inc.

#630

Post by zztop44 »

vineyardmike wrote: post #593
reissbaker wrote: post #577

Earlier quoted context omitted.

I'm pretty sure "Altman and company" don't have much to do with this — this is Ilya, who pretty famously tried to get Altman fired, and then himself left OpenAI in the aftermath. Ilya is a brilliant researcher who's contributed to many foundational parts of deep learning (including the original AlexNet); I would say I'm somewhat pessimistic based on the "safety" focus — I don't think LLMs are particularly dangerous,…

> I don't think LLMs are particularly dangerous “Everyone” who works in deep AI tech seems to constantly talk about the dangers. Either they’re aggrandizing themselves and their work, or they’re playing into sci-fi fear for attention or there is something the rest of us aren’t seeing. I’m personally very skeptical there is any real dangers today. If I’m wrong, I’d love to see evidence. Are foundation models before fi…

They reduce the marginal cost of producing plausible content to effectively zero. When combined with other societal and technological shifts, that makes them dangerous to a lot of things: healthy public discourse, a sense of shared reality, people’s jobs, etc etc

But I agree that it’s not at all clear how we get from ChatGPT to the fabled paperclip demon.

Post reply on HN