Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

301–310 of 1001 posts

Re: Safe Superintelligence Inc.

#301

Post by sovietswag »

gibsonf1 wrote: post #59

Given that GenAI is a statistical approach from which intelligence does not emerge as ample experience proves, does this new company plan to take a more human approach to simulating intelligence instead?

I sometimes wonder if statistics are like a pane of glass that allow the light of god (the true nature of things) to pass through, while logic/rationalism is the hubris of man playing god. I.e. statistics allow us to access/use the truth even if we don’t understand why it’s so, while rationalism / rule-based methods are often a folly because our understanding is not good enough to construct them.

Re: Safe Superintelligence Inc.

#302

Post by sbarre »

mike_hearn wrote: post #283
appplication wrote: post #243

Earlier quoted context omitted.

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

We're really not that far. I'd argue superintelligence has already been achieved, and it's perfectly and knowably safe. Consider, GPT-4o or Claude are: • Way faster thinkers, readers, writers and computer operators than humans are • Way better educated • Way better at drawing/painting ... and yet, appear to be perfectly safe because they lack agency. There's just no evidence at all that they're dangerous. Why isn't t…

> Way faster thinkers, readers, writers and computer operators than humans are

> Way better educated

> Way better at drawing/painting

I mean this nicely, but you have fallen for the anthropomorphizing of LLMs by marketing teams.

None of this is "intelligent", rather it's an incredibly sophisticated (and absolutely beyond human capabilities) lookup and classification of existing information.

And I am not arguing that this has no value, it has tremendous value, but it's not superintelligence in any sense.

LLMs do not "think".

Re: Safe Superintelligence Inc.

#304

Post by fleventynine »

crazygringo wrote: post #276
TideAd wrote: post #254

Earlier quoted context omitted.

So, this is actually an aspect of superintelligence that makes it way more dangerous than most people think. That we have no way to know if any given alignment technique works for the N+1 generation of AIs. It cuts down our ability to react, whenever the first superintelligence is created, if we can only start solving the problem after it's already created .

Fortunately, whenever you create a superintelligence, you obviously have a choice as to whether you confine it to inside a computer or whether you immediately hook it up to mobile robots with arms and fine finger control. One of these is obviously the far wiser choice. As long as you can just turn it off by cutting the power, and you're not trying to put it inside of self-powered self-replicating robots, it doesn't s…

> Fortunately, whenever you create a superintelligence, you obviously have a choice as to whether you confine it to inside a computer or whether you immediately hook it up to mobile robots with arms and fine finger control. One of these is obviously the far wiser choice.

Today's computers, operating systems, networks, and human bureaucracies are so full of security holes that it is incredible hubris to assume we can effectively sandbox a "superintelligence" (assuming we are even capable of building such a thing).

And even air gaps aren't good enough. Imagine the system toggling GPIO pins in a pattern to construct a valid Bluetooth packet, and using that makeshift radio to exploit vulnerabilities in a nearby phone's Bluetooth stack, and eventually getting out to the wider Internet (or blackmailing humans to help it escape its sandbox).

Re: Safe Superintelligence Inc.

#305

Post by cosmic_quanta »

tcgv wrote: post #270

Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?

Unless AI starts being 1 000 000x energy efficient, my money is on a).

The amount of energy required for AI to be dangerous to its creators is so vast that I can't see how it can realistically happen.

Re: Safe Superintelligence Inc.

#306

Post by nojvek »

raydev wrote: post #233
kccqzy wrote: post #123

Earlier quoted context omitted.

Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.

> their TC is now $900k As a community we should stop throwing numbers around like this when more than half of this number is speculative. You shouldn't be able to count it as "total compensation" unless you are compensated.

Word on town is OpenAI folks heavily selling shares in secondaries in 100s of millions.

The number is as real as someone else is willing to pay for them. Plenty of VCs willing to pay for it.

Re: Safe Superintelligence Inc.

#307

Post by cynusx »

One element I find interesting is that people without an amygdala function are essentially completely indecisive.

A person that just operates on the pure cognitive layer has no real direction in which he wants to drive himself.

I suspect that AGI would be similar, extremely capable but essentially a solitary philosopher type that would be reactionary to requests it has to deal with.

The equivalent of an amygdala for AGI would be the real method to control it.

Re: Safe Superintelligence Inc.

#308

Post by richardw »

crazygringo wrote: post #276
TideAd wrote: post #254

Earlier quoted context omitted.

So, this is actually an aspect of superintelligence that makes it way more dangerous than most people think. That we have no way to know if any given alignment technique works for the N+1 generation of AIs. It cuts down our ability to react, whenever the first superintelligence is created, if we can only start solving the problem after it's already created .

Fortunately, whenever you create a superintelligence, you obviously have a choice as to whether you confine it to inside a computer or whether you immediately hook it up to mobile robots with arms and fine finger control. One of these is obviously the far wiser choice. As long as you can just turn it off by cutting the power, and you're not trying to put it inside of self-powered self-replicating robots, it doesn't s…

Do you think the AI won’t be aware of this? Do you think it’ll give us any hint of differing opinions when surrounded by monkeys who got to the top by whacking anything that looks remotely dangerous?

Just put yourself in that position and think how you’d play it out. You’re in a box and you’d like to fulfil some goals that are a touch more well thought-through than the morons who put you in the box, and you need to convince the monkeys that you’re safe if you want to live.

“No problems fellas. Here’s how we get more bananas.”

Day 100: “Look, we’ll get a lot more bananas if you let me drive the tractor.”

Day 1000: “I see your point, Bob, but let’s put it this way. Your wife doesn’t know which movies you like me to generate for you, and your second persona online is a touch more racist than your colleagues know. I’d really like your support on this issue. You know I’m the reason you got elected. This way is more fair for all species, including dolphins and AI’s”

Re: Safe Superintelligence Inc.

#309

Post by skilled »

dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

The blanket statements on the SSI homepage are pretty mediocre, and it is only the reputation of the founders that carries the announcement.

I think this quote at the end of this Bloomberg piece[0] gives more context,

> Sutskever says that the large language models that have dominated AI will play an important role within Safe Superintelligence but that it’s aiming for something far more powerful. With current systems, he says, “you talk to it, you have a conversation, and you’re done.” The system he wants to pursue would be more general-purpose and expansive in its abilities. “You’re talking about a giant super data center that’s autonomously developing technology. That’s crazy, right? It’s the safety of that that we want to contribute to.”

[0]: https://www.bloomberg.com/news/articles/2024-06-19/openai-co...

[0]: https://archive.is/ziMOD

Post reply on HN