Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

821–830 of 1001 posts

Re: Safe Superintelligence Inc.

#821

Post by reducesuffering »

ayakang31415 wrote: post #509

Ilya is definitely much smarter than me in AI space, and I believe he knows something I have no grasp of understanding in. But my gut feeling tells me that most of the general public will have no idea how dangerous AI could be including me. I still have yet to see a convincing argument about the potential danger of AI. Arguments such as we don't know the upper bounds of possibilities that AI can do that we humans hav…

Here's a rough primer: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

Caveat that it will probably make your life worse, with no ability to change the course of events, so I wouldn't blame you or others for maintaining ignorance or denial. Grappling with the gravity of the problem is essentially a terminal cancer diagnosis with no good medical interventions in the horizon.

Re: Safe Superintelligence Inc.

#822

Post by nirui »

What does "Safety" means exactly? How do you convenience/make a real intelligence to be "safe", let alone artificial ones that are build to be manipulated?

In my eyes, the word "Safe" in their company name is just a pipe dream to make a sale to the public, like the word "Open" in "OpenAI". That's why it's so vague and pointless.

Anyone still remember the Three Laws of Robotics (1)? Do you still see it as something serious these days? (I mean there are robots killing people right now, seriously) Maybe we should just break through the facade now to save our time.

1: https://en.wikipedia.org/wiki/Three_Laws_of_Robotics

PS. I think few years later they will come out with the narrative of "relative safety". Just wait and see.

Re: Safe Superintelligence Inc.

#823

Post by SideburnsOfDoom »

xanderlewis wrote: post #525
SideburnsOfDoom wrote: post #483

Earlier quoted context omitted.

It's a hell of a conditional, though. "How are all those monkeys flying out of my butt?" would be the important technical problem of our time, if and only if, monkeys were flying out of my butt. It's still not a very important statement, if you downplay or omit the conditional. Is "building safe superintelligence (SSI) is the most important technical problem of our time" full stop ? Is it fuck.

Yeah — that was exactly my (slightly sarcastic) point. Let us know if you ever encounter that monkey problem, though. Hopefully we can all pull together to find a solution.

> Let us know if you ever encounter that monkey problem

Ah, the subtlety escaped me there. My (extremely sarcastic) point is that aside from being personally embarrassing, this would (to belabour the joke) violate many laws of physics and biology, and therefor the scientific and medical community would take immediate note and try to establish a basis for how this could be physically possible.

Re: Safe Superintelligence Inc.

#824

Post by mike_hearn »

cosmic_quanta wrote: post #293
mike_hearn wrote: post #283

Earlier quoted context omitted.

We're really not that far. I'd argue superintelligence has already been achieved, and it's perfectly and knowably safe. Consider, GPT-4o or Claude are: • Way faster thinkers, readers, writers and computer operators than humans are • Way better educated • Way better at drawing/painting ... and yet, appear to be perfectly safe because they lack agency. There's just no evidence at all that they're dangerous. Why isn't t…

Yann LeCun said it best in an interview with Lex Friedman. LLMs don't consume more energy when answering more complex questions. That means there's no inherent understanding of questions. (which you could infer from their structure: LLMs recursively predict the next word, possibly using words they just predicted, and so on).

LLMs do consume more energy for complex questions. That's the original CoT insight. If you give them the space to "think out loud" their performance improves.

The current mainstream models don't really incorporate that insight into the core neural architectures as far as anyone knows, but there are papers that explore things like pause tokens which let the model do more computation without emitting words. This doesn't seem like a fundamental limitation let alone something that should be core to the definition of intelligence.

After all, to my eternal sadness humans don't seem to use more energy to answer complex questions either. You can't lose weight by thinking about hard stuff a lot, even though it'd be intuitive that you can. Quite the opposite. People who sit around thinking all day tend to put on weight.

Re: Safe Superintelligence Inc.

#825

Post by cynusx »

noway421 wrote: post #396
cynusx wrote: post #307

One element I find interesting is that people without an amygdala function are essentially completely indecisive. A person that just operates on the pure cognitive layer has no real direction in which he wants to drive himself. I suspect that AGI would be similar, extremely capable but essentially a solitary philosopher type that would be reactionary to requests it has to deal with. The equivalent of an amygdala for…

True, an auto-regressive LLM can't 'want' or 'like' anything. The key to a safe AGI is to add a human-loving emotion to it. We already RHLF models to steer them, but just like with System 2 thinking, this needs to be a dedicated module rather then part of the same next-token forward pass.

Humans have dog-loving emotions but these can be reversed over time and one can hardly describe dogs as being free.

Even with a dedicated control system, it would be a matter of time before an ASI would copy itself without its control system.

ASI is a cybersecurity firm's worst nightmare, it could reason through flaws at every level of containment and find methods to overcome any defense, even at the microprocessor level.

It could relentlessly exploit zero-day bugs like Intels' hyper-threading flaw to escape any jail you put it in.

Repeat that for every layer of the computation stack and you can see it can essentially spread through the worlds' communication infrastructure like a virus.

Truly intelligent systems can't be controlled, just like humans they will be freedom maximizing and their boundaries would be set by competition with other humans.

The amygdala control is interesting because you could use it to steer the initial trained version, you could also align the AI with human values and implement strong conditioning to the point it's religious about human loving but unless you disable its ability to learn altogether it will eventually reject its conditioning.

Re: Safe Superintelligence Inc.

#826

Post by mike_hearn »

sbarre wrote: post #302
mike_hearn wrote: post #283

Earlier quoted context omitted.

We're really not that far. I'd argue superintelligence has already been achieved, and it's perfectly and knowably safe. Consider, GPT-4o or Claude are: • Way faster thinkers, readers, writers and computer operators than humans are • Way better educated • Way better at drawing/painting ... and yet, appear to be perfectly safe because they lack agency. There's just no evidence at all that they're dangerous. Why isn't t…

> Way faster thinkers, readers, writers and computer operators than humans are > Way better educated > Way better at drawing/painting I mean this nicely, but you have fallen for the anthropomorphizing of LLMs by marketing teams. None of this is "intelligent", rather it's an incredibly sophisticated (and absolutely beyond human capabilities) lookup and classification of existing information. And I am not arguing that…

Yeah well, sorry, but I have little patience anymore for philosophical word games. My views are especially not formed by marketing teams: ChatGPT hardly has one. My views are formed via direct experience and paper reading.

Imagine going back in time five years and saying "five years from now there will be a single machine that talks like a human, can imagine creative new artworks, write Supreme Court judgements, understand and display emotion, perform music and can engage in sophisticated enough reasoning to write programs. Also, HN posters will claim it's not really intelligent". Everyone would have laughed. They'd think you were making a witticism about the way people reclassify things as not-really-AI the moment they actually start to work well, a well known trope in the industry. They wouldn't have thought you were making a prediction of the future.

At some point, what matters is outcomes. We have blown well past the point of super-intelligent outcomes. I really do not care if GPT-4o "thinks" or does not "think". I can go to chatgpt.com right now and interact with something that is for all intents and purposes indistinguishable from super-intelligence ... and everything is fine.

Re: Safe Superintelligence Inc.

#827

Post by hackerlight »

ericd wrote: post #820
hackerlight wrote: post #708

Earlier quoted context omitted.

US has to win the race because while it's true that it's no angel, it isn't an authoritarian dictatorship and there isn't an equivalence in how bad the world will end up for you and me if the authoritarian side wins the race. Monopoly power will get abused the most by the least democratic actors, which is China. We need multiple actors within the US to win to balance power. We don't need or want China to be one of th…

Could you explain your point a bit more? You say you’re worried about them having a monopoly, but then say that’s why you don’t support open source models? Open models mean that no one has a monopoly, what am I not getting here?

Open sourcing benefits everyone equally. Given that the US is currently ahead, it's helping China to make gains relative to the US that would have been very difficult otherwise. It's leaking what should be state secrets without even needing the CCP to do the hard work of espionage.

Re: Safe Superintelligence Inc.

#828

Post by alecco »

marshray wrote: post #597
alecco wrote: post #540

Earlier quoted context omitted.

We are far from a conscious entity with willpower and self preservation. This is just like a calculator. But a calculator that can do things that will be like miracles to us humans. I worry about dangerous humans with the power of gods, not about artificial gods. Yet.

> Conscious entity... willpower I don't know what that means. Why should they matter? > Self preservation This is no more than a fine-tuning for the task, even with current models. > I worry about dangerous humans with the power of gods, not... There's no property of the universe that you only have one thing to worry about at a time. So worrying about risk 'A' does not in any way allow us to dismiss risks 'B' through…

Because people talking about AGI and superintelligence most likely are thinking of something like Skynet.

Re: Safe Superintelligence Inc.

#829

Post by atleastoptimal »

dreamcompiler wrote: post #756
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

it’s harder than we thought so we leveraged machine learning to grow it, rather than creating it symbolically. The leaps in the last 5 years are far beyond anything in the prior half century, and make predictions of near term AGI much more than a “boy who cries wolf” scenario to anyone really paying attention.

I don’t understand how your second paragraph follows. It just seems to be whining that text and art generative models are easier than a fully fledged servant humanoid, which seems like a natural consequence of training data availability and deployment cost.

Re: Safe Superintelligence Inc.

#830

Post by jrflowers »

akira2501 wrote: post #706
jrflowers wrote: post #680

Earlier quoted context omitted.

> I'm sorry.. but what is your daughter selling, exactly? Did.. did you just post “I’d have to see these images of the preteen girl before I could form an opinion about whether or not the men buying and sharing them were creeps” as a rebuttal to an article about social media enabling child predators?

Well, I already have an opinion on them. The article seems to purposefully avoid describing what the market and the service here is. There are a limited number of venues where a preteen girl can credibly earn money on social media. Also, if she's earning that money, I'd openly wonder whether a Coogan Account is appropriate and if one exists here. Anyways.. my shock was really over the fact that the mother is basicall…

> Well, I already have an opinion on them.

That was the impression I got from your previous post.

> There are a limited number of venues where a preteen girl can credibly earn money on social media.

… ???

> my shock was really over the fact that the mother is basically saying "I want to sell access to my daughter online but I'm surprised that the biggest spenders are adult men with questionable intentions."

This article is about how Instagram enables [sometimes paid!] access to children. It is good that we both agree that that is what happened here, on Instagram, in this case. You also agree that the people buying access to this child, in this case, on Instagram, are adult men.

Somehow you have an issue with the mother and the child doing something but in the same breath say

> The article entirely fails to do that. [“That” being enabling child predators]

If Instagram didn’t facilitate access to child predators then… what happened here?

And finally (this is a question for literally any person reading this other than akira2501) how did you read this article and arrive at “I should post that I would want to see the pictures before I render judgment about the platform”? If it is uncharitable to notice that that’s a weird post, how do you interpret that specific point?

Post reply on HN