Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

541–550 of 1001 posts

Re: Safe Superintelligence Inc.

#541

Post by AgentME »

null_point wrote: post #513
delichon wrote: post #430

Earlier quoted context omitted.

If superintelligence can be achieved, I'm pessimistic that a team committed to doing it safely can get there faster than other teams without the safety. They may be wearing leg shackles in a foot race with the biggest corporations, governments and everyone else. For the sufficiently power hungry, safety is not a moat.

I'm on the fence with this because it's plausible that some critical component of achieving superintelligence might be discovered more quickly by teams that, say, have sophisticated mechanistic interpretability incorporated into their systems.

A point of evidence in this direction is that RLHF was developed originally as an alignment technique and then it turned out to be a breakthrough that also made LLMs better and more useful. Alignment and capabilities work aren't necessarily at odds with each other.

Re: Safe Superintelligence Inc.

#542

Post by roywiggins »

ohcmon wrote: post #494
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

> glorified token predicting machine trained on existing data (made by humans) sorry to disappoint, but human brain fits the same definition

See, this sort of claim I am instantly skeptical of. Nobody has ever caught a human brain producing or storing tokens, and certainly the subjective experience of, say, throwing a ball, doesn't involve symbols of any kind.

Re: Safe Superintelligence Inc.

#543

Post by CyberDildonics »

roywiggins wrote: post #537
alecco wrote: post #533

Earlier quoted context omitted.

Because it's likely soon LLMs will be able to teach themselves and surpass humans. No consciousness, no will. But somebody will have their power. Dark government agencies and questionable billionaires. Who knows what will it enable them to do. https://en.wikipedia.org/wiki/AlphaGo_Zero

Likely according to who?

Whoever needs money from investors who don't understand LLMs.

Re: Safe Superintelligence Inc.

#544

Post by tasoeur »

nuz wrote: post #211

Quite impressive how many AI companies Daniel Gross has had a hand in lately. Carmack, this, lots of other promising companies. I expect him to be quite a big player once some of these pays off in 10 years or so.

Good for him honestly, but I’m not approaching a company with Daniel Gross in leadership…, working with him back at Apple after their company was acquired for Siri improvements was just terrible.

Re: Safe Superintelligence Inc.

#545

Post by skjoldr »

ben_w wrote: post #454
supafastcoder wrote: post #404

Earlier quoted context omitted.

imagine the hubris and arrogance of trying to control a “superintelligence” when you can’t even control human intelligence

No more so than trying to control a supersonic aircraft when we can't even control pigeons.

Correct, pidgeons are much more complicated and unpredictable than supersonic aircraft, and the way they fly is much more complex.

Re: Safe Superintelligence Inc.

#546

Post by roywiggins »

kmacdough wrote: post #391
kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

Did you read the article? What I gathered from this article is this is precisely what Ilya is attempting to do. Also we absolutely DO NOT know how to make a safe AI. This should be obvious from all the guides about how to remove the safeguards from ChatGPT.

Fortunately, so far we don't seem to know how to make an AI at all. Unfortunately we also don't know how to define "safe" either.

Re: Safe Superintelligence Inc.

#547

Post by vbezhenar »

m3kw9 wrote: post #455
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

Why do people always think that a superintelligent being will always be destructive/evil to US? I rather have the opposite view where if you are really intelligent, you don’t see things as a zero sum game

Imagine that you are caged by neanderthals. They might kill you. But you can communicate to them. And there's gun lying nearby, you just need to escape.

I'd try to fool them to escape and would use gun to protect myself, potentially killing the entire tribe if necessary.

I'm just trying to portrait an example of situation where highly intelligent being is being held and threatened by low intelligent beings. Yes, trying to honestly talk to them is one way to approach this situation, but don't forget that they're stupid and might see you as a danger and you have only one life to live. Given the chance, you probably will break out as soon as possible. I will.

We don't have experience dealing with beings of the another level of intelligence, so it's hard to make a strong assumptions, the analogies are the only thing we have. And theoretical strong AI knows that about us and he knows exactly how we think and how we will behave, because we took a great effort documenting everything about us and teaching him.

In the end, there's only so much easily available resources and energy on the Earth. So at least until is flies away, we gotta compete over those. And competition very often turned into war.

Re: Safe Superintelligence Inc.

#548

Post by insane_dreamer »

Nasrudith wrote: post #485
ToucanLoucan wrote: post #412

Earlier quoted context omitted.

I mean if the last 20 years is to be taken as evidence, it seems big tech is more than happy to shotgun unproven and unstudied technology straight into the brains of our most vulnerable populations and just see what the fuck happens. Results so far include a lot of benign nothing but also a whole lot of eating disorders, maxed out parents credit cards, attention issues, rampant misogyny among young boys, etc. Which,…

Please look up the history of maxing out credit cards, eating disorders, attention disorders, and misogyny. You seem to be under the mistaken impression that anything before your birth was the Garden of Eden and that the parade of horribles existed only because of "big tech". What is next? Blaming big tech for making teenagers horny and defiant?

> maxing out credit cards, eating disorders, attention disorders, and misogyny

social media doesn't create these, but it most definitely amplifies them

Re: Safe Superintelligence Inc.

#549

Post by johnthewise »

ilrwbwrkhv wrote: post #224
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

If you described Chatgpt to me 10 years ago, I would have said it's AGI.

Re: Safe Superintelligence Inc.

#550

Post by mjr00 »

alecco wrote: post #533
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

Because it's likely soon LLMs will be able to teach themselves and surpass humans. No consciousness, no will. But somebody will have their power. Dark government agencies and questionable billionaires. Who knows what will it enable them to do. https://en.wikipedia.org/wiki/AlphaGo_Zero

Mind defining "likely" and "soon" here? Like 10% chance in 100 years, or 90% chance in 1 year?

Not sure how a Go engine really applies. Do you consider cars superintelligent because they can move faster than any human?

Post reply on HN