Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

811–820 of 1001 posts

Re: Safe Superintelligence Inc.

#811

Post by ben_w »

mr_toad wrote: post #771
ben_w wrote: post #443

Earlier quoted context omitted.

A human-level AI can do anything that a human can do (modulo did you put it into a robot body, but lots of different groups are already doing that with current LLMs). Therefore, please imagine the most amoral, power-hungry, successful sociopath you've ever heard of. Doesn't matter if you're thinking of a famous dictator, or a religious leader, or someone who never got in the news and you had the misfortune to meet in…

> power-hungry That has nothing to do with intelligence.

That's why it's a problem.

An AI can be anywhere on that axis, and we don't really know what we're doing in order to prevent it being as I have described.

Re: Safe Superintelligence Inc.

#812

Post by Blammar »

antihipocrat wrote: post #565
drowntoge wrote: post #408

Earlier quoted context omitted.

Agreed. The AI of our day (the transformer + huge amounts of questionably acquired data + significant cloud computing power) has the spotlight it has because it is readily commoditized and massively profitable, not because it is an amazing scientific breakthrough or a significant milestone toward AGI, superintelligence, the benevolent Skynet or whatever. The association with higher AI goals is merely a mixture of pur…

It's a massive attractor of investment funding. Is it proven to be massively profitable?

I read in Forbes about a construction company that used AI-related tech to manage the logistics and planning. They claimed that they were saving upwards of 20% of their costs because everything was managed more accurately. (Maybe they had little control boxes on their workers too; I don't know.)

The point I am trying to make is that the benefits of AI-related tech is likely to be quite pervasive and we should be looking at what corporations are actually doing. Sort of what this poem says:

For while the tired waves, vainly breaking / Seem here no painful inch to gain, / Far back through creeks and inlets making, / Comes silent, flooding in, the main.

Re: Safe Superintelligence Inc.

#813

Post by Dr_Birdbrain »

tarsinge wrote: post #434
waihtis wrote: post #259

Earlier quoted context omitted.

Safety is just enforcing political correctness in the AI outputs. Any actual examples of real world events we need to avoid are ridiculous scenarios like being eaten by nanobots (yes, this is an actual example by Yud)

What does political correctness means for the output of a self driving car system or a code completion tool? This is a concern only if you make a public chat service branded as an all knowing assistant. And you can have world threatening scenarii by directly plugging basic automations to nuclear warheads without human oversight.

How could a code completion tool be made safe?

One natural response seems to be “it should write bug-free code”. This is the domain of formal verification, and it is known to be undecidable in general. So in this formulation safe AI is mathematically impossible.

Should it instead refuse to complete code that can be used to harm humans? So, it should read the codebase to determine if this is a military application? Pretty sure mainstream discourse is not ruling out military applications.

Re: Safe Superintelligence Inc.

#814

Post by fauigerzigerk »

buckinkb wrote: post #776
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

We should be less concerned about super intelligence and more about the immediate threat of job loss. An AI doesn’t need to be Skynet to wreak massive havoc on society. Replacing 20% of jobs in a very short period of time could spark global unrest resulting in WW3

Replacing 20% of jobs in, say, 10 years wouldn't be that unusual [1]. It can mean growing prosperity. In fact, productivity growth is the only thing that increases wealth overall.

It is the lack of productivity growth that is causing a lot of extremism and conflict right now. Large groups of people feel that the only way for them to win is if others lose and vice versa. That's a recipe for disaster.

The key question is what happens to those who lose their jobs. Will they find other, perhaps even better, jobs? Will they get a piece of the growing pie even if they don't find other jobs and have to retire early?

It's these eternal political problems that we have to solve. It's nothing new. It has never been easy. But it's probably easier than managing decline and stagnation because at least we would have a growing pie to divvy up.

[1] https://www.britannica.com/money/productivity/Historical-tre...

Re: Safe Superintelligence Inc.

#815

Post by mdp2021 »

beardedwizard wrote: post #784
mdp2021 wrote: post #324

Earlier quoted context omitted.

> understand consciousness We do not call Intelligence something related to consciousness. Being able to reason well suffices.

That is something I hear over and over, particularly as a rebuttal to the argument that llm is just a stochastic parrot. Calling it "good enough" doesn't mean anything, it just allows the person saying it to disengage from the substance of the debate. It's either reasons or it doesn't, and today it categorically does not.

That some will remark that you do not need consciousness to achieve reasoning does not lose truth because a subset sees in LLMs something that appears to them as reasoning.

I do not really understand who you are accusing of a «good enough» stance: we have never defined "consciousness" as a goal (cats are already there and we do not seem to need further), we just want something that reasons. (And that reasons excellently well.)

The apparent fact that LLMs do not reason does is drily irrelevant to an implementation of AGI.

The original poster wrote that understanding consciousness would be required to «crack AGI» and no, we state, we want AGI as a superhuman reasoner and consciousness seems irrelevant.

Re: Safe Superintelligence Inc.

#816

Post by WatchDog »

sgregnt wrote: post #792
benreesman wrote: post #753

Earlier quoted context omitted.

It’s just clearly military R&D at this point. And it’s not even a little bit controversial that cutting edge military R&D is classified in general and to an extreme in wartime. The new thing is the lie that it’s a consumer offering. What’s new is giving the helm to shady failed social network founders with no accountability. These people aren’t retired generals with combat experience. They aren’t tenured professors a…

To me it is not clear at all, can you please elaborate why you make such a strong claim?

One data point:

https://openai.com/index/openai-appoints-retired-us-army-gen...

Re: Safe Superintelligence Inc.

#817

Post by Yoric »

Lockal wrote: post #659
Yoric wrote: post #600

Earlier quoted context omitted.

I actually feel that they can be very dangerous. Not because of the fabled AGI, but because 1. they're so good at showing the appearance of being right; 2. their results are actually quite unpredictable, not always in a funny way; 3. C-level executives actually believe that they work. Combine this with web APIs or effectors and this is a recipe for disaster.

3. Sorry, but how do you know what do they believe in?

My bad, I meant too many C-level executives believe that they actually work.

And the reason I believe that is that, as far as I understand, many companies are laying off employees (or at least freezing hiring) with the expectation that AI will do the work. I have no mean to quantify how many.

Re: Safe Superintelligence Inc.

#818

Post by mdp2021 »

dreamcompiler wrote: post #756
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

That "tweet" loses a veneer if you see that we value what has Worth as a collective treasure, and the more Value is produced the better - while that one engages in producing something of value is (hopefully but not necessarily) a good exercise in intelligent (literal sense) cultivation.

So, yes, if algorithms strict or loose could one day produce Art, and Thought, and Judgement, of Superior quality: very welcome.

Do not miss that the current world is increasingly complex to manage, and our lives, and Aids would be welcome. The situation is much more complex than that wish for leisure or even "sport" (literal sense).

Re: Safe Superintelligence Inc.

#819

Post by reducesuffering »

lucubratory wrote: post #690
bsaul wrote: post #687

Recently saw an interview of yann lecun that really dismissed the idea of using llm for building « true » intelligence. He even encouraged phds to NOT do their research on that tech but rather on what’s coming after. How should one reconcile this with what iliya is doing atm ? Is he planning on working using some other tech ?

The easiest way to reconcile it is that Yann LeCunn is consistently wrong in the predictions he makes about this field and he's wrong here too.

Yes, case in point: https://twitter.com/liron/status/1659618568282185728

Re: Safe Superintelligence Inc.

#820

Post by ericd »

hackerlight wrote: post #708
richrichie wrote: post #700

Earlier quoted context omitted.

US is no angel and it cannot be the only one which wins the race. We have hard evidence of how monopoly power gets abused in the case of the US e.g. as the sole nuclear power, it used nukes on civilians. We need every one to win this race to keep things on balance.

US has to win the race because while it's true that it's no angel, it isn't an authoritarian dictatorship and there isn't an equivalence in how bad the world will end up for you and me if the authoritarian side wins the race. Monopoly power will get abused the most by the least democratic actors, which is China. We need multiple actors within the US to win to balance power. We don't need or want China to be one of th…

Could you explain your point a bit more? You say you’re worried about them having a monopoly, but then say that’s why you don’t support open source models? Open models mean that no one has a monopoly, what am I not getting here?
Post reply on HN