Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

261–270 of 1001 posts

Re: Safe Superintelligence Inc.

#261

Post by WXLCKNO »

ilrwbwrkhv wrote: post #224
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

No one is saying there is. Just that we've reached some big milestones recently which could help get us there even if it's only by increased investment in AI as a whole, rather than the current models being part of a larger AGI.

Re: Safe Superintelligence Inc.

#262

Post by bugbuddy »

deegles wrote: post #225
TaupeRanger wrote: post #181

Earlier quoted context omitted.

Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.

'Fearing a rise of killer robots is like worrying about overpopulation on Mars.' - Andrew Ng

At the current Mars’ carrying capacity, one single person could be considered an overpopulation problem.

Re: Safe Superintelligence Inc.

#263

Post by kjkjadksj »

mdp2021 wrote: post #234
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

It says «technical» problem, and probably implies that other technical problems could dramatically benefit from such achievement.

If you want a real technical revolution, you teach the masses how to code their own tailored software, and not just use abstractions and software built by people who sell software to the average user. What a shame we failed at that and are even sliding back in a lot of ways with plummeting technical literacy in smartphone-raised generations.

Re: Safe Superintelligence Inc.

#264

Post by cwillu »

rafaelero wrote: post #106
frenchie4111 wrote: post #14

I am not on the bleeding edge of this stuff. I wonder though: How could a safe super intelligence out compete an unrestricted one? Assuming another company exists (maybe OpenAI) that is tackling the same goal without spending the cycles on safety, what chance do they have to compete?

It's probably not possible, which makes all these initiatives painfully naive.

It'd be naive if it wasn't literally a standard point that is addressed and acknowledged as being a major part of the problem.

There's a reason OpenAI's charter had this clause:

“We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.””

Re: Safe Superintelligence Inc.

#265

Post by dudeinhawaii »

localfirst wrote: post #152

This feels awfully similar to Emad and stability in the beginning when there was a lot of expectations and hype. Ultimately could not make a buck to cover the costs. I'd be curious to see what comes out of this however but we are not seeing the leaps and bounds with new llm iterations so wonder if there is something else in store

Interesting, I wish you had elaborated on Emad/etc. I'll see if Google yields anything. I think it's too soon to say "we're not seeing leaps and bounds with new LLMS". We are in-fact seeing fairly strong leaps, just this year, with respect to quality, speed, multi-modality, and robotics. Reportedly OpenAI started their training run for GPT-5 as well. I think we'd have to wait until this time next year before proclaiming "no progress".

Re: Safe Superintelligence Inc.

#266

Post by cwillu »

frenchie4111 wrote: post #14

I am not on the bleeding edge of this stuff. I wonder though: How could a safe super intelligence out compete an unrestricted one? Assuming another company exists (maybe OpenAI) that is tackling the same goal without spending the cycles on safety, what chance do they have to compete?

There's a reason OpenAI had this as part of its charter:

“We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.””

Re: Safe Superintelligence Inc.

#267

Post by TwoCent »

Filligree wrote: post #100
arbuge wrote: post #80

Earlier quoted context omitted.

The current intelligences of the world (us) have organized their civilization in a way that the conforming members of society are the norm and criminals the outcasts. Certainly not a perfect system, but something along those lines for the most part. I like to think AGIs will decide to do that too.

They well may, the problem is ensuring that humanity also survives.

The example from our environment suggests that the apex intelligences in the environment treat all other intelligent agents in only a few ways:

1. Pests to eliminate 2. Benign neglect 3. Workers 4. Pets 5. Food

That suggests that there are scenarios under which we survive. I'm not sure we'd like any of them, though "benign neglect" might be the best of a bad lot.

Re: Safe Superintelligence Inc.

#268

Post by fromMars »

015a wrote: post #120
PheonixPharts wrote: post #89

Earlier quoted context omitted.

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

Definitely true of even normal software engineering; my experience has been the opposite of expectations, that TC-creep has infected the industry to an irreparable degree and the most talented people I've ever worked around or with are in boring, medium-sized enterprises in the midwest US or australia, you'll probably never hear of them, and every big tech company would absolutely love to hire them but just can't fig…

I find this hard to believe having worked in multiple enterprises and in the FAANG world.

In my anecdotal experience, I can only think of one or two examples of someone from the enterprise world who I would consider outstanding.

The overall quality of engineers is much higher at the FAANG companies.

Re: Safe Superintelligence Inc.

#269

Post by azinman2 »

kjkjadksj wrote: post #258
azinman2 wrote: post #215

Earlier quoted context omitted.

Exactly. Or who gets the results of its outputs. How do we prioritize limited compute?

Even not just the compute but energy use at all. All the energy burned on training just to ask it the stupidest questions, by the numbers at least. All that energy that could have been used to power towns, schools, and hospitals the world over that lack sufficient power even in this modern age. Sure there's costs to bringing power to someplace, its not handwavy but a hard problem, but still, it is pretty perverse whe…

Unused electricity in one location is not fungible to be available elsewhere.

Re: Safe Superintelligence Inc.

#270

Post by tcgv »

Ten years from now will either be:

a) Remember all that fuss about AI destroying the world? Lol.

~ or ~

b) I'm so glad those people stepped in to save us from doom!

Which one do you think is more likely?

Post reply on HN