Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

341–350 of 1001 posts

Re: Safe Superintelligence Inc.

#341

Post by philwelch »

wffurr wrote: post #186
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

I think the idea is that a safe super intelligence would help solve those problems. I am skeptical because the vast majority are social coordination problems, and I don’t see how a machine intelligence no matter how smart can help with that.

Social coordination problems exist within a specific set of constraints, and that set of constraints can itself be altered. For instance, climate change is often treated as a social coordination problem, but if you could produce enough energy cheaply enough, you could solve the greenhouse gas problem unilaterally.

Re: Safe Superintelligence Inc.

#342

Post by whimsicalism »

dsign wrote: post #245

Our current obsession with super-intelligence reminds me the great oxidation event a few billion years ago. Super-photosynthesis was finally achieved, and then there was a great extinction. If you believe that super-intelligence is unavoidable and a serious risk to humanity, then the sensible thing to do is to prepare to leave the planet, ala Battlestart Galactica. That's going to be easier than getting the powers th…

If the human cooperation problem is unsolvable, I doubt creating a new human society with the same capabilities elsewhere would do much at all.

Re: Safe Superintelligence Inc.

#343

Post by [deleted] »

mike_hearn wrote: post #283
appplication wrote: post #243

Earlier quoted context omitted.

It’s amazing how someone so smart can be so naive. I do understand conceptually the idea that if we create intelligence greater than our own that we could struggle to control it. But does anyone have any meaningful thoughts on how this plays out? I hear our industry thought leaders clamoring over this but not a single actual concrete idea of what this means in practice. We have no idea what the fundamental architectu…

We're really not that far. I'd argue superintelligence has already been achieved, and it's perfectly and knowably safe. Consider, GPT-4o or Claude are: • Way faster thinkers, readers, writers and computer operators than humans are • Way better educated • Way better at drawing/painting ... and yet, appear to be perfectly safe because they lack agency. There's just no evidence at all that they're dangerous. Why isn't t…

[deleted]

Re: Safe Superintelligence Inc.

#344

Post by kjkjadksj »

cosmic_quanta wrote: post #305
tcgv wrote: post #270

Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?

Unless AI starts being 1 000 000x energy efficient, my money is on a). The amount of energy required for AI to be dangerous to its creators is so vast that I can't see how it can realistically happen.

That depends on how its used. See the terminator movies. One false positive is enough to end the world with even current AI tech if its merely mated to a nuclear arsenal (even a small one might see a global escalation). There have been false positives before, and the only reason why they didn't end in nuclear Armageddon was because the actual operators hesitated and defied standard protocol, which probably would have lead to the end the world as we know it.

Re: Safe Superintelligence Inc.

#346

Post by whimsicalism »

ofou wrote: post #130
wizzwizz4 wrote: post #97

Earlier quoted context omitted.

Backprop is neither commutative nor associative.

What do you mean? There's a bunch of proof-of-concepts such as Hydra, peer-nnet, Learnae, and so on.

just because it has a PoC or a wiki article doesn’t mean it actually works

Re: Safe Superintelligence Inc.

#347

Post by zackmorris »

dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

I believe that AGI is the last problem in computer science, so solving it solves all of the others. Then with AGI, we can solve the last remaining problems in physics (like unifying gravity with quantum mechanics), biology (administering gene therapy and curing death), etc.

But I do agree that innovations in tech are doing little or nothing to solve mass suffering. We had the tech to feed everyone in the world through farm automation by the 60s but chose not to. We had the tech in the 80s to do moonshots for AIDS, cancer, etc but chose not to. We had the tech in the 2000s to transition from fossil fuels to renewables but chose not to. Today we have the opportunity to promote world peace over continuous war but will choose not to.

It's to the point where I wonder how far innovations in tech and increases in economic productivity will get without helping people directly. My experience has been that the world chooses models like Dubai, Mexico City and San Francisco where skyscrapers tower over a surrounding homeless and indigent population. As long as we continue pursuing top-down leadership from governments and corporations, we'll see no change to the status quo, and even trends towards authoritarianism and fascism. It will take people at the bottom organizing to provide an alternate economic model before we have options like universal education/healthcare/opportunity and UBI from robot labor.

What gets me is that stuff like the ARC prize for AGI will "just work". As in, even if I had a modest stipend of a few thousand dollars per month to dabble in AI and come up with solutions the way I would for any other startup, certainly within 3 years, someone else would beat me to it. There simply isn't enough time now to beat the competition. Which is why I give AGI over 50% odds of arriving before 2030, where I used to think it was 2040 or 2050. The only thing that could stop it now is sabotage in the form of another global pandemic, economic depression or WWIII. Progress which threatens the power structures of the ultra wealthy is what drives the suffering that they allow to continue.

Re: Safe Superintelligence Inc.

#349

Post by spitfire »

brcmthrowaway wrote: post #218
nuz wrote: post #211

Quite impressive how many AI companies Daniel Gross has had a hand in lately. Carmack, this, lots of other promising companies. I expect him to be quite a big player once some of these pays off in 10 years or so.

What's Carmack?

John Carmack.

Re: Safe Superintelligence Inc.

#350

Post by HeatrayEnjoyer »

almostgotcaught wrote: post #206
kccqzy wrote: post #123

Earlier quoted context omitted.

Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.

> their TC is now $900k. Everyone knows that openai TC is heavily weighted by ~~RSUs~~ options that themselves are heavily weighted by hopes and dreams.

Everything OpenAI does is about weights.
Post reply on HN