Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

201–210 of 1001 posts

Re: Safe Superintelligence Inc.

#201

Post by novia »

RcouF1uZ4gsC wrote: post #12

The site is a good example of Poe's Law. If I didn't know that it was real, I would have thought it was parody. > We are assembling a lean, cracked team of the world’s best engineers and researchers dedicated to focusing on SSI and nothing else. Do you have to have a broken bone to join? Apparently, grammatical nuances are not an area of focus for safety, unless they think that a broken team ("cracked") is an asset i…

To me this is a good indication that the announcement was written by a human and not an LLM

Re: Safe Superintelligence Inc.

#202

Post by cynusx »

frenchie4111 wrote: post #14

I am not on the bleeding edge of this stuff. I wonder though: How could a safe super intelligence out compete an unrestricted one? Assuming another company exists (maybe OpenAI) that is tackling the same goal without spending the cycles on safety, what chance do they have to compete?

Not on its own but in numbers it could.

Similar to how law-abiding citizens turn on law-breaking citizens today or more old-fashioned, how religious societies turn on heretics.

I do think the notion that humanity will be able to manage superintelligence just through engineering and conditioning alone is naive.

If anything there will be a rogue (or incompetent) human who launches an unconditioned superintelligence into the world in no time and it only has to happen once.

It's basically Pandora's box.

Re: Safe Superintelligence Inc.

#203

Post by GeorgeTirebiter »

dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

C'mon. This one-pager is a recruiting document. One wants 'true believers' (intrinsically motivated) employees to execute the mission. Give Ilya some slack here.

Re: Safe Superintelligence Inc.

#205

Post by cynusx »

slashdave wrote: post #127
frenchie4111 wrote: post #14

I am not on the bleeding edge of this stuff. I wonder though: How could a safe super intelligence out compete an unrestricted one? Assuming another company exists (maybe OpenAI) that is tackling the same goal without spending the cycles on safety, what chance do they have to compete?

Since no one knows how to build an AGI, hard to say. But you might imagine that more restricted goals could end up being easier to accomplish. A "safe" AGI is more focused on doing something useful than figuring out how to take over the world and murder all the humans.

Hinton's point does make sense though.

Even if you focus an AGI on producing more cars for example, it will quickly realize that if it has more power and resources it can make more cars.

Re: Safe Superintelligence Inc.

#206

Post by almostgotcaught »

kccqzy wrote: post #123
PheonixPharts wrote: post #89

Earlier quoted context omitted.

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.

> their TC is now $900k.

Everyone knows that openai TC is heavily weighted by ~~RSUs~~ options that themselves are heavily weighted by hopes and dreams.

Re: Safe Superintelligence Inc.

#207

Post by auggierose »

PheonixPharts wrote: post #89
eigenvalue wrote: post #15

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

TC optimization being tail call optimization?

Re: Safe Superintelligence Inc.

#208

Post by enragedcacti »

mhardcastle wrote: post #103
weego wrote: post #75

Earlier quoted context omitted.

Honestly, what does it matter. We're many lifetimes away from anything. These people are trying to define concepts that don't apply to us or what we're currently capable of. AI safety / AGI anything is just a form of tech philosophy at this point and this is all academic grift just with mainstream attention and backing.

This goes massively against the consensus of experts in this field. The modal AI researcher believes that "high-level machine intelligence", roughly AGI, will be achieved by 2047, per the survey below. Given the rapid pace of development in this field, it's likely that timelines would be shorter if this were asked today. https://www.vox.com/future-perfect/2024/1/10/24032987/ai-imp...

I don't understand how you got 2047. For the 2022 survey:

    - "How many years until you expect: - a 90% probability of HLMI existing?" 
    mode: 100 years
    median: 64 years

    - "How likely is it that HLMI exists: - in 40 years?"
    mode: 50%
    median: 45%
And from the summary of results: "The aggregate forecast time to a 50% chance of HLMI was 37 years, i.e. 2059"

Re: Safe Superintelligence Inc.

#209

Post by rubyfan »

wffurr wrote: post #186
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

I think the idea is that a safe super intelligence would help solve those problems. I am skeptical because the vast majority are social coordination problems, and I don’t see how a machine intelligence no matter how smart can help with that.

So instead of a super intelligence either killing us all or saving us from ourselves, we’ll just have one that can be controlled to extract more wealth from us.

Re: Safe Superintelligence Inc.

#210

Post by sfink »

slashdave wrote: post #129
visarga wrote: post #20

Earlier quoted context omitted.

I have a strong intuition that chat logs are actually the most useful kind of data. They contain many LLM outputs followed by implicit or explicit feedback, from humans, from the real world, and from code execution. Scaling this feedback to 180M users and 1 trillion interactive tokens per month like OpenAI is a big deal.

Except LLMs are a distraction from AGI

That doesn't necessarily imply that chat logs are not valuable for creating AGI.

You can think of LLMs as devices to trigger humans to process input with their meat brains and produce machine-readable output. The fact that the input was LLM-generated isn't necessarily a problem; clearly it is effective for the purpose of prodding humans to respond. You're training on the human outputs, not the LLM inputs. (Well, more likely on the edge from LLM input to human output, but close enough.)

Post reply on HN