Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: Safe Superintelligence Inc.

#391

Post by kmacdough »

kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

Did you read the article? What I gathered from this article is this is precisely what Ilya is attempting to do.

Also we absolutely DO NOT know how to make a safe AI. This should be obvious from all the guides about how to remove the safeguards from ChatGPT.

Re: Safe Superintelligence Inc.

#393

Post by whimsicalism »

torginus wrote: post #382
PheonixPharts wrote: post #89

Earlier quoted context omitted.

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

Half the advancements around Stable Diffusion (Controlnet etc.) came from internet randoms wanting better anime waifus

advancements around parameter efficient fine tuning came from internet randoms because big cos don’t care about PEFT

Re: Safe Superintelligence Inc.

#394

Post by kevindamm »

aristofun wrote: post #388

What a waste of an intelligence. Pursuing artificial goal to solve a non existent problem to profit off meaningless hype around it. World would have been better off if he made a decent alternative to k8s or invested his skills into curing cancer or at least protecting world from totalitarian governments and dangerous ideologies (if he wants to belong to vague generic cause). You know, real problems, like the ones peo…

but would that stave off an impending recession?

Re: Safe Superintelligence Inc.

#395

Post by almost_usual »

almostgotcaught wrote: post #206
kccqzy wrote: post #123

Earlier quoted context omitted.

Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.

> their TC is now $900k. Everyone knows that openai TC is heavily weighted by ~~RSUs~~ options that themselves are heavily weighted by hopes and dreams.

You mean PPUs or smoke and mirrors compensation. RSUs are actually worth something.

Re: Safe Superintelligence Inc.

#396

Post by noway421 »

cynusx wrote: post #307

One element I find interesting is that people without an amygdala function are essentially completely indecisive. A person that just operates on the pure cognitive layer has no real direction in which he wants to drive himself. I suspect that AGI would be similar, extremely capable but essentially a solitary philosopher type that would be reactionary to requests it has to deal with. The equivalent of an amygdala for…

True, an auto-regressive LLM can't 'want' or 'like' anything.

The key to a safe AGI is to add a human-loving emotion to it.

We already RHLF models to steer them, but just like with System 2 thinking, this needs to be a dedicated module rather then part of the same next-token forward pass.

Re: Safe Superintelligence Inc.

#397

Post by kmacdough »

worldsayshi wrote: post #364
kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

Yeah this feels close to the issue. Seems more likely that a harmful super intelligence emerges from an organisation that wants it to behave in that way than it inventing and hiding motivations until it has escaped.

I think a harmful AI simply emerges from asking an AI to optimize for some set of seemingly reasonable business goals, only to find it does great harm in the process. Most companies would then enable such behavior by hiding the damage from the press to protect investors rather than temporarily suspending business and admitting the issue.

Re: Safe Superintelligence Inc.

#398

Post by whimsicalism »

gavin_gee wrote: post #232
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

This. Every nation-state will be in the game. Private enterprise will be in the game. Bitcoin-funded individuals will be in the game. Criminal enterprises will be in the game. How does one company building a safe version stop that? If I have access to hardware and data how does a safety layer get enforced? Regulations are for organizations that care about public perception, the law, and stock prices. Criminals and na…

I think the honest view (and you can scoff at it) is that winning the SI race basically wins you the enforcement race for free

Re: Safe Superintelligence Inc.

#400

Post by behnamoh »

kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

> Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures.

This tells me enough about why sama was fired, and why Ilya left.

Post reply on HN