Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: Safe Superintelligence Inc.

#411

Post by kmacdough »

eigenvalue wrote: post #15

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

Generally, the mindset that makes the best engineers is an obsession with solving hard problems. Anecdotally, there's not a lot of overlap between the best engineers I know and the best paid engineers I know. The best engineers I know are too obsessed with solving problems to be sidetracked the salary game. The best paid engineers I know are great engineers, but the spend a large amount of time playing the salary game, bouncing between companies and are always doing the work that looks best on a resume, not the best work they know how to do.

Re: Safe Superintelligence Inc.

#412

Post by ToucanLoucan »

crowcroft wrote: post #383

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]). When negative externalities exist, government should create regulation that appropriately accounts for that cost. I understand there's a bit of a paradigm shift and new attack vectors wi…

I mean if the last 20 years is to be taken as evidence, it seems big tech is more than happy to shotgun unproven and unstudied technology straight into the brains of our most vulnerable populations and just see what the fuck happens. Results so far include a lot of benign nothing but also a whole lot of eating disorders, maxed out parents credit cards, attention issues, rampant misogyny among young boys, etc. Which, granted, the readiness to fuck with populations at scale and do immeasurable harm doesn't really make tech unique as an industry, just more of the same really.

But you know, we'll feed people into any kind of meat grinder we can build as long as the line goes up.

Re: Safe Superintelligence Inc.

#413

Post by kjkjadksj »

kmacdough wrote: post #397
worldsayshi wrote: post #364

Earlier quoted context omitted.

Yeah this feels close to the issue. Seems more likely that a harmful super intelligence emerges from an organisation that wants it to behave in that way than it inventing and hiding motivations until it has escaped.

I think a harmful AI simply emerges from asking an AI to optimize for some set of seemingly reasonable business goals, only to find it does great harm in the process. Most companies would then enable such behavior by hiding the damage from the press to protect investors rather than temporarily suspending business and admitting the issue.

Not only will they hide it, they will own it when exposed, and lobby to ensure it remains legal to exploit for profit. See oil industry.

Re: Safe Superintelligence Inc.

#414

Post by pillefitz »

modeless wrote: post #8

This makes sense. Ilya can probably raise practically unlimited money on his name alone at this point. I'm not sure I agree with the "no product until we succeed" direction. I think real world feedback from deployed products is going to be important in developing superintelligence. I doubt that it will drop out of the blue from an ivory tower. But I could be wrong. I definitely agree that superintelligence is within…

Who would pay for safety, though?

Re: Safe Superintelligence Inc.

#415

Post by YetAnotherNick »

liamconnell wrote: post #249
gnicholas wrote: post #139

Earlier quoted context omitted.

This would work for very small companies...but I'm not sure how one can avoid product cycles forever, even without scrum masters and the like. More to the point, how can you make a good product without something approximating product cycles?

Jane street did it for a long time. They are quite large now and only recently started bringing in program managers and the like.

The point is not exactly product cycles, but some way to track progress. Jane street also tracks progress and for many people it's the direct profit someone made for the firm. For some it is improving engineering culture so that other people can make better profits.

The problem with safety is that no one knows how to track it, or what they even mean by it. Even if you ignore tracking, wouldn't one unsafe AGI by one company in the world nullifies all their effort? Or safe AI would somehow need to take over the world, which is super unsafe in itself.

Re: Safe Superintelligence Inc.

#416

Post by amirhirsch »

Starlevel004 wrote: post #198
dougb5 wrote: post #125

> Building safe superintelligence (SSI) is the most important technical problem of our time. Call me a cranky old man but the superlatives in these sorts of announcements really annoy me. I want to ask: Have you surveyed every problem in the world? Are you aware of how much suffering there is outside of your office and how unresponsive it has been so far to improvements in artificial intelligence? Are you really sayi…

This all makes more sense when you realise it's Calvinism for programmers.

Calvinism for Transcendentalist techno-utopians -- an Asimovian Reformation of Singulatarianism

Re: Safe Superintelligence Inc.

#417

Post by [deleted] »

lannisterstark wrote: post #338
kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

>Short of solving this issue Solving human nature is indeed, hard.

[deleted]

Re: Safe Superintelligence Inc.

#419

Post by whimsicalism »

ToucanLoucan wrote: post #412
crowcroft wrote: post #383

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]). When negative externalities exist, government should create regulation that appropriately accounts for that cost. I understand there's a bit of a paradigm shift and new attack vectors wi…

I mean if the last 20 years is to be taken as evidence, it seems big tech is more than happy to shotgun unproven and unstudied technology straight into the brains of our most vulnerable populations and just see what the fuck happens. Results so far include a lot of benign nothing but also a whole lot of eating disorders, maxed out parents credit cards, attention issues, rampant misogyny among young boys, etc. Which,…

i am very skeptical of narratives saying that young boys or men are more misogynistic than in the past. we have a cognitive bias towards thinking the past is better than it was, but specifically on gender issues i just do not buy a regression

Re: Safe Superintelligence Inc.

#420

Post by whimsicalism »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

i don’t fully agree, but i do agree that this is the better narrative for selling people on the dangers of AI.

don’t talk about escape, talk about harmful actors - even if in reality it is both to be worried about

Post reply on HN