Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

741–750 of 1001 posts

Re: Safe Superintelligence Inc.

#741

Post by crowcroft »

akira2501 wrote: post #436
crowcroft wrote: post #383

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]). When negative externalities exist, government should create regulation that appropriately accounts for that cost. I understand there's a bit of a paradigm shift and new attack vectors wi…

> for society and young people (girls in particular [1]). I don't think the article with a single focused example bears that out at all. From the article: > "Even more troubling are the men who signed up for paid subscriptions after the girl launched a program for super-fans receive special photos and other content." > "Her mom conceded that those followers are “probably the scariest ones of all.”" I'm sorry.. but wh…

One example is all I need although I know there would be more to find if I had the time.

One is enough because the fact is, this just simply shouldn’t be possible. Not only does Meta allow it though, it produces a system that incentivises it.

Re: Safe Superintelligence Inc.

#742

Post by throwawaywright »

throwaway4good wrote: post #735
rqtwteye wrote: post #730

I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege…

It is a strange thing. We are not anywhere near “super intelligence” yet the priority is safety. If the Wright brothers had focused on safety, I am not sure they would have flown very far.

[flagged]

Re: Safe Superintelligence Inc.

#743

Post by Animats »

zucker42 wrote: post #504
Animats wrote: post #425

What does "safe" mean? 1. Will not produce chat results which are politically incorrect and result in publicity about "toxic" comments? 2. Will not return false factual information which is dangerously wrong, such as that bad recipe on YC yesterday likely to incubate botulism toxin? 3. Will not make decisions which harm individuals but benefit the company running the system? 4. Will not try to take over from humans?…

Ilya's talking about type 4.

As opposed to 3a, "Will not assist companies to take over".

Re: Safe Superintelligence Inc.

#744

Post by mr_toad »

AYBABTME wrote: post #725
hackerlight wrote: post #708

Earlier quoted context omitted.

US has to win the race because while it's true that it's no angel, it isn't an authoritarian dictatorship and there isn't an equivalence in how bad the world will end up for you and me if the authoritarian side wins the race. Monopoly power will get abused the most by the least democratic actors, which is China. We need multiple actors within the US to win to balance power. We don't need or want China to be one of th…

If anyone doubts this. Recent (<100y) leaders of China and Russia internally displaced and caused the death of large % of their population, for essentially fanciful ideological reasons.

You don’t need to be an autocracy to commit genocide.

https://en.wikipedia.org/wiki/Trail_of_Tears

Re: Safe Superintelligence Inc.

#745

Post by throwawaywright »

ilrwbwrkhv wrote: post #224
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

Imagine a system that can do DNS redirection, MITM, deliver keyloggers, forge authorizations and place holds on all your bank accounts, clone websites, clone voices, fake phone and video calls with people that you don’t see a lot. It can’t physically kill you yet but it can make you lose your mind which imo seems worse than a quick death

Re: Safe Superintelligence Inc.

#746

Post by [deleted] »

hackerlight wrote: post #389
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

China can not win this race and I hate that this comment is going to be controversial among the circle of people that need to understand this the most. It is damn frightening that an authoritarian country is so close to number one in the race to the most powerful technology humanity has invented, and I resent people who push for open source AI for this reason alone. I don't want to live in a world where the first sup…

[deleted]

Re: Safe Superintelligence Inc.

#747

Post by comfysocks »

rqtwteye wrote: post #730

I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege…

The Industrial Revolution devalued manual labor. Sure, new jobs were created, but on the whole this looked like a shift to knowledge work and away from manual labor.

Now AI is beginning to devalue knowledge work. Although the limits of current technology is obvious in many cases, AI is already doing a pretty good job at replacing illustrators and copy writers. It will only get better.

Who owns the value created by AI productivity? Ultimately it will be shareholders and VCs. It’s no surprise that the loudest voices in techno-optimism are VCs. In this new world they win.

Having said all this, I think Ilya’s concerns are more of the existential type.

Re: Safe Superintelligence Inc.

#748

Post by rattray »

tasoeur wrote: post #544
nuz wrote: post #211

Quite impressive how many AI companies Daniel Gross has had a hand in lately. Carmack, this, lots of other promising companies. I expect him to be quite a big player once some of these pays off in 10 years or so.

Good for him honestly, but I’m not approaching a company with Daniel Gross in leadership…, working with him back at Apple after their company was acquired for Siri improvements was just terrible.

Is there more you can share, or point to?

Sounds like he might become an important player soon and it'd be helpful for people to learn what they can.

Re: Safe Superintelligence Inc.

#750

Post by rattray »

notresidenter wrote: post #49
aridiculous wrote: post #36

Surprising to see Gross involved. He seems to be pretty baked into the YC world, which usually means "very commercially oriented".

His latest project ( https://pioneer.app/ ) recently (this year I think) got shutdown. I guess he's pivoting.

What do you mean "got shutdown"? What happened?
Post reply on HN