Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

581–590 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#581

Post by TheIronYuppie »

apsec112 wrote: post #88
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

To my knowledge, there's been no country that said "no rich people." Yes marginal rates have been very high in the past - sometimes it had no effect (UK example - though it only went up to maximum 50% marginal tax rate), sometimes it coincided with large times of expansion (US had marginal tax rates as high as 94% and hovered around 90% between 1944 and 1964).

Further, there's never been proof it "kills your economy." I've never met a phenomenally wealthy person who said "well, if tax rates go up to x%, that's when I stop working." These folks LIKE working, money is great, but it's not their driver. And, even if they DID stop working, would it be the worst thing in the world? Honestly - do we really think there's only one Gates, Zuckerberg, Ellison, Page, etc?

Re: Superintelligence: The Idea That Eats Smart People

#582

Post by TheIronYuppie »

erikbye wrote: post #417
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

They don't give back? Microsoft employs 100k+ people. That's giving back. These people all pay taxes and give back to society because Microsoft gave them a job. Because Gates happened. And in the case of Gates, let's not forget Bill & Melinda Gates Foundation. What the hell have you done for society? This story is very similar for most billionaires. They create A LOT of jobs and careers.

I'm a former employee of MSFT, and a great admirer of the Gates' dedication. But let's be clear - the fact that they employee 100k people isn't giving back in the least. They have a business, the business needs functions done, they're getting hours worked for money paid. That's not "giving back to society" in any way shape or form.

His billions of dollars donated to mosquito nets in Africa (among many other things) where he gets nothing back ... that's giving.

Re: Superintelligence: The Idea That Eats Smart People

#583

Post by adriand »

jameswilsterman wrote: post #580
adriand wrote: post #563

Earlier quoted context omitted.

That's the opposite of how history has played out with respect to violence, however. It is far easier (and less risky) to kill vast numbers of people now than it was hundreds and thousands of years ago, and yet the risk of any one person being killed by violence is far lower than at any previous point in history.

How are you calculating the 'risk of any one person being killed by violence' today? To make that claim you need to consider tail risk and black swan events that are possible but have little precedent. What weight are you giving the likelihood of a person dying by nuclear weapon?

Well, for example, "about 15% of people in prestate eras died violently, compared to about 3% of the citizens of the earliest states".

http://www.wsj.com/articles/SB100014240531119041067045765832...

As far as predicting the future goes, I can't.

Re: Superintelligence: The Idea That Eats Smart People

#584

Post by Nevermark »

MaysonL wrote: post #390
idlewords wrote: post #190

Earlier quoted context omitted.

It was Gorbachev who refused, not the generals.

He couldn't have refused without the support of the generals, who had just withdrawn from Afghanistan.

Very true.

And economic problems were a big incentive for everyone in charge to step back from wars. For country's with a strong economy there would be little incentive to stop, especially if AI makes war cheaper.

Re: Superintelligence: The Idea That Eats Smart People

#585

Post by Nevermark »

zamalek wrote: post #400
tvural wrote: post #326

Earlier quoted context omitted.

People are worried about AI risk because ensuring that the strong AI you build to do X will do X without doing something catastrophic to humanity instead is a very hard problem, and people who have not thought much about this problem tend to vastly underestimate how hard it is. Whatever goals the AI has, it will certainly be better at achieving them if it can stay alive. And it will be more likely to stay alive if th…

> to do X will do X without doing something catastrophic to humanity instead is a very hard problem This scenario I agree with. For instance: the AI decides that it doesn't want to live on this planet and consumes our star for energy, or exploits our natural resources leaving us with none. The whole AI war scenario is highly unlikely. As per the article, the opponents of AI are all regarded as prime examples of human…

>Surely something more intelligent than humans would be even less inclined to wage war.

The default mode for a machine would be to not care if people died, just as we don't care about most lower life forms.

> Furthermore, every argument against AI posits that humans are far more important than they really are. How much time of day do you spend thinking about bacteria in the Marianas Trench?

Exactly.

Which is why worrying about ourselves in a world with superintelligence is not wasted effort.

The extreme difference in productive abilities of superintelligence, vs a human population who's labor and intelligence has been devalued into obsolescence, suggests there will be serious unrest.

Serious unrest in a situation where a few have all the options tends to lead to extermination, as is evident every time an ant colony attempts to raid a home for food crumbs.

The AI's might not care whether we live or not, but they won't put up with us causing them harm or blocking their access to resources, even if we are doing it not to hurt them but only to survive.

Re: Superintelligence: The Idea That Eats Smart People

#586

Post by Nevermark »

wyager wrote: post #352
AaronFriel wrote: post #346

Earlier quoted context omitted.

I think the current state of deep neural networking designs and research funding pouring into generalizing the simple neural nets we're working with now suggests that we are, in fact, pulling them out of a hat. Right now we're just discarding all the ones that are defective, at a stupendously high rate as we train neural nets. I can't speak to what method would generate the first strong AI, but I suspect the overall…

That's not how neural network training works. It's not magic or guesswork; it's essentially glorified curve fitting (over a much more complicated space than the polynomials). It's also not random in any respect. The space of all neural networks plausibly generated from a training set is very, very small compared to the space of all networks of that size.

The initial conditions of neural networks are almost always chosen randomly or pseudo-randomly. Data sets are sometimes, although not always, presented with random sample order or selection.

Either way, the randomness in initial conditions means the solution found is one of many different solutions that could have been found, and depending on the problem, different initial conditions can result in very different solutions even on the same data.

Re: Superintelligence: The Idea That Eats Smart People

#587

Post by Nevermark »

copenja wrote: post #544
breuleux wrote: post #224

Earlier quoted context omitted.

> In this scenario, they only need enough factories to meet the needs of themselves and the other aristocrats. That could still end up taking all of the planet's resources, if enough aristocrats try to maximize their military capacity to protect themselves or play some kind of planet-wide game of Risk, or if they create new goods that require absolutely insane amounts of resources. I mean, ultimately, the issue is th…

What about a scenario in which the common people revolt and kill the aristocrats? It has happened before.

That just hits "replay" and provides convincing evidence to any new or surviving overlords that the capabilities of the masses must be suppressed more successfully next time.

Re: Superintelligence: The Idea That Eats Smart People

#588

Post by Nevermark »

mdpopescu wrote: post #365
ovibos wrote: post #163

Earlier quoted context omitted.

Money is just a stand-in for resources and labor, and if automation makes labor very very cheap, the rich will only need the natural resources the poor sit on, not anything from the poor themselves.

Which means these alleged poor will be in the same position everyone on the planet is right now , without having this magic automation.

Alleged poor? When machines are cheaper than humans for any given task (as a result of both AI and robotic improvements) human beings will be destitute except for any ownership of resources they already have and can defend.

That means the vast majority of people will have no means of income or resources, unless they appropriate the use land or resources they don't own in a shadow economy.

But the appropriation of resources is not likely to be perceived positively by those that own the resources, given that ownership is the only thing separating the rich from the destitute.

Re: Superintelligence: The Idea That Eats Smart People

#589

Post by Nevermark »

rlucas wrote: post #524
openasocket wrote: post #148

Earlier quoted context omitted.

I'll push back against the idea of smart factories leading to "a vast chasm between a new, tiny Hyperclass and the destitute masses." I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. Income inequality obviously benefits the rich (in that they by definition have more money), but only up to a point. We won't devolve…

It's a classic tragedy of the commons scenario. If you as a manufacturer move to a jobless production system, you gain net margin. If everybody moves to jobless production, the topline demand shrinks radically. Yet, for each individual mfg, the optimal choice is jobless production (aka, "loot the commons", aka "defect").

Except that at some point the very rich will have enough AI systems in place to provide for each other.

At that point, the economy will continue to grow, despite what will seem like an economic disaster to most of the human race.

This happens all the time to other species today, where humans suck up all the resources leaving incumbent creatures to die off.

Re: Superintelligence: The Idea That Eats Smart People

#590

Post by Nevermark »

notahacker wrote: post #448
FeepingCreature wrote: post #329

Earlier quoted context omitted.

That's not how risk works. If we want to grow to adulthood as a species and manage to colonize the cosmos, we need to pass _every_ skill challenge. If in 50 years there'll be a risk of unfriendly superintelligence and it'll have needed 40 years of run-up prep work to make safe, then it will do us absolutely zero good to claim that we instead concentrated on risk of military AI and hey, we got this far, right? Conside…

I think the article dealt pretty well with risk: you survive by focus finite resources on the X% chance of stopping Y% chances of the near elimination of humanity, not the A% chance of stopping a B% probability of an even worse than the near elimination of humanity event where X is large, Y is a small fraction and the product of A and B is barely distinguishable from zero, despite it getting more column inches most o…

Well yes, we have finite resources to deploy.

But most resources are not spent on risk mitigation. The question of what priority to give risk mitigation naturally (should) go up the more credible existential risks are identified.

Post reply on HN