Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

181–190 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#181

Post by MaysonL »

gbhn wrote: post #145
singlow wrote: post #131

Earlier quoted context omitted.

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button. In Nuremberg we developed a way to think about this: people outside the central circles of power have a tremendous amount of pressures they are considering. It's definitely the case that some are sadistic and horrible, but more are just following orders and trying to ge…

> True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button.

Actually, the unwillingness of the Red Army to do any more invasions was arguably the precipitating factor in the fall of the Soviet empire. In 1988, Gorbachev gave a speech to the Warsaw pact meeting where he told them that the Brezhnev doctrine was no more. No socialist government would be able any longer to count on Russian aid in putting down popular uprisings. A year later, the empire crumbled. Of course, it wasn't the soldiers per se who refused, but the generals who were afraid of the potential mutinies (and the occasional actual ones).

Re: Superintelligence: The Idea That Eats Smart People

#182

Post by edblarney »

modeless wrote: post #74
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

" They will give their human masters practically unlimited power over everyone else."

Anyone power that has significant leverage over another power already has that ability.

A bunch of 'super smart evil AI robots' will not be able to physical deter/control 500 million Europeans - but - a small Army of them would be enough to control the powers that be, and from there on in it trickles down.

Much the same way the Soviets controlled Poland et. al. with only small installations. The 'legitimate threat of violent domination' is all that is needed.

So - many countries already have the power to do those things to many, many others via conventional weapons and highly trained soldiers. That risk is already there. Think about it: a decent soldier today is already pretty much a 'better weapon' than AI will be for a very, very long time. And it' not that hard to make decent soldiers.

The risk for 'evil AI robots' is that a non-state, inauthentic actor - like a terrorist group, militia etc. gets control of enough of them to do project power.

The other risk I think, is that given the lack of bloodshed, states may employ them without fear of political repercussions at home. We see this with drones. If Obama had to do a 'seal team 6' for every drone strike, many, many of those guys would have died, and people coming home on body bags wears on the population. Eventually the war-fever fades and they want out.

Re: Superintelligence: The Idea That Eats Smart People

#183

Post by [deleted] »

modeless wrote: post #74
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

[deleted]

Re: Superintelligence: The Idea That Eats Smart People

#184

Post by seiferteric »

Almaviva wrote: post #171
seiferteric wrote: post #65

Earlier quoted context omitted.

One thing I rarely see discussed is the possibility that the brain is just too complex to practically reproduce. That is to say, that it is technically possible, but not practical. Evolution had billions of years, working in a massively parallel way to work this out after all. It's possible that the brain is a huge tangled mess of rules and special cases that we will never be able to fully understand and reproduce. A…

> Evolution had billions of years, working in a massively parallel way to work this out after all. And it also got to work on it with a hundred billion stars with earth-like planets, in each of a hundred billion galaxies, and (maybe) in a basically infinite space of universes. All in parallel with infinite time to play with. Is it so far-fetched to postulate that humans will hit our limits before getting this far?

Great point, the anthropic principle dictates we only see the successful outcome. Evolution is working across the entire universe (or even multiverse) so we really don't know how long it could take for human (or greater) level intelligence to emerge.

Re: Superintelligence: The Idea That Eats Smart People

#185

Post by state_less »

cloudhead wrote: post #166
AnimalMuppet wrote: post #90

Earlier quoted context omitted.

> SR and GR explicitly allow time-travel into the future. I presume you're talking about time dilation here. That's... a little bit true, but not really? At a minimum, it's sure a strange way of looking at it.

It allows wormholes to exist, which can connect arbitrary points in spacetime, hence time travel.

I time travelled here. What's the big deal? :)

State machines can be checkpoint at any state, load up whatever time you like, is there time per se?

Re: Superintelligence: The Idea That Eats Smart People

#186

Post by [deleted] »

pmyjavec wrote: post #77
apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

[deleted]

Re: Superintelligence: The Idea That Eats Smart People

#187

Post by openasocket »

myrmidon wrote: post #162
openasocket wrote: post #148

Earlier quoted context omitted.

I'll push back against the idea of smart factories leading to "a vast chasm between a new, tiny Hyperclass and the destitute masses." I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. Income inequality obviously benefits the rich (in that they by definition have more money), but only up to a point. We won't devolve…

> I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. I think that's beside the point. Why would the "aristocracy" owning the factories care about money, when they have all the goods (or can trade for them with other aristos)? It's not like they need money to pay other people (the destitute masses are useless to them)…

> Why would the "aristocracy" owning the factories care about money, when they have all the goods (or can trade for them with other aristos)?

In this scenario, they only need enough factories to meet the needs of themselves and the other aristocrats. Which means there will be massive demand for goods for the common people, which means there will be jobs for people. The only way your scenario works is if the aristocracy produces enough goods for the entire world cheaper than anyone else can, but also making them too expensive for anyone to afford. But that would mean the factories are producing enough goods for everyone despite the fact to very few can actually buy them. Piles of good wasting away outside the factory. The only reason to do this is if the aristocracy was malicious: trying to hurt the rest of humanity despite the fact that there is no real benefit.

Re: Superintelligence: The Idea That Eats Smart People

#188

Post by Analemma_ »

_cbsz wrote: post #170
Analemma_ wrote: post #39

Earlier quoted context omitted.

> The problem with argument ad hominem isn't that it never works. It often leads to the correct conclusion ... You immediately self-contradicted here. If a decision heuristic often leads to the correct conclusion, then it's a good heuristic (if you are under time constraints, which we are). Of course, given unlimited time to think about it, we would never use ad hominem reasoning and consider each and every argument…

> If a decision heuristic often leads to the correct conclusion, then it's a good heuristic (if you are under time constraints, which we are). So, is it a good heuristic to conclude that since crime is related to poverty and minorities tend to be poor, minorities qua minorities ought to be shunned?

No. The base rate of having a crime committed against you is extremely low, and the posterior probability of having a crime committed against you by a poor minority- even if higher- is still extremely low. My point refers to the absolute value of the posterior probability of one's heuristic being correct, not the probability gain resulting from some piece of evidence (like being a poor minority).

Re: Superintelligence: The Idea That Eats Smart People

#189

Post by kazagistar »

They always miss a critical and subtle assumption: that intelligence scales equal to or faster then the computational complexity of improving that intelligence.

This is the one assumption I most skeptical of. In my experience, each time you make a system more clever, you also make it MUCH more complex. Maybe there is not hard limit on intelligence, but maybe each generation of improved intelligence actually takes longer to find the next generation, due to the rapidly ramping difficulty of the problem.

I think people see the exponential-looking growth of technology over human history, and just kinda interpolate or something.

Re: Superintelligence: The Idea That Eats Smart People

#190

Post by idlewords »

MaysonL wrote: post #181
gbhn wrote: post #145

Earlier quoted context omitted.

True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button. In Nuremberg we developed a way to think about this: people outside the central circles of power have a tremendous amount of pressures they are considering. It's definitely the case that some are sadistic and horrible, but more are just following orders and trying to ge…

> True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button. Actually, the unwillingness of the Red Army to do any more invasions was arguably the precipitating factor in the fall of the Soviet empire. In 1988, Gorbachev gave a speech to the Warsaw pact meeting where he told them that the Brezhnev doctrine was no more. No socia…

It was Gorbachev who refused, not the generals.
Post reply on HN