Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

161–170 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#161
post #26

Earlier quoted context omitted.

Here's a response about why complexity theory may not constrain AI that much: https://www.gwern.net/Complexity%20vs%20AI It is something "AI alarmists" think about.

What's his reason? I got halfway through that and realized I didn't understand.

That even if you see diminishing returns to what you can accomplish with greater intelligence, a small advantage will still compound into a large one over the theoretically infinite lifespan of an AI.

Re: Superintelligence: The Idea That Eats Smart People

#162

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

I'll push back against the idea of smart factories leading to "a vast chasm between a new, tiny Hyperclass and the destitute masses." I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. Income inequality obviously benefits the rich (in that they by definition have more money), but only up to a point. We won't devolve…

> I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money.

I think that's beside the point. Why would the "aristocracy" owning the factories care about money, when they have all the goods (or can trade for them with other aristos)?

It's not like they need money to pay other people (the destitute masses are useless to them). With their only inherent "capital", --the ability to work-- made worthless by automation, the destitute masses have no recourse-- they get slowly extinguished, until only a tiny fraction of humanity is left.

Re: Superintelligence: The Idea That Eats Smart People

#163

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

I'll push back against the idea of smart factories leading to "a vast chasm between a new, tiny Hyperclass and the destitute masses." I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. Income inequality obviously benefits the rich (in that they by definition have more money), but only up to a point. We won't devolve…

Money is just a stand-in for resources and labor, and if automation makes labor very very cheap, the rich will only need the natural resources the poor sit on, not anything from the poor themselves.

Re: Superintelligence: The Idea That Eats Smart People

#164
post #155
post #150

Earlier quoted context omitted.

What percentage of people are Navy SEALs? If I were a tiger, I'd probably think this island sounded like an excellent place for a fun adventure holiday with my tiger friends, and I'd be right to do so. Most likely outcome: good food, some exercise, and we can take some monkey heads home for our tiger cubs to play with.

Fun, until the navy seal appears. It takes only one. The same thing with AI, it takes only one, no matter how many dumber ones we entertained ourselves with before.

[deleted]

Re: Superintelligence: The Idea That Eats Smart People

#165

Moore's Law (on a single CPU core) effectively ended around 2009. Our programming idioms for parallel processing are still so crude, hardly anyone can write maintainable parallelized code for anything but toy problems. Until we can restart Moore's Law again, this is all academic. http://www.globalnerdy.com/2007/09/07/multicore-musings/

But the law says nothing about cores or clock speeds etc. just transistors and their sizes. The regular reductions in feature size on dies are still happening, as I understand it.

Re: Superintelligence: The Idea That Eats Smart People

#166

Earlier quoted context omitted.

SR and GR explicitly allow time-travel into the future. Which isn't a fully general Time Machine, of course, but is a huge change from 19th-century physics. If SR had just been invented today, and someone who thought it was crazy and didn't know the math was writing a blog post about it, I 100% expect they'd call it "the time travel theory" or some such thing.

> SR and GR explicitly allow time-travel into the future. I presume you're talking about time dilation here. That's... a little bit true, but not really? At a minimum, it's sure a strange way of looking at it.

It allows wormholes to exist, which can connect arbitrary points in spacetime, hence time travel.

Re: Superintelligence: The Idea That Eats Smart People

#167

"The second premise is that the brain is an ordinary configuration of matter, albeit an extraordinarily complicated one. If we knew enough, and had the technology, we could exactly copy its structure and emulate its behavior with electronic components, just like we can simulate very basic neural anatomy today." We could have a computer program that perfectly simulates the brain, but has some nasty O(2^N) complexity a…

I don't buy the AI quasi-religious stuff. But your argument here is flawed. If protein folding can do the process in constant time, we may be able to find another process (but electronic rather than wet chem) that can also do it in constant time.

Being able to find constant time algorithms for algorithms that currently take exponential time is not at all assured.

Re: Superintelligence: The Idea That Eats Smart People

#168

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

I'll push back against the idea of smart factories leading to "a vast chasm between a new, tiny Hyperclass and the destitute masses." I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. Income inequality obviously benefits the rich (in that they by definition have more money), but only up to a point. We won't devolve…

What matters is the total size of the market and the total size of the labor pool. If the Hyperclass have more wealth than all of humanity had prior, they don't need to sell to the masses to make money. If the labor pool is mostly machines they own, they don't need to pay the masses, or even a functioning market among the masses to enable it. In the degenerate case, where a single individual controls all wealth, if they have self-running machines that can do everything necessary to make more self-running machines, that individual can continue to get richer (in material goods; money makes no sense in this case).

Re: Superintelligence: The Idea That Eats Smart People

#169
Glad to see a voice from the strong AI skeptic camp here. Reminds me of a book I read a long time ago called "Great Mambo Chicken and the Transhuman Condition." I used to drink the kool-aid myself until a friend of mine snapped me out of it by saying, "Dude, you're telling me you actually want Skynet??" I gave him my copy of the book.

Re: Superintelligence: The Idea That Eats Smart People

#170
post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

> The problem with argument ad hominem isn't that it never works. It often leads to the correct conclusion ... You immediately self-contradicted here. If a decision heuristic often leads to the correct conclusion, then it's a good heuristic (if you are under time constraints, which we are). Of course, given unlimited time to think about it, we would never use ad hominem reasoning and consider each and every argument…

> If a decision heuristic often leads to the correct conclusion, then it's a good heuristic (if you are under time constraints, which we are).

So, is it a good heuristic to conclude that since crime is related to poverty and minorities tend to be poor, minorities qua minorities ought to be shunned?

Post reply on HN