Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

241–250 of 256 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#241

Post by red75prime »

Nasrudith wrote: post #229
Isamu wrote: post #185

>What I find particularly suspect is the idea that "intelligence" is like CPU speed, in that any sufficiently smart entity can emulate less intelligent beings (like its human creators) no matter how different their mental architecture. >With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be…

Really O(N!) should nip that in the bud in itself alone. There aren't enough particles in the universe for even 1000 factorial. The scaling isn't trivial at all as the problem space grows and the interactions with itself would start eating up speed further. While we may one day build computers able of out-competing us mentally in many fields but they would be many different machines and not just one uber-machine for…

OTOH, we have phones, which are telephone, TV, radio, notebook, phone book, calendar, calculator, computer, compass, GPS, camera, video recorder, sound recorder hybrids.

I'm not sure why do you think that algorithms for solving different problems will require different hardware.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#242

Post by red75prime »

red75prime wrote: post #68
krisoft wrote: post #31

Earlier quoted context omitted.

So what is the brain if not just matter?

If there's no god who prohibits connecting immaterial parts to anything but human brain, then it's a technical problem.

Sorry, misread your question as "So what if the brain is not just matter?"

Re: Superintelligence: The Idea That Eats Smart People (2016)

#243

Post by heavenlyblue »

gnode wrote: post #209
nine_k wrote: post #199

Earlier quoted context omitted.

I don't think re-packaging subprime debt was so far away from willful and reckless.

Reckless intent requires that the person knew there is was significant risk of the guilty act happening (e.g. driving with your eyes closed to show off has a likely and known risk of causing injury and death). I don't think we can just assume that market makers thought their actions were likely to upend the economy.

Yeah. But in this scenario reckless driving provides personal gains for the owners of the businesses.

What if you can invent new methods of "reckless driving" before they are put in the legal system and thus reap their benefits?

It's just like designer drugs.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#244

Post by gnode »

heavenlyblue wrote: post #243
gnode wrote: post #209

Earlier quoted context omitted.

Reckless intent requires that the person knew there is was significant risk of the guilty act happening (e.g. driving with your eyes closed to show off has a likely and known risk of causing injury and death). I don't think we can just assume that market makers thought their actions were likely to upend the economy.

Yeah. But in this scenario reckless driving provides personal gains for the owners of the businesses. What if you can invent new methods of "reckless driving" before they are put in the legal system and thus reap their benefits? It's just like designer drugs.

> It's just like designer drugs.

I think an important distinction is that drug prohibition often works in terms of specific chemicals by statute. E.g. drug X is now illegal for use and distribution.

Securities fraud law instead works in concepts and intentions and is fleshed out by case law. E.g. did the seller give the buyer sufficient information for the buyer not to have been defrauded?

A better analogy to securities using chemicals would be poisoning. The law doesn't need to catch up to specific chemicals; if it's evident that you wilfully, or recklessly added a poison to someone's food, it's illegal, and it doesn't matter how you did it. If it turns out that some ingredient is unknowingly toxic and the whole industry was adding it, then it's difficult to argue criminal intent. The SEC is more like the FDA, than it is the DEA.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#245

Post by Nasrudith »

red75prime wrote: post #241
Nasrudith wrote: post #229

Earlier quoted context omitted.

Really O(N!) should nip that in the bud in itself alone. There aren't enough particles in the universe for even 1000 factorial. The scaling isn't trivial at all as the problem space grows and the interactions with itself would start eating up speed further. While we may one day build computers able of out-competing us mentally in many fields but they would be many different machines and not just one uber-machine for…

OTOH, we have phones, which are telephone, TV, radio, notebook, phone book, calendar, calculator, computer, compass, GPS, camera, video recorder, sound recorder hybrids. I'm not sure why do you think that algorithms for solving different problems will require different hardware.

What I am saying is that it won't be some magical omnitool that can go from curing death to undetectably forging nanotech instadeath rockets.

Even if hardware is shares configurations will not for the same reason Steven Hawkings wasn't drafted as a general during any of the brief Falkland Islands wars even when they were considering if MAD called for nuking Argentina if they lost. There are very different specialties even if the hardware is the same. Having superhuman General and superhuman Surgeon skill level would be two seperate major projects to create and with lesser redundancies and in the worst case would call for two modules due to lack of shared synergies. You can't really combine processing and RAM hungry CAD programs with weather simulations and expect to see same memory footprint with the same performance.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#246

Post by ahel »

grkvlt wrote: post #183
crunchyfrog wrote: post #149

> If Einstein tried to get a cat in a carrier, and the cat didn't want to go, you know what would happen to Einstein. > > He would have to resort to a brute-force solution that has nothing to do with intelligence, and in that matchup the cat could do pretty well for itself. Wouldn't it be better to simply motivate the cat to get in the carrier with a can of tuna fish? This is a very simple form of modeling the cat's…

> Wouldn't it be better to simply motivate the cat to get in the carrier with a can of tuna fish? One does not simply motivate a cat with a can of tuna fish. I mean, have you even owned a cat? If the little bastard doesn't want to get in the cat carrier, no amount of delicious tuna will encourage it, and you're going to end up using brute force and getting mauled...

Reduce the cat in a state of prolonged hunger and it will jump on the tuna when you offer it.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#247

Post by lsc »

vidarh wrote: post #239
lsc wrote: post #225

Earlier quoted context omitted.

>I've seen humans deal with novel situations. The only assumption needed is that humans and AI are subject to the same laws of nature (that is, that there is no supernatural / outside element to human intelligence) right, but all our computers right now are based on lumping transistors together so you can do math. It's likely that our brains work on a different (but still physical) principle. It's possible our brains…

What computers are like right now is irrelevant to the issue being discussed. If/when we develop AGI we will necessarily have figured out what the computational model needed is. And the odds that we can not then improve on it seems extremely unlikely to me. > As we approach the limits of our abilities to make tiny transistors, this is looking less likely. Is it? Computational power is still growing like before. What…

>Is it? Computational power is still growing like before.

I don't think it is. transistors per watt and transistors per dollar are not going up at the same rate as they used to be, even if you have costless multi-threading.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#248

Post by lsc »

vidarh wrote: post #240
lsc wrote: post #224

Earlier quoted context omitted.

>Slower progress in microprocessors is a good point but specialized hardware like TPUs could overcome some limitations. Also, slower does not mean stop. When we talk about decades into the future, even a 20% average annual improvement amounts to 3 orders of magnitude in 40 years. My understanding is that there are physical limits, and we will asymtodically approach those physical limits. So while progress will never…

> But there are physical limits, and it seems that we are approaching them.) Unless you believe in something supernatural providing extra computational capacity, we know that the physical limits allows the manufacture of a sub 20W computational device in about 1.5kg of matter with the capacity of a human brain. It may be impossible to reach the same using silicon, but we know it is possible to get there somehow withi…

>Unless you believe in something supernatural providing extra computational capacity, we know that the physical limits allows the manufacture of a sub 20W computational device in about 1.5kg of matter with the capacity of a human brain.

(I point out that there is a lot of evidence that the size of the brain in the human has little to do with the IQ of that human. It's not at all clear that you could scale something that works on whatever principles the human brain works on by adding more neurons the way you can scale a transistor based computer by adding more transistors.)

If the self-improving AI stops when it reaches human intelligence, that's amazing, but it's not the singularity.

What I'm saying is that the 'hard takeoff' theory is wrong because making a thing that can make smarter copies of itself... isn't going to mean you get something infinitely smart. It means that you get something that bumps up against the limits of whatever technologies it can use or discover.

Yeah, maybe we'll come up with some new tech that makes slightly-better-than-human brains possible, and those brains will come up with a technology that makes god-brains possible... but there's no evidence at all that what we'd perceive as a god-level intelligence is even possible.

We have no idea how close to the theoretical limit the brain is.

>It may be impossible to reach the same using silicon, but we know it is possible to get there somehow within those constraints.

To get to a human-level intelligence, sure. I'm not saying that AGI is completely impossible, just that a hard takeoff to god-intelligence AGI before we wipe ourselves out or die from other causes is... unlikely.

really, my most important point is that human-level AGI does not necessarily lead to that AI creating smarter copies of itself until we get to the singularity /because physical limits exist/

(I mean, I guess then you are placing your bets on where those physical limits are)

Re: Superintelligence: The Idea That Eats Smart People (2016)

#249

Post by lsc »

nl wrote: post #106
Piskvorrr wrote: post #102

Earlier quoted context omitted.

(6) It is easier to move a problem around (for example, by moving the problem to a different part of the overall network architecture) than it is to solve it. In other words, saying "it's the definition that we don't have, not the implementation" just pushes the magic smoke around.

I don’t understand what you are saying. I’m saying that I don’t think human intelligence is doing anything more than pattern matching, logical inference and post-hoc justification. Give me an example of something that falls outside that. There is no magic.

There is a lot in 'logical inference' that we still don't really know how to describe well enough to tell computers how to do it. (I mean, I think it's super interesting how we figured out how to teach computers to do pattern matching... but I think logical inference is a different sort of thing that won't fall to the same tools)

But really, I think your idea that we're mostly 'post-hoc justification' is interesting. I mean, I've read some research to that effect, too; but it's... creepy and doesn't line up with anything reasonable about free will or really about long-term planning.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#250

Post by lsc »

red75prime wrote: post #132
TeMPOraL wrote: post #128

Earlier quoted context omitted.

I don't think GP means "magic smoke" as in actual magic , some new physics. It's more like we just have few potential building blocks (e.g. pattern matching), but we're not sure if we have all of them, and we definitely don't know how to connect them together to create sentience. (And no, more GPUs running faster DNNs doesn't seem like an answer.)

> to create sentience The goal is to create AI that can solve any problem a human can solve. Whether sentience is a necessary side effect to that is an open question.

I thought the idea was to create an AI that was way smarter than a human? I mean, if you just have human level AI, that's not gonna cause the singularity unless there aren't any physical limits preventing it from scaling itself up, and scaling itself up a lot
Post reply on HN