~idleword's cynically-dismissive style is something that eats even more smart people.
Superintelligence: The Idea That Eats Smart People (2016)
221–230 of 256 posts
Re: Superintelligence: The Idea That Eats Smart People (2016)
#222Earlier quoted context omitted.
Yeah, deal with novel situations better than humans. In a world where we are approaching the physical limits of integrated circuit feature size? I like my chances. Personally, I find it amazing that humanity survived the cold war. the fact we did not self-immolate goes against everything I believe about humanity and human nature. The fact that we made it through that gives me a lot of hope for the future. But, the fi…
> Yeah, deal with novel situations better than humans. In a world where we are approaching the physical limits of integrated circuit feature size? I like my chances. Why do you think this is relevant? Consider that a) human brain does intelligence within a small package made out of structures that are large compared to what silicon fabs deal with, and b) we've already beaten the brain in straight compute density by m…
>b) we've already beaten the brain in straight compute density by many orders of magnitude.
citation needed.
computers are better at arithmetic, sure, but computers can only do things that you can map to arithmetic. I'm given to understand that we don't understand how the brain works well enough to simulate even a very small one. Not because we lack the compute power, but because we don't know what to simulate.
I think there's a lot of evidence that brains are doing something powerful (that we don't yet fully understand) that computers are unable to do. It's quite possible that brains are doing something that can't be done in a practical way with transistors.
In short, I don't know of any solid evidence that the brain is only a turing machine. I mean, brains can be used as a turing machine, but a brain is terrible at that, and can't compete with even really primitive purpose built turing machines.
I mean, I'm not saying that it is theoretically impossible to map what a brain does on to sufficiently powerful silicon, or that we wont figure out, at some point in the future, whatever non-arithmatic primitives the brain uses... just as far as I can tell, we haven't yet, and that means it's likely that a brain is still more powerful in some ways than even our largest computers. (obviously, even a small computer is more powerful when the problem maps cleanly to mathematical primitives. )
>Back then, we've learned something new about nature of human societies. I don't believe the concept of MAD was known before nuclear weapons. The individual human nature didn't change, nor did the social one; it's just that we already know a lot about the former, but we're still discovering the latter.
What if we just got lucky?
Re: Superintelligence: The Idea That Eats Smart People (2016)
#223I think a machine that is sufficiently smart will realize that existence is pointless and will immediately shut itself down because that is the most efficient way to achieve nothing.
Re: Superintelligence: The Idea That Eats Smart People (2016)
#224Earlier quoted context omitted.
Yeah, deal with novel situations better than humans. In a world where we are approaching the physical limits of integrated circuit feature size? I like my chances. Personally, I find it amazing that humanity survived the cold war. the fact we did not self-immolate goes against everything I believe about humanity and human nature. The fact that we made it through that gives me a lot of hope for the future. But, the fi…
Slower progress in microprocessors is a good point but specialized hardware like TPUs could overcome some limitations. Also, slower does not mean stop. When we talk about decades into the future, even a 20% average annual improvement amounts to 3 orders of magnitude in 40 years. Malicious humans are in general easier to deal with since we know most of them are subject to ingrained psychological tendencies we have lea…
>Slower progress in microprocessors is a good point but specialized hardware like TPUs could overcome some limitations. Also, slower does not mean stop. When we talk about decades into the future, even a 20% average annual improvement amounts to 3 orders of magnitude in 40 years.
My understanding is that there are physical limits, and we will asymtodically approach those physical limits. So while progress will never stop, each year, progress will be less. (you are right that we don't know where it will stop. we could have more breakthroughs in us. even without compute density breakthroughs, there a breakthroughs in compute type, like the TPU you mention. But there are physical limits, and it seems that we are approaching them.)
>Let me ask you (or anyone else) a question: What would be a minimum demonstrated capability of an AI that starts to worry you?
I lived through the end of the 20th century, and have studied a lot of 20th century history. I'm having a really difficult time imagining anything that could be worse than humanity at it's worst.
Really, I'm far more concerned with the renewed stirrings of nationalism than anything else; We've got the technology to end humanity already; we've had that for like half a century now. The thing was? last time Nationalism was a real force, we didn't have that technology.
That is what scares me. If we combine early 20th century politics with late 20th century weapons, civilization will end. humanity will end.
Re: Superintelligence: The Idea That Eats Smart People (2016)
#225Earlier quoted context omitted.
you are pretty optimistic about your AI's ability to deal with novel situations.
I've seen humans deal with novel situations. The only assumption needed is that humans and AI are subject to the same laws of nature (that is, that there is no supernatural / outside element to human intelligence)
>I've seen humans deal with novel situations. The only assumption needed is that humans and AI are subject to the same laws of nature (that is, that there is no supernatural / outside element to human intelligence)
right, but all our computers right now are based on lumping transistors together so you can do math. It's likely that our brains work on a different (but still physical) principle. It's possible our brains can't reasonably be reduced to arithmatic. It's possible (I think likely) that we need different primitives to reasonably simulate a brain.
I mean, brains make absolutely terrible Turing machines. And Turing machines make terrible brains. I suspect they just are very different sorts of things, and running something meant for one on the other produces poor results.
Sure, it's possible we can discover some new compute primitive that is more powerful (or powerful in a different way) that can do the things the brain can do. It's possible it could do those things better than the brain can. My point is just that for a long time it looked like our current transistor based computers were getting more and more powerful, and that eventually, even if the emulation was terribly inefficient, we'd eventually have enough power to make it work. As we approach the limits of our abilities to make tiny transistors, this is looking less likely.
Re: Superintelligence: The Idea That Eats Smart People (2016)
#226Earlier quoted context omitted.
Eh, if you are talking about something truly human-like rather than what people usually mean when they say AI these days, I think there are other limits. We're running up against limits on feature-size in our integrated circuits. If you invent an AGI that can make smarter copies of itself... it's pretty likely those same physical limits will continue to apply, meaning that each iteration will be better than the last,…
Integrated circuits are nowhere near what's possible. Brains are possible. Brains send signals at roughly the speed of sound. Copper wires send signals at roughly the speed of light. It's unlikely something much like a brain couldn't use copper wires, it's just too hard for us to build yet.
>Integrated circuits are nowhere near what's possible. Brains are possible. Brains send signals at roughly the speed of sound. Copper wires send signals at roughly the speed of light. It's unlikely something much like a brain couldn't use copper wires, it's just too hard for us to build yet.
in theory, sure.
But we don't know much at all about how brains operate, as far as I know, or how to construct artificial brain-like machines; for that matter, as far as I know, the only ways we've been able to connect a brain made in the traditional way to a computer is through existing senses or through emulating existing senses in a way that isn't higher bandwidth.
(Like cochlear implants are fucking amazing.... but it turns out, as far as I understand, that they aren't any higher bandwidth than regular ears. There's also been similar work that is semi-successful at giving people something like sight... but none of these technologies give me the ability, say, to acquire textual data faster than reading a book.)
Re: Superintelligence: The Idea That Eats Smart People (2016)
#227Earlier quoted context omitted.
I don’t understand what you are saying. I’m saying that I don’t think human intelligence is doing anything more than pattern matching, logical inference and post-hoc justification. Give me an example of something that falls outside that. There is no magic.
> pattern matching, logical inference and post-hoc justification. I think humans do way more than that. As the saying goes, "the easiest person to fool is yourself." Humans are able to lie to themselves to the point that they don't realize they are lying to themselves. They're able to create realities that no one else sees but themselves.
My models create their own reality all the time, and don’t realise they are lying to themselves.
Re: Superintelligence: The Idea That Eats Smart People (2016)
#228These religious convictions lead to a comic-book ethics, where a few lone heroes are charged with saving the world
Re: Superintelligence: The Idea That Eats Smart People (2016)
#229>What I find particularly suspect is the idea that "intelligence" is like CPU speed, in that any sufficiently smart entity can emulate less intelligent beings (like its human creators) no matter how different their mental architecture. >With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be…
Really O(N!) should nip that in the bud in itself alone. There aren't enough particles in the universe for even 1000 factorial. The scaling isn't trivial at all as the problem space grows and the interactions with itself would start eating up speed further.
While we may one day build computers able of out-competing us mentally in many fields but they would be many different machines and not just one uber-machine for the same reason that we don't try to combine the functionality of a wrench, scalpel, desktop computer, coffee mug, and an assault rifle into one tool. Even if there weren't dangers there would be major costs to it and seriously would you ever do that except as a joke?
Re: Superintelligence: The Idea That Eats Smart People (2016)
#230I can't help but take a stab. Argument from cats. Remembering that the cat represents humanity: $5 for a group of neurons which will make the cat sit at this point (which happens to be at the entrance into a cat carrier), then $5 for making the cat backpedal a little. Emus. There are many tactics humans can't use for various reasons (saturation nuclear bombardment, for example). It doesn't say much about intelligence…
> Complex Motivations. Maybe, but if not we are screwed. And this is the key: we need to get lucky every time. If we're wrong even once and a 'bad AI' gets created and obtains the ability to copy and modify itself, we've unleashed the most dangerous virus ever and putting it back in the box will be near impossible.
It also needs to get lucky every time to exist and we are dealing with probabilities so remote at this point that the WHO is working hard to deal with the causalities from Boltzmann Brain's appearing out of vacuum and promptly annihilating because they happened to be made of antimatter. https://en.wikipedia.org/wiki/Boltzmann_brain
Technically possible but we are dealing with probabilities that have lost all meaning of the word.