We should figure out intelligence before speculating on super-intelligence. I think Kurzweil and Hofstadter have a compelling model in 'How to Create a Mind' and 'Surfaces and Essences: Analogy as the Fuel and Fire of Thinking', but it's not exactly rigorous and we still haven't created anything that could pass the Turing test which wouldn't even require high intelligence - just 2nd grade level or something.
Superintelligence: The Idea That Eats Smart People
41–50 of 607 posts
Re: Superintelligence: The Idea That Eats Smart People
#42"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…
That's just vitalism: https://en.wikipedia.org/wiki/Vitalism Either everything exists in the material world, or it doesn't. There's really no way of avoiding that binary. If you think there's something else, that is by definition supernatural.
There's no reason that something else can't exist in the material world, I think the parent is just pointing out the third way is thinking there's something else material outside our current understanding.
If God exists, there's no reason there couldn't be new phyisics describing how his world works.
Re: Superintelligence: The Idea That Eats Smart People
#43I think brains very probably use quantum effects in ways we might not even be able to study with anything close to today's technology. As a result, individual neurons or groups of neurons can be way more complicated than we are expecting them to be. I'd say we're at least 2 major revolutions away from even coming close to what a chimpanzee's intellect, much less a human's.
This is at least partly relevant to the discussion: http://www.smbc-comics.com/comic/the-talk-3 As a shorthand "quantum effects" are often hand-waved as crazy powerful "weird" things, but we can certainly model them and they really aren't as "magic" as a lot of popular science would have one believe (as much as even some of us otherwise rational people so strongly wish to believe in quantum magic).
Love the bit at the end: "Quantum computing and consciousness are both weird and therefore equivalent."
Quantum consciousness doesn't sound very different from plain old vitalism.
Re: Superintelligence: The Idea That Eats Smart People
#44"The assumption that any intelligent agent will want to recursively self-improve, let alone conquer the galaxy, to better achieve its goals makes unwarranted assumptions about the nature of motivation." This isn't just an unreflective assumption. The argument is laid out in much more detail in "The Basic AI Drives" (Omohundro 2008, https://selfawaresystems.files.wordpress.com/2008/01/ai_driv... ), which is expanded o…
So? Just because it's in some paper doesn't mean much. There are tons of BS string theory papers for example.
Re: Superintelligence: The Idea That Eats Smart People
#45"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…
Any recommendations on decent pop-sci books that explore the latest understanding of the origin and mechanism of consciousness? Is anyone exploring this third way at all?
Pop-sci is useless in this area. There is plenty of good research on the topic, but pop-sci on the brain is just as bad as pop-sci on quantum mechanics or computers.
EX: People making decisions based on eye tracking not matching up with the subjective experiences. Or the classic you chose not to do something, but your arm is already in motion and you can't stop it fast enough.
Re: Superintelligence: The Idea That Eats Smart People
#46"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…
Any recommendations on decent pop-sci books that explore the latest understanding of the origin and mechanism of consciousness? Is anyone exploring this third way at all?
[deleted]
Re: Superintelligence: The Idea That Eats Smart People
#47A key missing piece for intelligence: how many measurements can you make on the world? It doesn't matter how much computational power something has if it can't make measurements and ask and test hypotheses.
Edit: should state that this is a corollary of "> The Argument From Actual AI," but generalized to any 'intelligent' system, not just neural nets.
Re: Superintelligence: The Idea That Eats Smart People
#48"Hopefully you'll leave this talk a little dumber than you started it, and be more immune to the seductions of AI that seem to bedevil smarter people. "
Nice.
All in all, effectively reasoned. I've been making similar arguments for the last few years. AI is likely to create a lot of problems, and solve a lot of problems. But I think both aspects are messy and our relationship with our future technology will be complicated and fraught with regular human issues.
Some of those potential issues are very serious, yes, but serious like automating jobs and not solving the employment issue, or creating a very effective army of automated drones and single-handedly taking over a country (or, sure, the world), not issues of AI destroying the planet and/or enslaving all of mankind.
Re: Superintelligence: The Idea That Eats Smart People
#49"The assumption that any intelligent agent will want to recursively self-improve, let alone conquer the galaxy, to better achieve its goals makes unwarranted assumptions about the nature of motivation." This isn't just an unreflective assumption. The argument is laid out in much more detail in "The Basic AI Drives" (Omohundro 2008, https://selfawaresystems.files.wordpress.com/2008/01/ai_driv... ), which is expanded o…
Certainly the assumption that every intelligent agent will want to recursively self-improve is unwarranted. But it only takes one intelligent agent that wants to self-improve for the scary thing to happen.
>But it only takes one intelligent agent that wants to self-improve for the scary thing to happen.
Only if all sorts of other conditions (several of which are mentioned in the post) also apply. Merely "wanting to self-improve" is not enough.
Re: Superintelligence: The Idea That Eats Smart People
#50In 1945, as American physicists were preparing to test the atomic bomb, it occurred to someone to ask if such a test could set the atmosphere on fire...Los Alamos physicists performed the analysis and decided there was a satisfactory margin of safety. Exactly how big was this margin? Please someone tell me this was quite a large margin!
If I remember right, something like a safety factor of 30. So not really as big as people would like!
More detail in this great summary: http://large.stanford.edu/courses/2015/ph241/chung1/
And the Los Alamos paper is really good reading:
http://www.sciencemadness.org/lanl1_a/lib-www/la-pubs/003290...