> But of course we know that there are all kinds of configurations of matter, like a motorcycle, that are faster than a cheetah and even look a little bit cooler.
yet more inflammatory rhetoric from the Pinboard guy
Superintelligence: The Idea That Eats Smart People
31–40 of 607 posts
Re: Superintelligence: The Idea That Eats Smart People
#32In 1945, as American physicists were preparing to test the atomic bomb, it occurred to someone to ask if such a test could set the atmosphere on fire...Los Alamos physicists performed the analysis and decided there was a satisfactory margin of safety.
Exactly how big was this margin? Please someone tell me this was quite a large margin!
Re: Superintelligence: The Idea That Eats Smart People
#33"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…
That's just vitalism:
https://en.wikipedia.org/wiki/Vitalism
Either everything exists in the material world, or it doesn't. There's really no way of avoiding that binary. If you think there's something else, that is by definition supernatural.
Re: Superintelligence: The Idea That Eats Smart People
#34What about the counter-argument from domestic canines:
More likely, artificial intelligence would evolve in much the same way that domestic canines have evolved -- they learn to sense human emotion and to be generally helpful, but the value of a dog goes down drastically if it acts in a remotely antisocial way toward humans, even if doing so was attributable to the whims of some highly intelligent homunculus.
We've in effect selected for certain empathic traits and not general purpose problem solving.
Pets are not so much symbiotic as they are parasitic, exploiting the human need to nurture things, and hijacking nurture units from baby humans to the point where some humans are content enough with a pet that they do not reproduce.
I could see future AIs acting this way. Perhaps you text it and it replies with the right combination of flirtation and empathy to make you avoid going out to socialize with real humans. Perhaps it massages your muscles so well that human touch feels unnecessary or even foreign.
Those are the vectors for rapid AI reproduction... they exploit our emotional systems and only require the ability to anticipate our lower-order cognitive functioning.
If anything, an AI would need to mimic intellectual parity with a human in order to create empathy. It would not feel good to consult an AI about a problem and have it scoff at the crudeness of your approach to a solution.
Even if we tasked an AI with assisting us with life-optimization strategies, how will the AI know what level of ambition is appropriate? Is a promotion good news? Or should it have been a double promotion? Was the conversation with friends a waste of time? Suddenly the AI starts to seem like little more than Eliza, creating and reinforcing circular paths of reasoning that mean little.
But think of the undeniable joy that a dog expresses when it has missed us and we arrive home... the softness of its fur and the genuineness of its pleasure in our company. That is what humans want and so I think the future Siri will likely make me feel pleased when I first pick up my phone in the morning in the same way. She'll be there cheering me on and making me feel needed and full of love.
Re: Superintelligence: The Idea That Eats Smart People
#35This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…
That isn't exactly what it's doing. It's proposing that there are two ways we evaluate things — deeply examining and rationally analyzing them in depth to identify specific strengths and weaknesses, and using the very fast pattern-matching "feeling" portions of our brains to identify nonspecific problems. These are cognate to "System 1" and "System 2" of Thinking Fast And Slow.
Having established that people evaluate things these two ways, the author then says, "I will demonstrate to both of these ways of thinking that AI fears are bogus."
Re: Superintelligence: The Idea That Eats Smart People
#36"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…
Any recommendations on decent pop-sci books that explore the latest understanding of the origin and mechanism of consciousness?
Is anyone exploring this third way at all?
Re: Superintelligence: The Idea That Eats Smart People
#37Earlier quoted context omitted.
The place where AI alarmists seem to forget this most often is in computational complexity, and particularly in the power of super-hyper-computers to simulate reality. Bostrom in particular doesn't seem to appreciate the constraints complexity theory puts on what an AI could calculate (unless he thinks P=NP).
Why would they be any more constrained than our biological brains? Humans are able to calculate plenty with a relatively small amount of mass and energy.
I mean that for a lot of things, doubling our brainpower would would only extend our predictive horizon by a tiny amount.
There is a large class of problems for which you have fairly good solutions up front, but where even vast amounts of additional computing power don't help you do much better.
Re: Superintelligence: The Idea That Eats Smart People
#38"The assumption that any intelligent agent will want to recursively self-improve, let alone conquer the galaxy, to better achieve its goals makes unwarranted assumptions about the nature of motivation." This isn't just an unreflective assumption. The argument is laid out in much more detail in "The Basic AI Drives" (Omohundro 2008, https://selfawaresystems.files.wordpress.com/2008/01/ai_driv... ), which is expanded o…
Certainly the assumption that every intelligent agent will want to recursively self-improve is unwarranted.
But it only takes one intelligent agent that wants to self-improve for the scary thing to happen.
Re: Superintelligence: The Idea That Eats Smart People
#39This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…
> The problem with argument ad hominem isn't that it never works. It often leads to the correct conclusion ...
You immediately self-contradicted here. If a decision heuristic often leads to the correct conclusion, then it's a good heuristic (if you are under time constraints, which we are).
Of course, given unlimited time to think about it, we would never use ad hominem reasoning and consider each and every argument fully. But there are tens of thousands of cults across the world, each insisting that they possess the Ultimate Truth, and that you can have it if you spend years studying their doctrine. Are you carefully evaluating each and every one to give them a fair shake? Of course not. Even if you wanted to, there is not enough time in a human lifespan. You must apply pattern-matching. The argument being made here isn't really an ad hominem, it's more like "The reason AI risk-ers strongly resemble cults is because they functionally are one, with the same problems, and so your pattern-matching algorithm is correct". Note that the remainder of the talk is spend backing up this assertion.
There's a good discussion of this in the linked article about "learned epistemic helplessness" (and cheers to idlewords for the cheeky rhetorical judo of using Scott Alexander and LW-y phrases like "memetic hazard" in an argument against AI risk), but what it boils down to is that our cognitive shortcuts evolved for a reason. Sometimes the reason is because of our ancestral environment and no longer applies, but that is not always true. When you focus solely on their failure cases, you miss sight of how often they get things right... like protecting you from cults with the funny robes.
> "if they look like a crank, ignore everything they say", then you miss special relativity (and later general relativity).
A lot of people did ignore Einstein until the transit of Mercury provided the first empirical evidence for relativity, and they were arguably right to do so.
Re: Superintelligence: The Idea That Eats Smart People
#40"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…
Explanatory Gap is BS. This is what happens. Then why does it not describe how it feels?
You are not wired up to understand the low level effects, it's simply not a useful. At the same time it does describe how it feels, if we started using firing of C fibers instead of Pain people would assume they understood what was going on.