321–330 of 607 posts
> But of course we know that there are all kinds of configurations of matter, like a motorcycle, that are faster than a cheetah and even look a little bit cooler. yet more inflammatory rhetoric from the Pinboard guy
It's one thing to worry about AI's taking over the world someday . It's quite another matter entirely to think about current military automation of WMD deployment. Everyone's probably seen Dr. Strangelove at some point in time. If you haven't , stop reading immediately and go watch it. You will not regret this. Those who have watched it are familiar with a contrived, hilarious, but mostly plausible, scheme by which h…
Any kind of worrying about theoretical future death is silly, especially accidental death like the kind you are asking us to devote resources to. Why should I care about theoretical future death when there is real, actual death happening all around me?
Heck, if you want to worry about future death, why not worry about climate change deaths, which we can actually model? At least that gives us some chance to do something about it.
Of course I'm assuming you actually care about public health. Maybe you are just worried about your own health, and nuclear strike is relevant to you because it has the potential to pierce through the barriers that distance you from people who are actually facing mortal peril.
I think brains very probably use quantum effects in ways we might not even be able to study with anything close to today's technology. As a result, individual neurons or groups of neurons can be way more complicated than we are expecting them to be. I'd say we're at least 2 major revolutions away from even coming close to what a chimpanzee's intellect, much less a human's.
This is at least partly relevant to the discussion: http://www.smbc-comics.com/comic/the-talk-3 As a shorthand "quantum effects" are often hand-waved as crazy powerful "weird" things, but we can certainly model them and they really aren't as "magic" as a lot of popular science would have one believe (as much as even some of us otherwise rational people so strongly wish to believe in quantum magic).
Our ability to model quantum systems is feeble indeed. Anything more than about 40 qubits and it starts costing millions of dollars. 50 qubits just forget about it.
There's plenty of quantum condensed matter physics that is just completely impossible to model.
Even in chemistry, there are huge swathes of molecules that we fail to be able to model. This is hopefully going to be the "killer app" of quantum computers, btw.
I think brains very probably use quantum effects in ways we might not even be able to study with anything close to today's technology. As a result, individual neurons or groups of neurons can be way more complicated than we are expecting them to be. I'd say we're at least 2 major revolutions away from even coming close to what a chimpanzee's intellect, much less a human's.
Why are quantum effects complicated? They are random, however our behavior is not very random at all.
because of entanglement.
They always miss a critical and subtle assumption: that intelligence scales equal to or faster then the computational complexity of improving that intelligence. This is the one assumption I most skeptical of. In my experience, each time you make a system more clever, you also make it MUCH more complex. Maybe there is not hard limit on intelligence, but maybe each generation of improved intelligence actually takes lon…
I think the issue is that once do manage to build an AI that matches human capabilities in every domain, it will be trivial to exceed human capabilities. Logic gates can switch millions of times faster than neurons can pulse. The speed of digital signal also means that artificial brains won't be size-limited by signal latency in the same way that human brains are. We will be able to scale them up, optimize the hardwa…
You're equating speed with quality. There's no reason to assume that. Do you think an AI will be better at catching a fieldmouse than a falcon? Do you think the falcon is limited by speed of thought? Many forms of intelligence are limited by game theory, not raw speed. The challenge isn't extracting large quantities of information, it's knowing which information is relevant to your ends. And that knowledge is just as limited by the number of opportunities for interaction as the availability of analytic resources.
Think of it this way: most animals could trivially add more neurons. There's plenty of outliers who got a shot, but bigger brainded individuals obviously hit diminishing returns, otherwise the population would've shifted already.
While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…
Whatever goals the AI has, it will certainly be better at achieving them if it can stay alive. And it will be more likely to stay alive if there are no humans around to interfere. Now you might say, why don't we just hardcode in a goal to the AI like "solve aging, and also don't hurt anyone"? And ensure that the AI's method of achieving its goals won't have terrible unintended consequences? Oh, and the AI's goals can't change? This is called the AI control problem, and nobody's been able to solve it yet. It's hard to come up with good goals for the AI. It's hard to translate those goals into math. It's hard to prevent the AI from misinterpreting or modifying its own goals. It's hard to work on AI safety when you don't know what the first strong AI will look like. It's hard to prove with 99.999% certainty that your safety measures will work when you can't test them.
Things will not turn out okay if the first organization to develop strong AI is not extremely concerned about AI risk, because the default is to get AI control wrong, the same way the default is for planets to not support life.
My counterpoint to the risks of more limited AI is that limited AI doesn't sound as scary when you rename it statistical software, and probably won't have effects much larger in magnitude than the effects of all other kinds of technology combined. Limited AI already does make militaries more effective, but most of the problem comes from the fact that these militaries exist, not from the AI. It's hard for me to imagine an AI carrying out a military operation without much human intervention that wouldn't pose a control problem.
--------- Edited in response to comment--------
Earlier quoted context omitted.
It's a military AI that correctly interprets a command to kill a particular group of people, so effectively that its masters start thinking about the next group, and the next. This has been done many times by human-run militaries; would AI make it worse somehow? Groups of humans acting collectively can look a lot like an "AI" from the right perspective. Corporations focused on optimizing their profit spend a huge amo…
> This has been done many times by human-run militaries; would AI make it worse somehow? Solders in developed countries no longer want to die en masse.
Earlier quoted context omitted.
Reading 2% of your stuff is like reading 400% of my stuff, and I'm verbose as hell. Edit.
First off, that's a blatant ad hominem. Moreover, if your best argument against the guy you're claiming is defrauding everyone is "I can't be bothered to read his work"...
Earlier quoted context omitted.
There are also nearer-term threats (although I'd likely disagree on many specifics), but I don't see how that erases longer-term threats. One nuclear bomb being able to destroy your city now doesn't mean that ten thousand can't destroy your whole country ten years down the line.
I think the point (which is addressed with Maciej's Almogadro callback near the end) is that the longer term threat being speculated about and dependent on lots of very hypothetical things being true is pretty much irrelevant in the face of bigger problems. I mean, yes, a superpower that had hard military AI could wreak a lot of havoc. On the other hand, if a superpower wants to wipe out my corner of civilisation it…
If we want to grow to adulthood as a species and manage to colonize the cosmos, we need to pass _every_ skill challenge. If in 50 years there'll be a risk of unfriendly superintelligence and it'll have needed 40 years of run-up prep work to make safe, then it will do us absolutely zero good to claim that we instead concentrated on risk of military AI and hey, we got this far, right?
Considering the amount of human utility on the line, "one foot short of the goal" is little better than "stumbled at the first hurdle".
Earlier quoted context omitted.
I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.
What if you're the first human on an island full of baby tigers? I think most AI alarmists would argue that this analogy is vastly more appropriate.