I read “A Thousand Brains: A New Theory of Intelligence” by Jeff Hawkins, and it now is clear to me that our neocortex (like computers, or AI) is just a lot of general purpose computing infrastructure with ZERO aims. Our emotions (which drive everything in the interest of gene propagation- as there is no purely logical reason to do anything) would need to be intentionally duplicated to give AI a reason to desire anything beyond what we instruct it to do.
We need a Butlerian Jihad against AI
81–90 of 161 posts
Re: We need a Butlerian Jihad against AI
#82Wasn’t it Andrew Ng that said something like: Worrying about Superintelligent AI is like worrying about over crowding/population on Mars.
Andrew Ng is a luminary, but this isn't a great analogy. Humans aren't adapted to a Mars environment, and there's little incentive for us to move there. AI has so many practical uses that everyone is getting in the game and pushing the envelope. A huge amount of money is being spent on ML, and a wealth of expertise is being developed.
I can teach a human to drive in < 1 day. An AI… so far 10+ years. I agree there is crazy progress and investment, but it’s also not entirely certain how far we can push AI.
Re: We need a Butlerian Jihad against AI
#83The problem is, we don't really know how consciousness works (I assume consciousness the the part the author takes issue with; most of our cognitive faculties in isolation are not that special). We don't even have a great definition of consciousness, or good tests for it, or know whether it is a linear spectrum, or if it emerges abruptly with the evolution of certain reasoning and attention faculties, and we don't kn…
Also, if we model a brain effectively, then study that model, would our brain not develop new skills from reflecting on such a model and therefore develop again beyond the model?
Perhaps, but debatable whether knowledge itself adds something to cognitive capacity. Two thoughts: (1) Some theory of mind researchers equate having strong ToM abilities as roughly equivalent to having consciousness. I think knowing precisely how consciousness emerges may help develop our ToM. It would also be a tough sell to claim a mind with zero knowledge (a so called tabula rasa) has consciousness. On the other hand (2) Do feral or isolated tribes of humans have the same level of consciousness as those in developed societies? I suspect they do.
Re: We need a Butlerian Jihad against AI
#84It says a lot about what injustices we have learned to accept that AI alarmists focus almost exclusively on the scifi-level hypothetical dangers of AI, rather than the very real problems it already causes today. Those problems largely fall into three categories that I can think of off the top of my head at 1am: 1. AI is a convenient way to justify potentially uncomfortable decisions you would have made otherwise (idl…
"AI alarmists" are worried because the worst-case outcomes of AGI are mistakes you cannot ever fix. All the rest of these are bad, but they are problems we can fix given time and thought, because we will still exist. Extinction-level events decrease all future human utility to zero, and so should be treated with extraordinary care.
As you are talking about extinction level events, I'm not very confident that if humans have ultimate say over nuclear weapons, we will continue to not end our species with them.
It might in fact be a good idea to establish an AGI overlord which watches over humans and enforces nuclear non proliferation policies. If you look through history, it's full with war, genocide, and similar. Human societies are bound for change, and while it's been a peaceful few decades in which we had the nuclear button, it's basically ensured that we'll press it in the next 10 thousand years. How will technological civilization become million years old if not with the help of an AGI that enforces basic rules like "don't nuke each other"?
Re: We need a Butlerian Jihad against AI
#85>His point was that there are no odds that would rationally allow a parent to bet the life of their child for a quarter. Human nature just doesn’t work that way, and it shouldn’t work that way. People have done this for most of history. Working a farm, for example, is non-trivially dangerous and fairly low profit. Children often helped on the farm in rural communities from a young age. So every time you had your chil…
People drive fast to get their children to school in a few minutes less time daily, in the millions.
That’s betting their child’s life (at low probability) by increasing the risk of a serious collision... because they’re in a mild hurry.
Parents bet their children’s lives all the time.
The privilege is being taken seriously while saying something so afactual.
Re: We need a Butlerian Jihad against AI
#86The international community agrees that human/animal hybrids are abominations—we shouldn’t make them to preserve the dignity of the human, despite their creation being well within our scientific capability.
Theoretically, how would we create human-animal hybrids? That’s a strong claim to say it’s within our power.
Re: We need a Butlerian Jihad against AI
#87I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's…
I dont really get how people see the progress of the last 10 or even 5 years and then imply that things won't improve all that much further than that for a whole generation.
Re: We need a Butlerian Jihad against AI
#88I like to think of AI-Genesis through the lens of what humanity has already done through domestication. We take something primitive and progressively adapt it to serve a greater utility. I think working dogs are the most interesting example of this. We've taken a species, the wolf, and made it smarter while also making it want to do work, learn tricks, and follow orders. Of course, you still need to train the animal…
> We've taken a species, the wolf, and made it smarter
In what universe is a dog smarter than a wolf?
I think you got the correlation between animal domestication and their intelligence backward :)
Re: We need a Butlerian Jihad against AI
#89It's unlikely that we'll ever be able to put the genie back in the bottle.
The potentially limitless power of AI is going to be too much of a temptation for some governments, militaries, powerful individuals, and various groups to resist.
Ban AI in one place and it'll just be developed somewhere else.
That's not to say we should simply march forward with our eyes closed, but efforts to stop AI are highly unlikely to succeed.
Jacques Ellul[1] talked about the almost inevitable and freedom-denying aspects of what he called "technique" (really the relentless drive towards efficiency across all disciplines) in The Technological Society[2].
Ellul's thought inspired the Unabomber, though Ellul himself never advocated violence as a response. As we all know, the Unabomber did not himself stop technological progress... he did manage to spread his ideas a bit, but that hasn't seem to have had any effect either.
I'm completely against violence myself, and unequivocally condemn terrorism of all sorts (including the Unabomber). Not only is it unethical and immoral, but it's ineffective and counterproductive.
[1] - https://en.wikipedia.org/wiki/Jacques_Ellul
[2] - https://en.wikipedia.org/wiki/The_Technological_Society
Re: We need a Butlerian Jihad against AI
#90The international community agrees that human/animal hybrids are abominations—we shouldn’t make them to preserve the dignity of the human, despite their creation being well within our scientific capability. Theoretically, how would we create human-animal hybrids? That’s a strong claim to say it’s within our power.
Likely referencing https://www.nature.com/articles/d41586-021-01001-2