Is Artificial Intelligence Permanently Inscrutable? (2016)

Do some of you work in data science or machine learning occupations? You will realize how unreal those predictions sound. It's mainly cleaning data up, getting some random correlations from plugging variables into deep learning algorithms and hoping that it doesn't break with the most silly outputs you can think of.

Many scientists warn that AI is the biggest threat to humanity. I think this is wishful thinking for now.

Let's talk about this in 20 years. I think we will have 2 GAI hypes until it can finally happen (in my eyes it's mainly two problems: computational power and trying to rebuild a biological brain i.e. its structure and pre-trained functional components). I think wetware [1] is very promising in the near future and more efficient methods like IBM's neurosynaptic processors in the middle term.

Machines will do what we tell them to do for a long time. Maybe they will use unconventional methods, but they won't destroy humanity because "they're tasked to minimize suffering and decides that this is the best option". They will search through all laws, assist doctors and even detect depression, but they won't do anything besides their task.

Even Boston Dynamics is mainly using control theory. Making things work with deep learning requires heavy computational power. As long as there is no real-world Umbrella Corp., it's mainly sci-fi. There are a lot more and cheaper ways to do harm. Infiltrating and poisoning groundwater, hacking critical infrastructure, building biological weapons etc.

It's hard enough to build bug-free software, it's even harder to build an AI that is unbelievably intelligent (reminder: we don't have such a thing. Most autonomous cars use rule-based systems, most ML is basically gluing models together which can be easily fooled and most of those things are very shaky and by no means intelligent) and contains a bug which leads to its world domination by accident or something similar. I would start to play lottery if this is even remotely possible because that would be a miracle (probability-wise).

[1]: https://koniku.com

I think part of the problem is that in most cases there are far too many dimensions at work for us to understand the explanation for anything but the most trivial of cases.

Imagine there is a job to identify Fred Rogers. You train for this job for weeks, looking up imagery of Fred as a young child, young man, and as an adult throughout his career. You become very familiar with Fred's physical characteristics...his posture, his body composition, the way he carries himself, his face and any identifying features therein.

After this extensive training session you are brought in front of an audit panel and asked to explain in excruciating detail how you make positive and negative identifications of Fred Rogers. Not just demonstrate your ability repeatedly, which would just be a sampling method, but to actually describe the approach.

I’m in agreement with you here. This trope that neural networks or other forms of ML are inscrutable is ridiculous. We know exactly what mathematical function is happening at every stage and the goal of the process; we can explain what architectures have the best results, and in some cases a deep understanding of why; with close enough scrutiny, you can probably tell what an individual neuron is doing to a layman (though the scale quickly gets out of hand). Why is it not sufficient to say “we’re using some basic math at a large scale using neural network architectures found to be best suited for the purpose of fitting our task, data, and loss functions?” These architectures are shown to find this sort of information, this is the general overview of how that works. What deeper explanation is desired?

I think the real issue is that tech journalists and laymen don’t know enough to understand. You can’t explain multivariable calculus to someone that doesn’t know algebra. Or option 2/3 are the problem.

I feel like a bulk of your premise here revolves around never understanding. But, lets say instead of AI, you just imagine some guy Bob being responsible for whatever you're worried about.

Bob is a reasonably intelligent guy, but he's not perfect, he makes an attempt to provide satisfactory explanations of his decisions, he tries to learn and improve, and he might secretly by manipulating the system either for entertainment or malice.

If you want to build a useful system around Bob, you have to consider all these limitations - you can't read his mind, you can't predict how he will make decisions in all situations, and you don't have a guarantee that he will give you a satisfactory explanation, or that any explanations are truthful or correct.

There's nothing dystopian about this situation - this is the world we live in. You just need to implement some resource constraints, ensure certain checks-and-balances for certain classes of behaviors, implement certain metrics to ensure behavior is according to your expectations, revise metrics and controls whenever you encounter problems, and don't give Bob unfettered control over nukes.

What you described isn't dystopia-land material unless you don't consider some very elementary controls that our society already has in place to prevent individuals from destabilizing society.

I've raised this a few times in the past, but, why would we want to build an AGI modeled after the thought patterns and processes of a human brain? We have plenty of human brains. it seems that we should want to avoid the same traps fallacies, biases, etc that humans have, and, if we accurately do so we won't have an AGI that is much like a human at all. I suspect we'll have something very powerful and capable long before we have something that approximates human personality and "intelligence", and even if we get to that latter state it will almost certainly be because the machine is faking it.

OTOH, if we try to directly replicate a human as we currently are, then I expect we should start with all the various mental shortcuts and cognitive biases we have. I'm no neuroscientist, but, one thing that is apparent in the comparison of machine learning models to humans is just how sparse the data that humans learn from is. I strongly suspect that our collection of mental shortcuts and biases, as well as our superior sensory input are what makes that difference. of course, if we succeed at making an AGI just like this, it will be just as good as humans at making very human mistakes.