Let me summarize the state of AI as a layman who has been following the main trends:
1) Cyc and reasoning - stuck for decades but making slow progress
2) GANNs - can detect a whole lot of things by being trained on data
3) MTCS - this is AlphaGo and AlphaZero. Adversarial networks that play each other. Have nearly solved any turn based game that can be expressed this way, and can be used for StarCraft etc.
4) Data fitting - can get scientific data and find new scientific theories and formulas to explain them, etc. by essentially building analytic functions from building blocks and optimizing for simplicity.
5) GPT3 and DALL-E - have very little understanding of anything, they just throw a lot of bullshit at a wall and see if it sticks. But since the bullshit is actually stitched from all serious human output that could be digitized, they sound quite serious at times (even though they show total lack of understanding when basic math is involved for instance, as numbers get higher).
Now I will be willing to admit (and have written about this) that logic and science may be a “poor man’s” million-dimensional vector data fitting. Maybe our binary logic and simple models are the thing that models the world worse than the million-vector models that capture the actual dynamics better. But humans won’t be able to understand these dynamics any more than a cat can understand most of your activities based on abstract systems.
All that other stuff — superhuman sensors, motors etc. gives them unfair advantages, like the finger on Geopardy. Auto-aim and self-driving cars are more about HUMANS improving, training, and then downloading the new software to every device.
What scares me most, though, is deepfakes and swarming. You don’t need GREAT AI to take over political discussions, or direct swarms of self driving cars or drones to wreak havoc. Online, swarms of sleeper bots can inftrate many communities and execute sybil attacks at scale. They can amass karma right here on HN with comments stitched through bullshit that sounds just like HN. It’s a positive EV strategy for the swarm. Eventually they can edge out all human participants where everything you see all day long is bots.
The kicker is THEY DONT NEED TO PASS THE TURING TEST in order to take over our systems because most people consume content non-interactively (read comments, watch videos) . “Truth” online is just a correlation of A vs B and every repository of “truth” can be flooded with a different correlation. Stephen Colbert fans even did it manually with elephants tripling in Africa! Imagine what botswarms can do to out sense of truth or political will or anything!