AI is not even close to having true logical reasoning, that's probably decades away. The issue is that cognitive scientists are clueless. Scientists have a good model for
associative reasoning, which is the basis of modern neural networks, but we don't have a clue how
abstract reasoning actually works. All birds and mammals have advanced abstract reasoning and are far more intelligent than GPT-4:
- birds and mammals are inherently able to count in almost any context because they understand what numbers actually mean; GPT-4 can only be trained to count in certain contexts. GPT-4 would be like a pigeon that could count apples, but not oranges, yet biological pigeons can count anything they can see, touch, or hear. There's a profound gap in true quantitative reasoning, even if GPT-4 can fake this reasoning on specific human math problems.
- Relatedly, birds and mammals are far faster at general pattern recognition than GPT-4, unless it has been trained to recognize that specific pattern.
- Birds and mammals can spontaneously form highly complex plans; GPT-4 struggles with even the simplest plans, unless it has been trained to execute that specific plan.
The "trained to do that specific thing" is what makes GPT-4 so much dumber than warm-blooded vertebrates. When we test the intelligence of an animal in a lab, we make sure to test them on a problem they've never seen before. If you test AI like you test an animal, AI looks incredibly stupid - because it is!
There was a devastating paper back in 2019[1] proving that Google's BERT model - which at the time was world-class at "logical reasoning" - was entirely cheating on its benchmarks. And another paper from this year[2] demonstrates that LLMs definitely don't have "emergent" abilities, AI researchers are just sloppy with stats. It is amazing how much bad science and wishful thinking has been accepted by the AI community.
[1] https://arxiv.org/abs/1907.07355
[2] https://arxiv.org/abs/2304.15004