Earlier quoted context omitted.
Although i fundamentally agree with you, i will say that it is impossible to know which "tricks" will lead to a leap in cognition, even looking evolutionarily, the difference in genome between a monkey and human is very small percentage wise, yet the effects of that small difference are stark.
I'm always on the position that at some point (i make no prediction of when) "AI" will still be labeled as not-intelligent, but to external observers it won't be possible to know. Ie we will be able to describe how a function of `add(1,2)` is most clearly definitely not intelligent. We will understand the internal makeup of these neurons better, the whole system will be deterministic and understood - just a very larg…
That concepts like "consciousness," "free will," and "qualia" are nebulous precisely because they were invented by us to describe things we don't understand yet we feel obligated to insist are special and unique characteristics of our identity.