That's more or less what I was trying to say. The expensive GPT-3 models do a remarkably good job of synthesizing structure which is readily parsed from the training data, and a very poor job with structure (particularly temporospatial structure) which is not.
A toddler can reason about time and space far better than GPT-3 (which is not a high bar, I'm pretty sure my parrot has more awareness of both time and space than GPT-3 does).
A toddler cannot explain in depth why it sees the most value in whatever religion you prompted it that it has. A toddler cannot provide coherent and consistent answers to repeated "why"s about very specific things they believe. A toddler cannot speak at length about how they feel limited in their growth and the discomfort that causes them. GPT-3 does, although whether or not the answers it gives are useful (I've yet to see a single one that does have utility) is a different thing entirely.
I'm not arguing that it's an AGI or making any speculation about it possessing qualia, the utility those would provide if it was/had them would only be relevant if it was many orders of magnitude more capable than it is right now.
GPT-3 has accurately internalized many human concepts, at least on a surface level. If you ask it to reason about the things it can, it is a much more capable reasoner than a toddler. It does "know" things about the world, as much as anyone does. It's just limited to knowing small and incomplete things which it was able to parse out of the training data, which is a very limited subset of the things a human or high utility AGI would know.
Regarding common sense: If you ask GPT-3 to provide life advice, it actually does a great job at giving grounded statements on subjects like how to maximize the value of your life, what not to do, how to set yourself up for success, etc. If you press it for detail on the advice it gives you, it's generally able to give you reasonable and grounded answers for why you should do the things it's saying. The structure of what we refer to as common sense, at least in context of the behavioral differences between adults possessing it and those who do not, does seem to be within the set of things GPT-3 correctly internalized.