Earlier quoted context omitted.
> I've heard it said that true understanding is demonstrated when someone can explain difficult concepts well. I think this not generally. I worked together with this amazing engineer, but he really struggled to sometimes explain what he was trying to do. He came up with great solutions, but often took us some time to figure out what he was trying to get at.
I've experienced something similar, I still think the quote works - If you can explain difficult concepts well, it's a demonstration of true understanding. Not that poor communication demonstrates lack of it.
You can do things extremely well without having the foggiest about the actual underlying principles, just from observations and intuition. Doubly so if the process can be machine automated, which by this point encompasses nearly everything to some extent. Sufficiently advanced overfitting is indistinguishable from generalization.