It is those skills, with the added nuance of doing it at speed. In this sense I think it's like getting fluent at reading and speaking a language: For some reason it gets harder as we get older.
Naturally there are some social barriers as well. The joy of playing music as a kid is that you're allowed to suck and everybody claps. As an adult, you're conscious of your own suckage, so you have to work on your skills alone. And when adding a skill like reading, you can already play and the reading is making it noticeably worse not better, until you get over that hump. Learning it by immersion gets harder when people expect you to be good at it.
Not saying it can't be done, but it just gets harder. It can certainly be done starting at middle school age, which is when most wind players start.
Now, get that fakebook out again, and use it for reading practice. That's a valuable resource. There's also a huge pile of classical guitar literature including etude books that are graded by level, so you can work through them progressively.
The weird thing about jazz is that the opportunities for doing actual reading are disappearing as the whole jazz and live music scene evolves. I've been lucky to play in larger jazz ensembles, where the charts are largely written out and reading is a vital skill, so I've been able to maintain my reading chops possibly better than some pro's.