There's a theory in memory research that says that you need to focus or at least notice an idea or thing before you can remember it. Example: given a picture, few people will be able to completely describe it but you'll be able to describe those areas you focused on or caught your attention.
We think that we remember everything we see or hear but that's impossible since in order to remember the data first has to be detected and it has to go through short-term memory before it can be stored in long-term memory. Short-term memory can hold about 7 items at one time. Also, our memory's nature is to forget unless it's important.
So given the above, it's impossible to retain much of what we hear vs reading. We get the illusion that we are capturing everything we hear but we are not. We capture what we think is important but it's going by so fast that it's just bits and pieces with our minds filling in the blanks. Basically, what audio is doing is keeping our minds busy.
So if you want to get the basic idea of a book you listen to it. If you want to retain the most of it you read and you try to recall what you just read.