Can I read a physical book when I'm running? Can I read a physical book when I'm doing house work? Or throwing pots in my studio? Or knitting? Or cooking? Or driving? Can I stay focused if I'm just sitting and listening instead of reading? Absolutely not. There's no 'vs' here. They occupy different spaces. All hail text, whatever form it takes.
How do you listen to audio books when you're engaged in other activities? You'll be distracted and won't get everything. That's important to me.
I think it entirely depends on what audiobook you're listening to. If I'm listening to something that's lower complexity or aimed at younger audiences (e.g. The Martian by Andy Weir or The Hitchhiker's Guide to the Galaxy by Douglas Adams), I can miss bits and pieces of the narration without losing too much of the story. As long as you don't lose track of several uninterrupted minutes at a time, you'll probably get most of the relevant context. The same applies if I'm in the mood to listen to a story that's already familiar to me, like Lord of the Rings.
It definitely becomes more difficult to multitask with harder reads; which is where I prefer to have a book in hand. I'll have to rewind several times per chapter to catch everything. Though it's still doable for some, I'm sure.