Yeah, humans suck at attention. But until we can program computers to tell me what is a foreground moving object, what is a shadow, and what is the background... you literally can't solve the "Should the car apply the brakes right now" problem.
https://arxiv.org/pdf/1801.02225.pdf
Just go through this recent paper I found on DDG.gg and see the amount of effort it takes to parse foreground / background data that a self-driving car needs.
You gotta figure out if there's a still-object on the road. And whether or not its a shadow (shadows don't move after-all but its safe to drive over). Like, neural networks can't do that stuff 100% reliably yet. And it may never happen.
Or some researcher next year might come out and discover a method to parse background / foreground / shadows out of pictures. But then there are a whole host of OTHER issues involved.
------------
I think self-driving technology has potential. But you need more than just neural networks hooked up to cameras. I really like Waymo's direction with advanced super-human sensors. Avoid the shadow / background / foreground problem entirely and just have LIDAR give you the precise coordinates of all objects within 100-feet of the car.
IMO, if self-driving technology ever happens, it will be because of advances in advanced LIDAR or other kinds of sensors. Stuff that can avoid the research-problems that the "Deep Learning" community hasn't been able to solve for the last 50 years.