There are some huge positives from driverless cars. You forgot to mention: driverless cars may have a positive impact on the economy (e.g. increasing worker productivity) and on the mental health of the general population (for those whose jobs won't be affected).
There will eventually be a time when cars are truly driverless. None of the passengers will need to be attentive to the car's driving. Once that becomes a reality, many people will have tremendous increases in leisure time during their day. Some people have massive commuting times to get to work.
Families will save money by not having to buy multiple cars. One car could be shared among all the members of a family. If each family member needs to be at different places at different times and all start at the same destination like the family residence, the car could pick up and drop off each person, while still returning to the same start destination. The children could be dropped off at school at an earlier time and you could be picked up later. In the meantime, while you are waiting for the car to return, you could find yourself having extra time to eat breakfast or to finish your coffee by not having to chauffeur anyone else. I find it strange that there will be driverless cars zipping around with no passengers.
I used to drive 50 miles back and forth every day, which ended up being an hour each way. If I can have a car that takes the stress and attention away from having to do the drive, that's 2 hours of my day that I can do whatever I want within the constraints of my car. I can watch a movie. I can read a book. I can get a head start on work. I can socialize with other passengers. I can learn that new programming language that I always wanted to learn. I can learn some other new skill like playing guitar. And, yes, even though driving can be an enjoyable experience, driving can definitely involve stress. Have you ever driven in a metropolitan area like New York City? Driving there can be completely chaotic.