Working out is bike rides for me, so that means taking my hand off the bars. I mean, in comparison to a phone, what does the watch offer when you're lifting? How will you dismiss a notification? What's the real difference between just picking up the phone or having the phone or a tablet mounted somewhere you can see at all times?
I do a lot of walking downtown and half the time I have a cup of coffee in my hands or something else. Responding or interacting with the watch becomes impossible without two hands or I guess talking into it.
You need two hands to use a watch: one to bring it your face, and the other to interact with it.
Believe me, I want to use a smart watch; it feels like every kid would have wanted one back in the day. But when you think about them, they're not as useful as they seem due to many factors, and I think most people get that, because you don't see many people wearing them. They're definitely still a niche product.
But how many people wear glasses? What if they did more than help you just see better? How many people might drop contacts for that value, even? (Some people wear glasses over contacts purely for fashion.)
You don't have to do anything to see information in your always-present glasses, and you only need one hand to operate them. Alternatively, a mic would be closer to your mouth so that's more functional than bringing your hand to your face to give commands as well.