I used to think the same thing but got into watches a few years ago and feel equipped to answer this. A few reasons:
1) The mechanisms. It's just cool. Many of us on HN are engineers — wristwatches and time-keeping more broadly is one of the most classic, widespread forms of engineering. It's a form of engineering that has been around for a millennia, and it's fun to have something on your wrist, full of tiny gears with a satisfying subtle noise when held up to your ear, that can keep time to around 1-2 seconds a day with the 86,400 seconds in each day.
2) The history. There's so much to find that's unique. One of my favorite watches is the Doxa Sub300 in orange, it was the favorite of Jacques Cousteau who inspired so many of us with regards to diving (invented Scuba) and the ocean in general through his documentaries. Many of these brands have so much interesting history down to each model along with admirable craftsmanship. Some people like models that look like out-of-normality for a brand, like "oh wow can you believe Rolex did THAT?", e.g. a pulsation dial for a doctor to measure someone's heart-rate, while some like the classic model that exemplifies everything about the brand.
3) The accessory. As men, we don't typically have the amount of jewelry that women do, or the ability to wear the same wide range of dress. Watches are that thing where we can show off some aspect of our personality, of who we are or who we want to be, whether that's classy and subtle, fun and unusual, or even ostentatious.
4) The bonding. This is a smaller thing, but when you're into watches, and you meet someone else out in the wild who is as well, you have the best conversations. It's this thing you mutually geek out over, and it's incredibly pure. It's a great feeling, it's like finding someone who roots for the same sports team as you in some random city abroad.
Why expensive? In points 1 and 2, the expensive watches are the ones with the best mechanisms and the best history. In point 4, the expensive watch is the one that signals that you really care about watches, because it epitomizes 1 and 2. :)