While a great coffee can be a pleasurable experience that can actually make you more productive and happy in the short-run with possible positive long-term flow-trough effects (e.g., on life satisfaction), there are likely going to be more impactful uses of that money. As the most obvious examples, for instance, saving childrens lifes in Africa [givewell.org], improving the messy factory farming situation [gfi.org], etc. So wouldn’t you have the moral imperative to spend that money in a way that it’s impact is higher or at least comparable than those possible alternatives?
I would argue that in many cases this is actually possible when you are buying a coffee but not regularly, though... then you could likely find more effective alternatives, thus, the habbit would be obviously harder to justify for the vast majority of people.
My major quip with all of this is that our institutions could be designed in a way that would help us to make smart decisions (for us and others) but at the moment it is computationally really expensive to find a good path that balances all interests in way that leads to a great life for all.