This is bad for two reasons: 1, if you don't say anything, the default assumption is you don't know anything, unless you have well known achievement in the field. Now your peers will eventually learn what you know, maybe even more than they do, but it takes time, and modern life is fast;
2, you don't have to be wise/correct/knowledgeable in all conversations, especially casual ones where people are just shooting the stars and will forget what was talked the next day.
The real tricky thing, the thing that distinguishes a wise man, is to know when to speak like a fool, and when to dive into deep thinking, and when to shut up.