Bright colors used to be a sign of quality, everything expensive had tons of colors. Industrial revolution changed that, now bright colors are associated with cheap toys instead because they are used to cover deficiencies in the production process. So it makes sense that people who care about quality moved away from colorful things.
Women’s clothing is more colorful today. I don’t think it is because women care less about quality than men.