Humans evolved colour vision to detect ripe fruit. Fruit has a, relatively, considerable amount of sugar. For such a substantial mutation, for which all of us share, it would seem counter-productive to evolve such a trait that harms us.
Earlier fruits didn't have anywhere near the sugar content of the cultivated ones today. In fact, many fruits of old were inedible (e.g. apples). Sources of sugar such as sweet fruits and honey were rare delicacies, not daily occurrences. Humans in various regions would have adapted over 1000 generations to better match their environment. We live in a VERY different world now.
I have heard that cereal crops (wheat, corn) were basically grass before farming started. This was a disadvantage for Aztecs and Mayans since it took a while for humans to migrate there, and then to cultivate them to a useful efficiency level.