>Talk about your sexist, racist , ageist comments!
That really depends on the definitions of those terms. In a pedantic "by the dictionary" sense, yes, those comments are racist, sexist and ageist. But not in a way that can be meaningfully compared to the usual referents of those terms.
You see, these concepts are tied up with the idea of privilege[1]. "Old white guys in the GOP" draws a boundary around a cluster of one of the most privileged groups in the entire world. There are important practical differences between prejudiced and generalized language directed at oppressed groups, and prejudiced and generalized language directed at a group in power.
When we hear someone say "old white guys are the problem with the GOP", we know that we're not talking about the unprivileged outliers in that group. We are talking specifically about one of the most privileged groups in America, and crucially, the problem we are talking about is directly tied to their privilege. The implied claim is not that they're sucking up public resources, or "taking our jobs", etc. It's that they have power and that their privilege prevents them from understanding many of the issues that are important to unprivileged groups like women and minorities. In short, the problem that "old white guys" are accused of causing is something that they have the ability to change.
Now don't get me wrong: this kind of prejudice is not harmless. It still attaches a stigma to those who fall within the extension of the term, but not the intension. But the harm it does cannot be seriously compared to the harm done by prejudice against unprivileged groups.
>Substituting the words "Democratic" and "young" and "of color" sounds pretty silly.
And now we can see what the difference is. "Young black men" draws a boundary around a cluster of one of the least privileged groups in America. Even though you only substituted apparently parallel terms in the sentence, the meaning of the sentence is drastically different. Once you take into consideration the societal context, you can see that "the problem with the Democrats is young black guys" isn't the other side of the coin. The "old white guys" statement is about a group wielding power while blinded by privilege. The "young black guys" statement is about... what? It can't be about power and privilege because power and privilege are outlier properties in that group. So whatever problem is, it's probably not something that the people in question have much control over.
And that is a huge difference.
1. http://en.wikipedia.org/wiki/Privilege_(social_inequality)