I'd say you're basically redefining elitism to mean something it doesn't mean (or at least not how it's used here).
That said, I do agree with you. I think that 1) it can be be very important to make distinctions between types of programmers, because 2) it makes it easier to actively explore the field and finding what suits your 'type'.
For example, for many people, someone who 'does' HTML/CSS with some jQuery plugins, is a programmer. But personally I'd not really call such a person a programmer. I don't mean that as a value judgment, but rather to make a distinction between 'types' of work.
Making that distinction earlier in my career could've helped me (whether the label it programmer/non-programmer is used or not), because I spent more time than I'd have liked being such a HTML/CSS/JS guy. I learned tons of very specific rules/tricks/lore that were necessary, but did not help as a programmer, and spent countless hours doing this kind of work, not really enjoying it all that much.
It was only when I started doing 'proper' javascript stuff that I realized how much fun programming is, and how my relative enjoyment of the whole front-end stuff depended on the bits of 'real programming' I would occasionally get to do. In hindsight I wish I'd figured that out earlier, and not spent so many brain cycles and storage on CSS layout tricks and discussing how much 'semantic HTML' matters.
(thankfully those things are still useful when I do full-stack type stuff, but still)