As a Hindu I'd like to take the opportunity to share that the implementation of the caste system is not based on what it really means to be Hindu. The Vedas state that one's caste is fluid (not based on lineage) and can change in one's lifetime, and that further a person of the worker class is not lesser than someone of the priest caste, they both perform dharma necessary for the community.
Not an expert on the Vedas thus not sure if any value judgement is made regarding the varnas in it or not but I am definitely SURE it nowhere says that all "varnas" are equal and that mobility between varnas is possible or based on one's "merit". the varna division just by articulation implies a hierarchy. Who wouldn't see the priestly class, the keepers of knowledge, as superior than the sewage workers? Saying that…
You're also missing my point - of course people have abused the Varna system, but that's not how it has been described that it should be so.