> Weren't castes based on race and conquering populations initially?
We don't know the exact process, but there were several culturally, ancestrally and often phenotypically distinctive populations that encountered and mixed with each over millennia in the subcontinent and engaged in both conflict and cooperation, mostly for arable and pasturable land resources and access to water.
The societal power and identity structures and population genetic patterns that emerged from that process manifest in, but are not limited to, the caste system.
A latter such migration brought the Indo European language and customs, and it has had an outsized impact on elite cultural artifacts like religious literature. Colonial philologists discovered the Indo European linguistic connection between Sanskrit and European sacred languages and some became fetishized on it due to their own ancestral connection to it, while discounting the relevance of other South Asian ethnolinguistic groups.
In response, current Indian nativists have promoted the wholly unfounded idea that South Asia was closed to human in-migration until the Islamic era, and that Indian culture and peoples are strictly indigenous in ancestry.
Both the colonial theories of a single conquering Indo European invasion and the nationalist out-of-india theory are highly oversimplified and marketed towards their respective societies' ethno-nationalist impulses.