I agree with you. One of the reasons why I think this happened for me in college was because I found some very accepting groups of people with whom I could unite around a common goal, in an almost pre-professional environment.
But there were a bunch of people on my floor freshman year who got sucked up into an entirely different social environment. They either pledged fraternities or sororities, or formed their own social groups that closely mimicked the ones they were used to in high school, which came with all of the expected drama.
So, advice for any younger folks: Just about every college has a student activity fair, where you can get to know all the clubs and groups and activities that are part of campus life. Try to find the ones that aren't 100% social, but maybe 25% social and 75% achieving some goal, whether it be putting on a musical or raising livestock or volunteering at a nursing home.