I went to Oxford and felt much like you.
I was totally underwhelmed by the quality of the kids. I'd won a math contest with a couple of Polish kids in my team the summer before, and I thought my new classmates would be much the same: engaged, sharp. Instead I got a couple of fellows who seemed to be there for the drinking. It was quite deflating.
I did eventually round up a number of people with a better attitude, mostly from international circles. It's not like we were teetotal, just the balance of work and play seemed to fit an elite university better. Luckily I managed to forge a group of friends across colleges.
I also started to see there was no logic to who got in. It seemed to be purely a confidence game, especially for the arts subjects. I mean sure, they were probably better academically than your average person, but above some low bar it seems to be a lottery. I started meeting a lot of people who spoke confidently about things, but their confidence did not seem to be grounded in reality. There were so many of these people it gave away the game, I'm not sure I'd have noticed if I hadn't seen this concentration of BSers as a young man.
What was really missing was a love of learning. It's as if the kids had spent years gaming exams under high parental pressure, and now they were at uni as long they could get out with an OK grade they wanted to relax and explore the social world.
The tutorial system was wasted on some of my classmates. It's cringeworthy to be in a lesson with either 2 or 3 students, and one or two of the other kids hasn't done work. I also got proper sparring partners from outside my college, so I did ok eventually.