Not such an impossible dream.
I believe this is what they do in France. This was explained to me years ago, so i may have this totally wrong, but in France, if you get a baccalaureate at school, you have the right to attend your local university. There is no admissions process. As a result, there are far too many students. So, the universities make the first-year exams as tough as they need to, and kick out anyone who doesn't pass. Effectively, they use those exams to do admissions, just a bit late.
France also has some strange elite universities, the grandes écoles. They have selective admission. To get into those, you first do khâgne [1], a couple of years of preparatory study, followed by an entrance exam. If you squint, you could see this as letting everyone have a go at the first two years, and then only letting a few progress. To be honest, thinking about the French education system makes my head hurt.
Similar but perhaps more straightforward are the UK foundation years [2], for example, Durham's [3]:
> Foundation Programme courses are fully integrated elements of Durham University degree programmes. Students who successfully reach the progression standard by the end of the foundation year automatically gain entry to year one of their registered programme without further action or application.
The difference being that foundation years are optional - in your example, the people with advanced maths would not have needed the foundation year, but those without would have. Accordingly, they are still quite rare, because in the UK, people pick their A-levels with a view to what subject they want to study at university, and we hate social mobility.
And, of course, in the world of MOOCs and online universities, coming real soon now, it will be easy to admit millions of students to the first year of a course, as long as all the teaching in that year is highly scalable - broadcast lectures, not interactive classes; multiple-choice exams which can be marked by machine; lab work done in VR?
[1] https://en.wikipedia.org/wiki/Kh%C3%A2gne
[2] https://en.wikipedia.org/wiki/Foundation_programme
[3] https://www.dur.ac.uk/dcad/study/foundation/