I went back to uni after a 15 year career writing software. I attended the all the office hours for all my classes. Instructors went from shocked to excited, and I learned so much!
I think 18-24 is just way too young for most people to study at a university (at least math/physics-based ones). I don't have a better solution, but only about 6-7 years after graduating I realized I would then have enjoyed these classes and learned from them much much more.
The only reason I wouldn't recommend this approach is that working full time and going to school was brutal.