Stop Arguing over Elite Schools. Just Make Public Colleges Free

i'd like to refer to this discussion why learning on the job can be a problem: https://news.ycombinator.com/item?id=37718230

especially for IT which can easily be learned in a school setting unlike some other professions where learning on the job is necessary.

learning on the job in IT would work if it is a formalized apprenticeship so that trainees are motivated to stay for the full three years that this usually takes and don't have an opportunity to run away for a higher paying job after the first year.

for your last point i pretty much agree with you here: https://news.ycombinator.com/item?id=37743011

Let’s be really honest about this, most people don’t move for education.

I’m sick of people saying we can afford everything! We really, truly can’t. We have the largest peace time deficit of all time - we are running deficits the size of WW2 when no emergency exists!

The good part is that all of this will sort itself out. Economic gravity will hit and democracy will sort out who pays the bill.

> That's why I wish there were exams just like high-school ending exams.

I'm not sure exams are good at measuring actual abilities. Measuring things is an insanely difficult problem that isn't given enough credit. All too often people think it is just to perform an experiment and measure the outcome but this is really only a tractable when we have a well defined mathematical theory to test our results against. Most often in the real world this is done empirically and in these cases there are loads of assumptions that are being made and often not realized being made. Assumptions that are highly impactful. They come from the data that is being measured as well as the metrics being used to measure them. Remember, metrics are models too, and like all models are "wrong."

I'll give two points of evidence to why having a testing system is likely ineffective:

1) These elite schools already have exams that correlate strongly with who is admitted. This gives evidence that exams like SAT and ACT or even the GRE do not do a good job at filtering the issue given the downstream observation.

2) You mention India and the testing structure. Several other countries have similar structures. Personal beliefs aside[0], the fact that India and other countries do this already results in a baked in effect into the US job market. Presuming the effect that I and others are observing is real, then these too look to be ineffective metrics. Since these metrics are likely going to correlate strongly with university rankings as well.

[0] Side note, I do not think these systems are good for countries. They appear to be highly ineffective and just promote gate-keeping. They additionally create a lot of strain and pressure on the society because they result in career bottlenecks. Without these systems people have larger opportunities to change later in life. People make highly effective career changes even into their late 30's but these tests are usually performed on people under 18.

[1] Other note: meritocracies are flawed at the core due to many effects and this can be quite counterintuitive. First off is the well known Goodhart's Law, where systems optimize the cost function rather than the intended goal of the cost function. This is seen in India's famous Cobra effect, reinforcement (and other ML) learning models, bureaucracies, and so on. This is because the metric is not perfectly aligned with the intention behind the metric, which is often ill-defined and static in nature compared to the dynamic solution space. Aligning metrics is nearly impossible and has only been achieved in the most simple of cases. To learn more take any optimization class, learn about optimal transport and non-convex solutions. Metric hacking is quite common in the wild and we should be really aware of it. This even happens in the precise example we're talking about. I'm sure you've known people who are really good at taking tests but are not so good at generalizing their knowledge (using it in the real world or adapting to new situations). It is because they overfit the cost function. Remember, costs functions are models too.

So the question here is about how to properly evaluate things (distinguishing from measuring). Evaluation requires nuance and processing the data and metrics. Most importantly interpreting them (this is where bureaucracies fail and why everyone hates them. Lack of nuance). The truth is that this is always fuzzy and inexact. The only real way to perform this properly is to incorporate noise and uncertainty into your evaluation method. Not to remove it completely, but to acknowledge it and have it play a role in your model. You should of course remove as much noise as possible (reduce), but you can never reduce it to zero and if you remove it from your model you won't be able to capture your uncertainties. These problems only get harder as we move into multi-dimensional data (things we try to measure). High dimensional spaces are highly non-intuitive and the geodesics are not going to be what you expect. This is actually a relatively new area of mathematics and it would be extremely naive to think we have a good understanding of this. (not to knock on the great work that people have done in these areas, but these are fucking hard problems and we haven't had as much time)

I think part of this too is that the top universities are really good at metric hacking. I do agree that the biases are baked in, but the other aspect is the compounding effects. E.g. MIT students have a better "foot in the door" for internships, research opportunities (especially at undergrad), and so on. Not only does their name do lifting, but they get more recruiters and the universities themselves do a lot of lifting. We know many internships are provided not through blind applications but connections (often grad students get internships via their advisors). These will definitely make the resumes look far better, but don't necessarily correlate with skills.

It gets worse because this metric hacking is everywhere. As a researcher I honestly believe that were I at a higher ranked university more of my papers would be published in conferences (instead of arxiv or dead), I'd have to refine less to get published, and I'd have double the citations (university evangelism. Just being a MIT student will get you more Twitter followers than being a UT Austin student, so more people see when you post about your work). None of these things have to do with the actual quality of the work. Though of course, they can also tend to do "better" work in the sense that they have more resources. In a world where we have ill-defined goals and metrics compute becomes a large edge because instead of performing nuanced analysis you instead perform a more exhaustive analysis (the sledgehammer approach). Which is not necessarily better, though can (often) have lower entropy.

As I posted in a sibling comment, measuring and evaluating things is insanely hard. Idk why as our world is increasing in complexity we've adopted to simplifying things more and more. Maybe the double edge sword of access to knowledge is feeling more knowledgeable than we are.