As somebody who has designed some of the tests Paul Graham is complaining about, it really is hard from the other side! For example, it's long been known in the physics education research community that students come away from introductory courses with very little physical understanding, even if they can do the plug and chug problems on typical tests just fine. Students can all recite Newton's third law, but immediat…
The Lesson to Unlearn
291–300 of 588 posts
Re: The Lesson to Unlearn
#292The tech industry has unfortunately adopted the methodology of centralized hackable tests as the canonical gatekeeping method in the form of programming interviews. Most big tech companies don't care about how good you have been at delivering some value through creating software: they want to see you deliver a very specific type of performance at a whiteboard. Interviewers are given specific math puzzle questions to…
This is exactly what I said in my retweet to PG's essay. Having hackable bad tests in the very tech industry proves the point that artificial tests are still the way of thinking for many and something we collectively need to unlearn. The question is how.
Beyond that, in my opinion, nearly anyone with average programming experience can hack a typical programming/whiteboard interview, so these kinds of interviews are definitely not the best way to find the best fit for particular roles. It's a good way to filter out the inexperienced, for sure, but it's a pretty low bar to set, especially if it's a company looking for top tier engineers, people with the ability to see the bigger picture.
It's also likely that a great engineer would pass the test with ease, but because their skillset and abilities have been inaccurately assessed due to the poor interview process, the company could have initially assigned them a much more effective role where both the company and the employee could have benefited to a much greater extent, but didn't. Overall, relatively speaking, the "hackable test" approach wastes time and hurts everyone in the long run, both employers and employees. A little investment up front with more personalized, specific interviews can go a long way.
I think we should start refusing these impractical interview processes, or at the very least, from an interviewee standpoint, turn down jobs (if you can) and let the potential employer know that their interview process is the reason.
Re: The Lesson to Unlearn
#293Re: The Lesson to Unlearn
#294As somebody who has designed some of the tests Paul Graham is complaining about, it really is hard from the other side! For example, it's long been known in the physics education research community that students come away from introductory courses with very little physical understanding, even if they can do the plug and chug problems on typical tests just fine. Students can all recite Newton's third law, but immediat…
> If the people aren't motivated, any standardized measure will miss the mark Shouldn't that be a sign that trying to come up with standardized measures is a fool's errand? Short of blatant social engineering, I don't see how we can have a system that could improve people's motivation and drive for education. And if we don't have a way to actually change human nature, why do we keep trying to design this system that…
We could give more autonomy to individual schools or teachers in terms of designing tests, but then we'd be facing two problems. One, the grades given by different schools/teachers would not be directly comparable. An A in one school could be awarded only for exceptional circumstances, and in another just for above-local-average. Two, once you start using those grades to judge people in a competitive situation like the job market, you'll have schools selling grades, teachers inflating grades to help their pupils compete against those from schools that sell grades, etc. and the whole system will collapse to uselessness.
Standardized testing mitigates these problems by forcing everyone to grade by the same criteria, but in order to do that, the grading must be done on things that are extremely easy to measure - which removes both individual judgement of the teachers and any relation between testing and understanding of the material.
The more I think about it, the more I feel it all boils down to scale. The grading system needs to be scaled up with the size of the market people compete on, otherwise it'll be either trivially gamed, or deeply unfair to unlucky people.
Re: The Lesson to Unlearn
#295Re: The Lesson to Unlearn
#296Re: The Lesson to Unlearn
#297Earlier quoted context omitted.
You actually can: if he had a better level of education than everyone else, why wouldn't he do better than people taking traditional educational paths? As someone who participated in many academic competitions as a child, the winners were consistently weighted 9:1 private:public.
No, you cannot. IQ is highly correlated with wealth and is also highly heritable. So in all likelihood what you've observed is that smart kids win academic competitions, and also happen to be from rich families. If you take a bunch of kids with an IQ of 100 (or 115, or even 130) and put them into private schools they'll do a little better but they won't be winning any academic competitions.
Re: The Lesson to Unlearn
#298There's no point looking at one of their idiosyncrasies in isolation.
But then Paul Graham plays himself because all the startups are looking to get or softbanked. Absorbed into the monopolies or put in permanent luxery lift support. What a joke. Playing the VCs is more important than playing the increasingly poorer consumer hoards of the decreasing yield suburban consumer farms. We're years deep into the startup fad, and blinded test takers appealing to their blinded test taker money holders is all that's left.
Re: The Lesson to Unlearn
#299The tech industry has unfortunately adopted the methodology of centralized hackable tests as the canonical gatekeeping method in the form of programming interviews. Most big tech companies don't care about how good you have been at delivering some value through creating software: they want to see you deliver a very specific type of performance at a whiteboard. Interviewers are given specific math puzzle questions to…
Being a hiring manager at a big company, I can tell you this is just as frustrating for me as it is for candidates. I hate “leet code” and frankly find algorithmic interviews to be very low signal compared to more practical, open-ended, domain-specific problems. I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room. Suppose you give each tea…
I'm mediocre, at best, at these tests. I don't come from a traditional CS background (started as an EE). I tend to take unusual, hybrid approaches to solving problems; often incorporating elements of new-fangled tech with patterns that have been around for thirty years, and I've found that people get VERY uncomfortable with "thinking outside the box."
I also tend to take some time arriving at the final release, going through iterations of improvement. My first bash is usually fairly naive and clumsy. It works, but not so well. I make each iteration improve on it, maintaining that working software throughout; so there's always something working and testable.
Despite all the aspiration to "disruption," it seems that people don't like to step out of their comfort zones.
Making software that SHIPS is a learned and earned skill, and one that I believe, can be most effectively demonstrated with portfolios.
When a designer goes for a job, they bring with them a large black case. It's filled with their designs and working drafts. Much of the interview consists of the designer going through this portfolio with the hiring manager; discussing each example, and talking about why they did this, or why they didn't do that, etc.
No design manager in their right mind would ignore that portfolio; instead, throwing a matchbook on the interview table, and asking the applicant to "Draw Spunky."
I'm actually shocked that there is so little importance placed on software portfolios. A portfolio represents fairly substantial work product. That's why I find open-source contribution work to be so attractive.
Re: The Lesson to Unlearn
#300As somebody who has designed some of the tests Paul Graham is complaining about, it really is hard from the other side! For example, it's long been known in the physics education research community that students come away from introductory courses with very little physical understanding, even if they can do the plug and chug problems on typical tests just fine. Students can all recite Newton's third law, but immediat…
It is amusing how many of my peers still hold to the notion that heavier things fall faster. Not a higher terminal velocity, but the intuition that they accelerate faster. Hilariously tough for folks to shake.
I'd actually take the lesson the other way. It's hilariously tough for model-makers to acknowledge real-world noise that prevents their theoretically precise models from delivering reliable results.