I love Paul Graham's articles, but isn't this Goodhart's Law but with more words? (When a measure becomes a target, it ceases to be a good measure)
The Lesson to Unlearn
121–130 of 588 posts
Re: The Lesson to Unlearn
#122I'm glad he made the leap from school tests to funding tests, as it seemed to be a thinly veiled analogy from the start. I'm disappointed, though, that he didn't take the final step and admit that money itself is also a poor, hackable test. Funding, growth, usage -- these are all still one level removed from something worthwhile or beneficial. Cigarettes have amazing usage numbers even in 2019. I'm waiting for the en…
Companies like Theranos and WeWork succeeded for a short while because they tried to "hack the test". E.g. Theranos hacked the metrics of "buzz", "investment amount", "notable people on your board", but they didn't actually have a working product. Adam Neumann certainly "hacked the test" to the tune of a billion plus dollars, but WeWork certainly didn't succeed (I'd note there is still a critical difference between Theranos and WeWork in that customers actually really like the product WeWork provides, the question is just can they provide it profitably).
I think hacking the test can get you so far in the startup world, but eventually reality rears its head and what matters is whether you can provide a product people want, profitability.
One quick note on your cigarettes example. The problem here is that companies are giving people what they want, they've just figured out how to "hack the test" so to speak with respect to what the human body wants. This is basically the case with all addictive substances - there is essentially a hackable part of the human "desire" subsystem that people have taken advantage of.
Re: The Lesson to Unlearn
#123> In theory you shouldn't have to prepare for a test in a class any more than you have to prepare for a blood test. The key here is the “in theory” part. It is very hard to design exams that actually test your understanding of some topic as opposed to your memorization skills. As somebody who recently got his bachelor’s CS degree (currently working on my master’s degree), and who also puts more emphasis on actually l…
All those domains are testable. But what is the practical work product of a deep knowledge of medieval history? It's extremely difficult to test the past, so it's certainly not predictions about what already happened. Nobody needs knowledge of medieval history for any practical purpose in the modern day. There is no possible way to test for this knowledge in a practical scenario because there is no practical outlet for the knowledge. Literary criticism is the same, along with much that is called "liberal arts" today. So for subjects like this, exams and essays are the only possible work product, and it's true that at that point you're going to have to use contrived tests, since real tests don't exist.
Re: The Lesson to Unlearn
#124Not sure I agree with this. The real world is very much about optimizing for and beating tests. It might be producing a specific CV or preparing for an interview whiteboard session to land a job. It might be socializing and networking in a specific way in order to land funding. The real world does not commonly reward just being really good at arbitrary things. It's almost always focused on meeting a need that someone…
> The real world It's not the real world - its the human world. The real world is very different, and getting more so imho. Skills to survive the human world are usually useless in the real physical world. It's a distinction I read recently from a philosopher that I lost track of, maybe someone here knows. Her point was (I think) that when someone says the real world, with relation to education, it's not really - it'…
is that a hard fact or wishful thinking?
In the context of this discussion, the rewards do not come from "the physical world" but from other humans (money) , so generic mountain-survival skills are irrelevant
Re: The Lesson to Unlearn
#125He doesn't admit any ways to actually hack market growth (as if there were none), which is weird. Seems like Use Dark Patterns, Pad Your Numbers, Fake It 'Til You Make It, Regulatory Capture, &c would all make the list.
I agree with the general idea, though: I think the least hackable tests are either Wimbledon/tennis or GSL/StarCraft tournaments; you simply cannot schmooze your way to even the lower brackets.
[0] like most things people put in 2 categories
Re: The Lesson to Unlearn
#126Not sure I agree with this. The real world is very much about optimizing for and beating tests. It might be producing a specific CV or preparing for an interview whiteboard session to land a job. It might be socializing and networking in a specific way in order to land funding. The real world does not commonly reward just being really good at arbitrary things. It's almost always focused on meeting a need that someone…
That is only true if your life ambition is securing a job with that successful tech company. If your ambition is being successful on your own, as an academic, an entrepreneur or as an artist, there are no standardized tests to beat.
Re: The Lesson to Unlearn
#127I know that learning how to hack systems is valuable because that's how I got into YCombinator. I set out specifically to hack the application process and succeeded. It wasn't even that difficult to do. If pg believes what he's saying in this essay, I have two questions: 1. why did he design a testing system that was easily hacked? 2. why does YCombinator itself focus so much on hacking the venture capital system, as…
If it were simply build the best product then there wouldn't be a need for YC. If it were simply build the best product there wouldn't be a need for marketing or sales or business degrees.
The hacks involved in marking and sales became so important to any companies success that they've over time become entire fields of their own. Same thing with an MBA - essentially a degree to hack business. Regardless of what you think of it, it's clear based on the financial returns of it that it works.
One of the biggest lessons constantly mentioned of startups is how naive engineers are to the business side of things and there is more than building a great product. In this essay PG takes all of that and throws it out the window. Is that suddenly no longer true? My read of it is that he (rightfully) likes YC and as a result doesn't see how it has the same flaws as the systems he describes. Essentially like every parent who only sees their kid as beautiful.
Re: The Lesson to Unlearn
#128Re: The Lesson to Unlearn
#129My wife and I are contemplating unschooling our kids. The longer our son is in primary school, the more we see the deterioration of his willingness to learn anything school-related. He hates homework (as do most kids), and this is even more painful to see given that it has been shown that homework is essentially useless for learning performance. He still likes to write and do math, but as long as he is in school, we…
> Btw, Gates, Page, Brin - all were not in a conventional school. This is Anecdata^3, you probably shouldn't make rash decisions about your children based on 3 successful technical entrepreneurs.
Re: The Lesson to Unlearn
#130Earlier quoted context omitted.
But the person who presented the very first paper won't have to care about the rest of the semester right? Or can you get randomly picked multiple times?
There were only 10 students allowed in the class. So you got picked about every 3rd class (sometimes more than once per class).