Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deliberate Practice and Performance: A Meta-Analysis [pdf]

scottbarrykaufman.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Deliberate Practice and Performance: A Meta-Analysis [pdf]

#11

Post by ncmiller »

This article is not really saying too much. Just that deliberate practice is not always a good predictor of performance.

This is already somewhat intuitive. Many have heard of the 10,000 hour rule, but in practice, there are so many more inter-related factors that determine the level of achievement (e.g. history, environment, passion, work ethic, "talent").

Trying to correlate a single factor (practice) with observed performance is an interesting pursuit, but ultimately doesn't say much.

Re: Deliberate Practice and Performance: A Meta-Analysis [pdf]

#12

Post by avz »

pgeorgi wrote: post #9
istjohn wrote: post #4

Earlier quoted context omitted.

That's a disappointing conclusion. I think most of us would like to take credit for our success, or if we have not been so fortunate, we hold out hope that we can be successful if we really want to and work hard enough for it. We'd like to believe that we are truly the masters of our fate. The protestant work ethic tells us that our sustained effort will be rewarded. This study suggests it may be more complicated tha…

> On the other hand, maybe we can beat ourselves up less for our failures and relax. From my point of view that would be the grand prize. It also allows for cooperation (since you don't risk working hard for someone else) and compassion (because like for you, the failing of others isn't necessarily their failure). All in all, a gentler society.

From my point of view that is a risk. It would allow complacency and underachievement (why work hard on your dreams if your achievement is already determined by your genes anyway and there are others better suited for whatever it is your dream to do).

All in all, a less prosperous and ambitious society.

But there is an even more insidious risk than that. The risk is that organizations like government agencies and companies will begin to consider your genes ahead of (or instead of) your ambition and motivation. Ultimately leading to a world not unlike that pictured in GATTACA.

Re: Deliberate Practice and Performance: A Meta-Analysis [pdf]

#13

Post by pjscott »

jfarmer wrote: post #7
istjohn wrote: post #4

Earlier quoted context omitted.

That's a disappointing conclusion. I think most of us would like to take credit for our success, or if we have not been so fortunate, we hold out hope that we can be successful if we really want to and work hard enough for it. We'd like to believe that we are truly the masters of our fate. The protestant work ethic tells us that our sustained effort will be rewarded. This study suggests it may be more complicated tha…

The paper only says the evidence available doesn't support the conclusion, i.e., we wouldn't be justified in claiming that deliberate practice is the key. This doesn't mean it's NOT the key, only that we haven't been able to measure whether it is or not so far. The only certainty is that the evidence isn't in yet. The sociological impact of the Protestant work ethic and whether it's value is supported by evidence wil…

But the evidence is in! That's what the paper is about! If deliberate practice were the Key to Success, then when you go to measure its effects, you'd expect to see a lot of the variance in outcomes explained by it. Well, lots of studies went to measure, and that's not what they found.

For claims like this, absence of evidence really is evidence of absence. (You could state this more precisely in terms of conditional probabilities, if you're bored and feel like doing some basic math.)

Re: Deliberate Practice and Performance: A Meta-Analysis [pdf]

#14

Post by avz »

pjscott wrote: post #13
jfarmer wrote: post #7

Earlier quoted context omitted.

The paper only says the evidence available doesn't support the conclusion, i.e., we wouldn't be justified in claiming that deliberate practice is the key. This doesn't mean it's NOT the key, only that we haven't been able to measure whether it is or not so far. The only certainty is that the evidence isn't in yet. The sociological impact of the Protestant work ethic and whether it's value is supported by evidence wil…

But the evidence is in! That's what the paper is about! If deliberate practice were the Key to Success, then when you go to measure its effects, you'd expect to see a lot of the variance in outcomes explained by it. Well, lots of studies went to measure, and that's not what they found. For claims like this, absence of evidence really is evidence of absence. (You could state this more precisely in terms of conditional…

Not so fast. You seem to assume that measurement in this case is simple and non-controversial. In fact measuring the amount of deliberate practice is very hard. How do you account for the difference in the types of activities individuals engaged in as children? How do you account for different levels of dedication and passion? How do you account for different amount of support from friends, family and institutions?

Re: Deliberate Practice and Performance: A Meta-Analysis [pdf]

#15

Post by pgeorgi »

avz wrote: post #12
pgeorgi wrote: post #9

Earlier quoted context omitted.

> On the other hand, maybe we can beat ourselves up less for our failures and relax. From my point of view that would be the grand prize. It also allows for cooperation (since you don't risk working hard for someone else) and compassion (because like for you, the failing of others isn't necessarily their failure). All in all, a gentler society.

From my point of view that is a risk. It would allow complacency and underachievement (why work hard on your dreams if your achievement is already determined by your genes anyway and there are others better suited for whatever it is your dream to do). All in all, a less prosperous and ambitious society. But there is an even more insidious risk than that. The risk is that organizations like government agencies and com…

> already determined by your genes

The paper doesn't draw any conclusion _what_ makes success (they refer to "genes" twice, as theories proposed by others). The claim is only that there's not enough evidence that success is based on effort.

There are lots of possible outcomes of this research as to what's the origin of success: genes, effort (just not measured correctly yet), pure chance, ...

I'm in the "pure chance" camp (though effort is obviously a necessary condition in most cases), and I don't believe it's genes because there are too many outliers.

But should it be "genes", that's actually useful knowledge: Why work hard on your dreams, if they're _indeed_ impossible to achieve? Individually that must be terribly frustrating, while globally it's a waste of resources, and will also lead to "a less prosperous and ambitious society" than what more efficient allocation could achieve. [Edit: I can propose such an outrageous system because I really don't believe that genes can do more than optimize _slightly_ in favor or against some aptitude.]

IMHO Brave New World shows the downsides of such a (gene-optimizing) society better than Gattaca, because there's no need for society to create oppressed castes for doing the menial work like in the latter. Brave New World provides a "smarter" approach.

> a less prosperous and ambitious society

I'm a huge fan of intrinsic (over extrinsic) motivation, and maybe that's why I also prefer cooperation over competition. Extrinsic motivation and competition are two sides of the same coin for me since both hunt scarce resources (where's the motivation or the competition, if all you get for your effort is already available in abundance?).

With this point of view, my assumption is that a more cooperative society isn't "less prosperous", it just achieves more balance.

> The risk is that organizations like government agencies and companies will begin to consider your genes ahead of (or instead of) your ambition and motivation.

Genes are a proxy of heritage in a way. That organizations optimize for heritage isn't exactly a new phenomenon: SAT mapping parental income more closely than aptitude; IQ tests having cultural differential, probably mapping ancestry; "who you know vs what you know"...

All of that is driven (and declared "standard") by powerful organizations that optimize for not rocking that particular boat.

Re: Deliberate Practice and Performance: A Meta-Analysis [pdf]

#16

Post by mkrump »

Is it so surprising that a single factor doesn't completely (or mostly) explain performance? At the same time the paper is not saying that deliberate practice has no effect on performance.

"What explains the variance in performance that deliberate practice does not explain? There are probably many factors. "

...

"We conclude that amount of deliberate practice although unquestionably important as a predictor of individual differences in performance from both a statistical and a practical perspective—is not as important as Ericsson and his colleagues have argued."

Re: Deliberate Practice and Performance: A Meta-Analysis [pdf]

#17

Post by istjohn »

avz wrote: post #14
pjscott wrote: post #13

Earlier quoted context omitted.

But the evidence is in! That's what the paper is about! If deliberate practice were the Key to Success, then when you go to measure its effects, you'd expect to see a lot of the variance in outcomes explained by it. Well, lots of studies went to measure, and that's not what they found. For claims like this, absence of evidence really is evidence of absence. (You could state this more precisely in terms of conditional…

Not so fast. You seem to assume that measurement in this case is simple and non-controversial. In fact measuring the amount of deliberate practice is very hard. How do you account for the difference in the types of activities individuals engaged in as children? How do you account for different levels of dedication and passion? How do you account for different amount of support from friends, family and institutions?

I'm confused. Everything you listed are things other than deliberate practice.
Post reply on HN