Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

ams.org

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#31

Post by waps »

tbrownaw wrote: post #16
johnbender wrote: post #14

Earlier quoted context omitted.

It's important to realize that common sense is never a replacement for a proof or emperical result. That is, common sense is not sufficient to refute the claims made by the original paper. The power of science is that it often defies reasoning and in so doing provides new insights into how things work.

But a clearly absurd or nonsensical result should be enough reason to double-check that the proof and results are actually valid. Common sense may have a somewhat low weight for evaluating how probable some scientific result is, but it doesn't have zero weight. And neither does the possibility of experimental error.

From the original paper :

Lu Hong†‡§ and Scott E. Page¶ †Michigan Business School and ¶Complex Systems, University of Michigan, Ann Arbor, MI 48109-1234; and ‡Department of Finance, Loyola University, Chicago, IL 60611

Now that "Business school" part, that's a good reason to recheck the conclusions.

Heh, I'd like to pretend that a rebuttal to business school ideas will actually change them, but we all know how this will play out.

The idea of diversity is imposed from above, and no-one will care about this. Politics is the origin of the diversity idea, nothing else.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#32

Post by jrapdx3 »

eropple wrote: post #28
dxbydt wrote: post #25

Earlier quoted context omitted.

Hey that's a great course. Pls do not avoid. Highly recommend signing up. The reason he wrote this famous paper & the book that evolved from that paper, in his own words - http://vserver1.cscs.lsa.umich.edu/~spage/thedifference_inte... He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together" Not too sure about that. At all.Especially not in math, t…

> He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together" > Not too sure about that. Why would that not be a thing? I mean, I am not the smartest person on my team. The list of things I know very little about is long . But I occasionally ask questions that are really obvious to me that reorient the discussion because people didn't think of them--p…

I'm not so sure diverse (divergent?) approaches to a problem are always better than one leader's effort.

For one thing, stand-out genius or talent shows up randomly in populations, it's not really predictable when or where it will occur. But on rare occasion when a natural leader does emerge, suppressing diversity (of problem-solving approaches) is likely the better strategy.

The true maxima of human effort have generally followed this pattern, born of singularity and not diversity. Once an innovation is widely enough known, it attracts a diverse range of followers who improve the idea and nurture its maturation. Diversity is useful for the "aftercare" of innovation, but not its creation.

The other major point is that value of diversity (however defined) vs. uniformity depends on context. In the context of one's home, there may be many ways to arrange furniture in a room, and with few exceptions, one way is as valid as another. By contrast, in a shared household over time diverse opinions are likely to converge to an arrangement using the space optimally. Diversity probably leads to a better workflow solution vs. one person's choices.

OTOH there is not a great diversity of "valid" ways to remove an inflamed appendix, that is, regardless of other considerations, a qualified surgeon is required and procedural diversity is constrained by the anatomical realities. This situation demands uniformity not diversity.

Finally, "diversity" has an indeterminate number of definitions, and applicability of the term is entirely dependent on context. On second thought, that would seem to drain it of specificity or meaning, on that ground, I should have a policy of using "diversity" carefully and sparingly.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#33

Post by Russell91 »

Am I missing something or is the author of this pdf's proof equally flawed. They assume each function is idempotent and injective ... meaning that it is the identity function. And the proof doesn't follow. A much more natural way to have "fixed" the original proof would be to require that V: X -> R be injective.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#34

Post by wisty »

yummyfajitas wrote: post #13
dxbydt wrote: post #6

Earlier quoted context omitted.

If this was html & written to the point instead of stuffed in a pdf & written up in this roundabout manner, it would be on top of HN with 100+ votes :) I read the whole pdf but here's the TL:DR; anyways - Randomization improves algorithms. This is so obvious to CS folks its taught in basic cs101 - the ones i'm familiar with are where you pick a random pivot element for quicksort, or say the one where you draw a circl…

This is in Notices of the AMS, which is about as mainstream (together with SIAM Review) as mathematical publications get. Randomization improves algorithms. This is a contentious claim. Random choices are actually very rarely the best - they are often good enough and versatile, but they are rarely the best. For example, consider Monte Carlo integration. You get O(N^{-1/2}) convergence. If you use a deterministic set…

Yeah, but having some random noise still often helps.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#35

Post by 4bpp »

Russell91 wrote: post #33

Am I missing something or is the author of this pdf's proof equally flawed. They assume each function is idempotent and injective ... meaning that it is the identity function. And the proof doesn't follow. A much more natural way to have "fixed" the original proof would be to require that V: X -> R be injective.

Assuming by function you refer to the "agents'" \Phi, where do they make the assumption that it is injective? I see nothing to the effect in the text, and the functions in the counterexample on page 6 are not injective.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#36

Post by 4bpp »

dxbydt wrote: post #6
jev wrote: post #4

Earlier quoted context omitted.

Probably because the mathematicians don't seem to have the ability to explain why it is not sound without devolving into obscure symbols and jargon.

If this was html & written to the point instead of stuffed in a pdf & written up in this roundabout manner, it would be on top of HN with 100+ votes :) I read the whole pdf but here's the TL:DR; anyways - Randomization improves algorithms. This is so obvious to CS folks its taught in basic cs101 - the ones i'm familiar with are where you pick a random pivot element for quicksort, or say the one where you draw a circl…

> Randomization improves algorithms.

The better intuition is "randomisation fixes algorithms that get stuck for superficial reasons" - if you wrote that instead of the diversity thing as the intuitive conclusion of the Hong and Page theorem, you could get away with it in a theoretical CS survey paper or introductory textbook. The set of assumptions they make about the different \Phi would be fairly reasonable, for instance, if we were talking about a number of different flawed heuristics for a search problem.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#37

Post by bobthechef »

johnbender wrote: post #14
kenjackson wrote: post #7

Earlier quoted context omitted.

It also seems to defy common sense. How many top ACM programming teams randomly pick programmers from the college? How many great bball teams are randomly picked from players? There are few activities, where excellence is easily measured, where randomly picking participants results in superior results.

It's important to realize that common sense is never a replacement for a proof or emperical result. That is, common sense is not sufficient to refute the claims made by the original paper. The power of science is that it often defies reasoning and in so doing provides new insights into how things work.

Are you being intentionally facetious? How could science defy "reasoning"? Is science some mystery cult that is shrouded in obscurantism and lack of intelligibility? Or poor choice of word perhaps? Science is most definitely about reasoning and reasoning properly.

You must realize that proof and empirical result are never a replacement for common or good sense. The very attitude that proof is self-sustaining is as silly as the idea of being able to pull yourself up by your own bootstraps. I encourage you to read Duhem's "German Science" in which he describes exactly the silliness of "science" when it is divorced from common sense. It's not magic, my friend. A man of little good sense will lack a sound place to begin reasoning from. Indeed, a man completely devoid of common sense can only begin by reasoning from arbitrary postulates. However, no amount of rigor can ever make up for the initially poorly chosen postulates. Theorems are only as true as their axioms. As Pascal once said, principles are intuited, propositions inferred.

Common sense is not infallible, of course, and certainly science can help us refine it. But common sense is the necessary starting point of all science (the ancient Greeks talk about doxa and endoxa). Without it, there would be no starting material for your empirical science or your proofs. There would be no interpreting of the results of your experiment or any understanding of what the experiment should consist of in the first place.

Of course, you must also be careful to understand that not everyone possesses a good deal of common sense even though they may appeal to it. Common sense is not the same as popular opinion.

Science isn't a magical thing that is somehow divorced from "everyday reality" or some gnostic religion that tells you how it really is without being somehow derived from experience.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#38

Post by DanBC »

kenjackson wrote: post #7
nostromo wrote: post #3

I was shocked to read the original article. I can't believe that such overarching conclusions about human nature were drawn from a little toy box computational experiment. You might as well publish a psychology paper after a weekend playing The Sims. http://vserver1.cscs.lsa.umich.edu/~spage/pnas.pdf

It also seems to defy common sense. How many top ACM programming teams randomly pick programmers from the college? How many great bball teams are randomly picked from players? There are few activities, where excellence is easily measured, where randomly picking participants results in superior results.

Moneyball shows us that all that knowledge and experience used to pick a baseball team was wrong and that people should ignore their "common sense" (which is usually just another word for "cognitive bias") and should use hard data.

In the absence of hard data just picking a team randomly froma pool of qualifying players seems to me like it would be as good as pickping players based on the team-picker's experience of the game and "cultural fit" and other bullshit.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#39

Post by Russell91 »

4bpp wrote: post #35
Russell91 wrote: post #33

Am I missing something or is the author of this pdf's proof equally flawed. They assume each function is idempotent and injective ... meaning that it is the identity function. And the proof doesn't follow. A much more natural way to have "fixed" the original proof would be to require that V: X -> R be injective.

Assuming by function you refer to the "agents'" \Phi, where do they make the assumption that it is injective? I see nothing to the effect in the text, and the functions in the counterexample on page 6 are not injective.

Oh, oops. I was reading their statement: This interpretation is incorrect without the additional hypothesis that V (x) is a one-to-one function. as - all of the phi functions must be one-to-one. Looks like they made the same assumption that I would have.

This was bothering me though so thanks for your response.

Re: Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

#40

Post by 4bpp »

The response paper seems to harp a lot on how the circumstance that typically, N_1 and N >> k, precludes any realistic applicability of the original theorem -- but it admits a fairly self-consistent and altogether much more sinister interpretation if you call the different \Phi "groups" (or "races" for maximum creepiness). The assumptions then say something to the effect of "every group's approach to problem solving produces fundamentally predictable results, and no group can be the best at solving each type of problem", which is a meme that in some form has been floating around in diversity activism circles for a long time.

The "mathematical theorem" then simply captures the triviality that under this sort of worldview, you want your team to contain Zorblaxians (sorry, SMBC) because there are some Zorblaxian problems that Zorblaxians have a natural affinity towards, and nobody without such an affinity could possibly make progress on. Since this is not stated explicitly, the statement becomes invokable even in settings in which otherwise a large number of people would raise eyebrows at the smell of exoticist quackery that the idea of "different ways of knowing" exudes.

Post reply on HN