Live data from Hacker News

Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

zedshaw.com

101–110 of 122 posts

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#101
post #92

Earlier quoted context omitted.

Thanks for posting this. I went from despising Shaw (based on his writings) to kind of liking him.

You like someone for being a double-faced troll? IMO, I like him less now because he actually doesn't believe in what he says online.

[deleted]

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#102

Serious question: who here who has at least a BS in CS didn't have a mandatory stats class where you learned all about picking sample sizes to give you an accuracy you're happy with or that an average without a standard deviation is nigh-meaningless? It's just a fact of our profession, there is a significant percentage of people who just slap together APIs and have zero understanding of the maths behind it. I don't s…

There's a big percentage of programmers who don't understand O(n^2). On multiple occasions, I've seemed as a guru because of this one little tidbit. What if the medical profession has as much egregious and widespread ignorance of the basics as programming? Would you be in favor of certification?

I find this incredibly hard to believe. Anyone who has gone through a single intro course in CS should understand at least the difference between linear and quadratic time. Perhaps I've been in the university too long. Are these people non technical majors who picked up programming in PHP?

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#103
post #51
post #11

So after Ruby and Ruby on Rails, Zed has learnt some R and statistics. Because after reading almost all the article, he basically talks about: average, median, standard deviation. He should learn a little bit of complexity and calculability by the way, best case, worst case, Big O notation, etc.

Because when edu writes a blog post, he includes his entire life history , and, for that matter, the entire history of the universe in it. I know you've already been smacked down but this is a big pet peeve of mine and worth pointing out. You can always complain that something should have had more information in it, and since that is always true no matter what, the complaint is information free. (A value that comes f…

I'm not so sure about that. A request for more information can be incredibly important (for instance: Tell me who committed the murder! also see: stubs on Wikipedia). Essentially, there is a point of diminishing returns, and after a while, possibly negative returns as the work becomes too hard to assimilate.

However, in this case, I tend to think that Zed included just enough information to get someone who's clueless started - he even included references at the end so that if you DO want more, you can easily get it.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#104
I think this article is great once you get past the rant-iness of it. He makes a bunch of valid points and it's true: a lot people actually don't take these pretty crucial details into account when they're dealing with statistics.

It's kind of a shame that his message is diluted by its delivery. Case in point: about half of the comments here aren't even about statistics!

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#105

Serious question: who here who has at least a BS in CS didn't have a mandatory stats class where you learned all about picking sample sizes to give you an accuracy you're happy with or that an average without a standard deviation is nigh-meaningless? It's just a fact of our profession, there is a significant percentage of people who just slap together APIs and have zero understanding of the maths behind it. I don't s…

I didn't, and I'm graduating with a BS in CS from Stanford in a few weeks. We have an introductory stats requirement, but I found that the stats class I took my sophomore year for my (dropped) economics major was far more enlightening than my CS stats requirement.

I think the fundamental issue is that there's very little focus on statistical practice in most of these courses. Social scientists have it good: they're always taught how to deal with and interpret statistics in the same way they'll have to in their line of work. It's totally useless to throw a bunch of theory at students. I think that teaching statistics in the context of real problems is the only way (most) students will actually learn and come to appreciate how useful it is.

The department revamped the major this year, and they've introduced a mandatory stats class tailored to CS students: cs109.stanford.edu. I haven't looked through it at all, but I think it's a step in the right direction.

Finally, I have to give a shout-out to The Little Handbook of Statistical Practice (http://www.tufts.edu/~gdallal/LHSP.HTM) in this thread. It's an amazing resource for anyone who works with statistics. I've referenced it while doing performance testing, building an A/B testing system, and working on problem sets. From the website:

"My aim is to describe, for better or worse, what I do rather than simply present theory and methods as they appear in standard textbooks. This is about statistical practice--what happens when a statistician (me) deals with data on a daily basis."

Read it now!

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#107
post #96
post #94

Earlier quoted context omitted.

> If it's not okay to say that men are better programmers simply because of their gender (it's not), then it's not okay to say it the other way around either. God forbid that someone make an argument and try to support it with data, when one person has already decided for all of us what the correct conclusion is. WT* ever happened to debate, intellectual discourse, and the marketplace of ideas? This is one of the thi…

I think you have some interesting points. I'm going to play Devil's advocate: An assumption that you make there is that all forms of social control used for discouraging debates/speculation on certain topics are inherently bad or stem from dishonesty. It could be that knowing in advance the emotional, legal, political, or otherwise time-wasting repercussions that a certain type of debate causes justifies avoiding the…

> It could be that knowing in advance the emotional, legal, political, or otherwise time-wasting repercussions that a certain type of debate causes justifies avoiding the discussion altogether

So person A, and B and C are interested in having a debate.

Person X "knows" that persons A,B,and C (and some bystanders) would be better off if they don't even speculate or speak on the topic.

...so person X responds "People need to stop writing shit like this" ?

My objections:

* Why should I accept person X's assertion that he knows - better than I do - what will make me happy, or what will waste my time ?

* If person X truly thinks that, he should make a compelling case, put it on up a website, and respond not with "People need to stop writing shit like this", but with "I think that this debate is fruitless and time-wasting - check out this blog post for why".

* Even if person X is right, for a large percent of people, trying to shut down speech "for someone's own good" is un-American, and illiberal. If it's done under the color of law it's called "prior restraint".

* Even if person X is right given the conditions on the ground, conditions change, and over time his flawless heuristic for when to force people to "to stop writing shit like this" will become more and more disconnected from reality. What's needed is a constant feedback loop that keeps in touch with reality. ...and "ongoing debate" is the name of the ongoing feedback loop.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#108
post #107
post #96

Earlier quoted context omitted.

I think you have some interesting points. I'm going to play Devil's advocate: An assumption that you make there is that all forms of social control used for discouraging debates/speculation on certain topics are inherently bad or stem from dishonesty. It could be that knowing in advance the emotional, legal, political, or otherwise time-wasting repercussions that a certain type of debate causes justifies avoiding the…

> It could be that knowing in advance the emotional, legal, political, or otherwise time-wasting repercussions that a certain type of debate causes justifies avoiding the discussion altogether So person A, and B and C are interested in having a debate. Person X "knows" that persons A,B,and C (and some bystanders) would be better off if they don't even speculate or speak on the topic. ...so person X responds "People n…

Agreed. As Sophocles famously wrote: "Knowledge must come through action; you can have no test which is not fanciful, save by trial."

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#109
post #94

> I think women are better programmers because they have less ego and are typically more interested in the gear rather than the pissing contest. People need to stop writing shit like this. If it's not okay to say that men are better programmers simply because of their gender (it's not), then it's not okay to say it the other way around either.

> If it's not okay to say that men are better programmers simply because of their gender (it's not), then it's not okay to say it the other way around either. God forbid that someone make an argument and try to support it with data, when one person has already decided for all of us what the correct conclusion is. WT* ever happened to debate, intellectual discourse, and the marketplace of ideas? This is one of the thi…

> God forbid that someone make an argument and try to support it with data

That's well and fine, but the author stated an opinion based on anecdotal evidence. On its own though, I think it's a really stupid thing to say but I think it fits fine given the tone of his rant.

While I do agree with your central point, I don't think there's any value in exploring whether or not women might make better programmers on average. If I were to hire someone, I'd base it on their past experience and how well the interview(s) went, not their race or sex. There's no harm in speculating and even doing the research if it interests one that much though, just like there's no harm in researching whether or not painting red stripes on your car will make it go faster.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#110
post #94

> I think women are better programmers because they have less ego and are typically more interested in the gear rather than the pissing contest. People need to stop writing shit like this. If it's not okay to say that men are better programmers simply because of their gender (it's not), then it's not okay to say it the other way around either.

> If it's not okay to say that men are better programmers simply because of their gender (it's not), then it's not okay to say it the other way around either. God forbid that someone make an argument and try to support it with data, when one person has already decided for all of us what the correct conclusion is. WT* ever happened to debate, intellectual discourse, and the marketplace of ideas? This is one of the thi…

Whether the statement (that women are better programmers) is actually true, or whether it belongs to Paul Graham's set of things you can't say doesn't even matter here.

It was not presented as a nuanced statistical statement. (Ironically, in an article about how statistical statements need to be more nuanced!) It was presented as a boorish, stupid, and unsupported assertion completely irrelevant to the focus of the article.

There are a few "debates" which should be shut down using social control. Among them are the ones that attempt to box people in and remove their individuality by asserting that some subset of their humanity is the most important thing about them in some context.

You don't need to cite a blog post to do that. There also are many debates which for all intents and purposes are closed, and for which social control or even condescension are appropriate. (NO SERIOUS SCIENTIST BELIEVES THAT THE EARTH IS 6000 YEARS OLD, OR THAT MERCURY IN VACCINES COULD BE A CAUSE OF AUTISM.)

Post reply on HN