Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

zedshaw.com

Post reply on HN
Print view View original

81–90 of 122 posts

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#81

Post by colomon »

stcredzero wrote: post #68
menloparkbum wrote: post #61

Earlier quoted context omitted.

I read his sentence as that he's not Irish, but plays Irish music.

Correct. And for some reason, I am often the only one who ever gets asked about their origins, even though some of my fellow musicians are of Cajun, Hungarian, German, Mexican, and Swedish extraction. For many, Irish music is tied up in the context of some sort of romanticized Irish Nationalism, and is only appreciated as some sort of ethnic tchotchke. Their imagination fails at the notion that I'm enjoying it in a p…

I've heard more than one person say no one from North America can play proper Irish music. Always struck me as completely insane -- if nothing else, there were enough emigrant Irish musicians in NYC and Chicago to form their own styles of the music!

Liz Carroll has that story about sitting in a session in Ireland, playing tunes for hours, when someone says "Let's play some of your music!" and launches into Turkey in the Straw....

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#82

Post by philwelch »

delano wrote: post #67
biohacker42 wrote: post #27

Earlier quoted context omitted.

I wonder if this is a "feature" of women or all minorities? For example, do male nurses show the same egolessness as female programmers?

I might be misunderstanding your comment, but women are not a minority. http://www16.wolframalpha.com/input/?i=male+population+in+no...

It's bad usage, but in sociological terms, women are a minority because they aren't the dominant subgroup.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#83

Post by billswift »

nazgulnarsil wrote: post #56
tokenadult wrote: post #48

Earlier quoted context omitted.

It's caused by tilt/angle of Earth, correct? Yes. My second-grade teacher made sure her class knew this. A year or two later I learned that the earth is closer to the sun (at perihelion) when the SOUTHERN Hemisphere is having its summer, and the Northern Hemisphere is having its winter.

which is why the northern hemisphere has a slightly milder climate.

I just checked my atlas - I remembered correctly - the northern hemisphere has significantly larger temperature variation than the southern. It's probably the difference in land versus water areas more than offsetting the tiny effects of solar distance.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#84

Post by ironkeith »

GHFigs wrote: post #79
ironkeith wrote: post #28

You know, I've read a bunch of Zed's articles, and I always end up thinking that he's the source of most of his problems. There was a talk I saw him give (can't find the link) to a bunch of college kids where he basically said "phone in your job, do the stuff you love on your own time, everyone else is retarded and they'll never understand you." Great attitude you have there. I know guys like that; guys who are extre…

phone in your job I know the talk you're referring to and he did not say this. He does suggest doing your best and working really hard to make sure your employers get what they want. Do the job you're paid to do and do it well. What he advises against is going the extra mile for a company that does not trust your judgement and that you have no stake in. Leave work at work. Your job is not your life. You don't owe the…

You'll forgive my hyperbole, but I was trying to make a point.

What I'm trying to get at is the reason the people in charge of most companies are "clueless about technology" is that some technical people aren't capable of speaking to them in their language. The people in charge are going to be more concerned with how these issues affect their bottom line, if you can't communicate in those terms you'll rarely be effective.

Zed goes on forever about how his solution was technically better in every manner possible, but he was unable to convince the powers that be that he was right. This is a communication issue likely brought about by his "I'm a genius, and you aren't smart enough to understand" attitude. This isn't because business people are arrogant, pig headed assholes (though some certainly are). No, they don't get the tech, but I for one am willing to take the time to help them understand why these things matter and why they should give a shit. Yes, it sucks, but I feel that not doing so would be doing my employer a disservice. What I dislike about that talk is that Zed basically abdicates responsibility for having to communicate with non-technical people because they'll never be smart enough to understand, when it's really his failing that they don't understand in the first place.

Further, Zed's attitude towards others is beyond condescending. He may pay lip service to the people in charge and say "they're just clueless about technology", but it's so abundantly clear how little he values them, and the jobs they provide. "Working hard and doing a good job", but doing something that you know is both wrong and dumb is phoning it in. It's doing what you're supposed to because you couldn't give a shit about the company, or what becomes of it.

It just rubs me the wrong way.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#85

Post by billswift »

callmeed wrote: post #38

Can anyone recommend a decent (read "not boring") book on statistics? Something along the lines of "Naked Economics" for the stats realm ...

How to Lie with Statistics (http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/039...) is a short, enjoyable read. It doesn't tell you how to do statistics, but it gives some warning about common problems.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#86

Post by billswift »

jhancock wrote: post #25

I've read this before. Zed does not provide any details on learning or understanding statistics in this rant.

Statistics and probability is a major course of study - far more than could be usefully taught in a blog. What he tries to do is to motivate people to learn it - which is the first major step to actually learning it.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#87

Post by jhancock »

billswift wrote: post #86
jhancock wrote: post #25

I've read this before. Zed does not provide any details on learning or understanding statistics in this rant.

Statistics and probability is a major course of study - far more than could be usefully taught in a blog. What he tries to do is to motivate people to learn it - which is the first major step to actually learning it.

ahh..the old "motivation by calling people stupid" trick ;)

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#89

Post by megaduck »

randome wrote: post #73
dschobel wrote: post #15

Serious question: who here who has at least a BS in CS didn't have a mandatory stats class where you learned all about picking sample sizes to give you an accuracy you're happy with or that an average without a standard deviation is nigh-meaningless? It's just a fact of our profession, there is a significant percentage of people who just slap together APIs and have zero understanding of the maths behind it. I don't s…

it seems pretty mind boggling that people are that bad at stats ... we have a mandatory stats class for CS ... it doesn't teach some of the stuff mentioned in the article but the stuff mentioned is completely intuitive -- you shouldn't need to take a stats class to understand that. like if the variance of your result is really high and you're estimating the mean its clear that the number of samples you take is import…

You're proving his point, which is that people's intuition is often dangerously wrong. The sample size is actually often nowhere near as important as your sampling method. For an extreme example, I can conduct a billion coin flips with a weighted coin, but it's going to tell me jack about the general behavior of coin flips. Avoiding sample bias is a hard problem, far harder than most people appreciate.

To complicate matters further, "accuracy" is an extremely tricky concept in statistical analysis because error rates work very differently than they do in, say, physics. In physics, when you measure something you can be sure that your results are accurate, so long as you stay outside your instrument's range of error. In stats, your confidence interval just tells you how likely it is that your results are completely wrong, or even worse, wrong by a completely unknown amount. Every statistical inference you make has a chance of completely blowing up on you. That chance can be defined and reduced, but it can never be eliminated. There's also things like frequentist vs Bayesian statistics, where the interpretation of the same data can be completely different.

Zed may be a jerk sometimes, but on this topic he's dead right. Most programmers are far more confident about this stuff than they should be.

Re: Programmers Need To Learn Statistics Or I Will Kill Them All (2005)

#90

Post by PonyGumbo »

batasrki wrote: post #33
ironkeith wrote: post #28

You know, I've read a bunch of Zed's articles, and I always end up thinking that he's the source of most of his problems. There was a talk I saw him give (can't find the link) to a bunch of college kids where he basically said "phone in your job, do the stuff you love on your own time, everyone else is retarded and they'll never understand you." Great attitude you have there. I know guys like that; guys who are extre…

The talk is from CUSEC, probably this one: http://vimeo.com/2723800 As for his "attitude", I've actually met him and talked to him face-to-face. His internet persona is a put-on and he only talks like that to piss people off. In actuality, he's a really nice guy who is super-smart and who does not take shit from people. It's actually too bad that there aren't more of him in our industry. Scope creeps, memory leaks, e…

Thanks for posting this. I went from despising Shaw (based on his writings) to kind of liking him.
Post reply on HN