Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacker News Analysis

blog.echen.me

Post reply on HN
Print view View original

41–50 of 62 posts

Re: Hacker News Analysis

#41

Post by palish »

eru wrote: post #40
palish wrote: post #37

Perfect example of how graphs can be subtly deceptive: 1) "Variance in number of comments" is visualized as a circle's radius. 2) ... But a circle's area increases as (pi)r^2. That's an exponential increase. A fair visualization would show a linear increase, not an exponential one. EDIT: To be more clear, the goal is "more pixels" = "more variance". Growing the visualization by a radius is inherently unfair. It's pre…

> 2) ... But a circle's area increases as (pi)r^2. That's an exponential increase. Oh, it's an exponential increase in 2, but a linear increase in Pi. Anyway, just pretend that the area of the circle is proportional to the variance, and the radius to the standard deviation.

I assume "variance of Foo" is linear quantity.

Is it true that a variance of 4.0 means "twice as much as 2.0"?

If so, then the circle's area (not its radius) must grow proportional to that variance. Growing the radius is an unfair visualization.

The goal is: more pixels = more variance.

Re: Hacker News Analysis

#42

Post by andreaja »

Joakal wrote: post #34
jamesbritt wrote: post #26

Earlier quoted context omitted.

Your last suggestion seems to make things less democratic understandably, but how would it be implemented? Weighted votes based on karma. Or restrict the number of votes a user can cast based on karma or account age. A problem with these kinds of things is determining what does or does not work and picking up on unintended side-effects, and doing so in a timely manner. For example, if "young" accounts have very limit…

I suggest similar to Reddit's anti-spam mechanism, fuzz the voting but give the impression that karma is somewhat important.

Maybe all we need is 'bad, average, good, great' to be shown alongside the comment. Or something along the lines of what slashdot do with allowing you to vote for comments as 'insightful' or 'funny' etc.

Re: Hacker News Analysis

#44

Post by dtby »

bermanoid wrote: post #21
tptacek wrote: post #7

You're working from a different definition of "good comment" than Graham is. The concern with "bad" comments isn't simply that they exist, but that they have deceptively positive scores.

...which is a problem with a crystal clear cause: if you allow purely democratic voting, then you'll end up with really popular crap reaching the top of the pile fairly often. Imagine how useless Pagerank would be as an algorithm if it considered links from every drooler with a domain name to count for the same amount as those coming from the frontpage of Yahoo - that's essentially what you're doing when you tally up…

Also, in case it was unclear to any still reading, Democracy is for suckers.

Re: Hacker News Analysis

#45

Post by eru »

palish wrote: post #41
eru wrote: post #40

Earlier quoted context omitted.

> 2) ... But a circle's area increases as (pi)r^2. That's an exponential increase. Oh, it's an exponential increase in 2, but a linear increase in Pi. Anyway, just pretend that the area of the circle is proportional to the variance, and the radius to the standard deviation.

I assume "variance of Foo" is linear quantity. Is it true that a variance of 4.0 means "twice as much as 2.0"? If so, then the circle's area ( not its radius) must grow proportional to that variance. Growing the radius is an unfair visualization. The goal is: more pixels = more variance.

Standard deviation is the square root of variance. A variance of 4 is twice the variance as one of 2, but that's just by definition. What is perhaps more interesting for your intuition: Adding two independent variables, the variance of the result will be the sum of the variances. Thus the standard deviation of the sum is the squareroot of the sum of squares of the standard deviations of the components. (It's easier to read in formulas.)

A look at http://en.wikipedia.org/wiki/Variance might help.

Re: Hacker News Analysis

#46

Post by eru »

ugh wrote: post #18
topomorph wrote: post #6

Earlier quoted context omitted.

Yeah, that measure was super hacky and I have no justification for it (but it was the simplest/only thing I could think of at the time). Do you have any suggestions? One thing I'm thinking of now is seeing how (# points at the xth percentile) / (# points of 10th highest rated post) changes over time. (Or maybe I should just take a closer look at how the overall distribution of points changes to get a better feel firs…

There is no way to get a handle on comment and submission quality without content analysis. You need a list of properties that define good comments and good submissions, you need some way to quantify those properties and then you need to invest obscene amounts of work coding those properties. All of this is hard work and full of uncertainty. Different people want HN to be different things, there can’t be one list of…

> This is a Ph.D. thesis worth of work. I wouldn’t try to do it as a hobby.

Why not? They won't hurt themselves, and it will be fun. Maybe they will actually learn something.

Re: Hacker News Analysis

#47

Post by palish »

eru wrote: post #45
palish wrote: post #41

Earlier quoted context omitted.

I assume "variance of Foo" is linear quantity. Is it true that a variance of 4.0 means "twice as much as 2.0"? If so, then the circle's area ( not its radius) must grow proportional to that variance. Growing the radius is an unfair visualization. The goal is: more pixels = more variance.

Standard deviation is the square root of variance. A variance of 4 is twice the variance as one of 2, but that's just by definition. What is perhaps more interesting for your intuition: Adding two independent variables, the variance of the result will be the sum of the variances. Thus the standard deviation of the sum is the squareroot of the sum of squares of the standard deviations of the components. (It's easier t…

Thank you! I really appreciate your explanation.

By those definitions, shouldn't the number of pixels (circle area) reflect the deviation, rather than what it currently does (the square deviation, aka variance)?

In that case, the area should grow proportionally to the deviation, not to the variance.

Re: Hacker News Analysis

#48

Post by eru »

palish wrote: post #47
eru wrote: post #45

Earlier quoted context omitted.

Standard deviation is the square root of variance. A variance of 4 is twice the variance as one of 2, but that's just by definition. What is perhaps more interesting for your intuition: Adding two independent variables, the variance of the result will be the sum of the variances. Thus the standard deviation of the sum is the squareroot of the sum of squares of the standard deviations of the components. (It's easier t…

Thank you! I really appreciate your explanation. By those definitions, shouldn't the number of pixels (circle area) reflect the deviation , rather than what it currently does (the square deviation, aka variance)? In that case, the area should grow proportionally to the deviation, not to the variance.

You're welcome.

I don't know what it should do. Showing variance is fine, and my professor for probability theory actually preferred variance over standard deviation.

Re: Hacker News Analysis

#49

Post by bullseye »

jamesbritt wrote: post #26
Joakal wrote: post #22

Earlier quoted context omitted.

Your last suggestion seems to make things less democratic understandably, but how would it be implemented?

Your last suggestion seems to make things less democratic understandably, but how would it be implemented? Weighted votes based on karma. Or restrict the number of votes a user can cast based on karma or account age. A problem with these kinds of things is determining what does or does not work and picking up on unintended side-effects, and doing so in a timely manner. For example, if "young" accounts have very limit…

>> if "young" accounts have very limited votes available will new users be less likely to stick around long enough to become more active voters?

I suppose that depends on whether or not they are on the site to read submitted news articles and comments or to play the voting and karma game. I rarely vote (or comment for that matter), but I get value from the content. For that reason, I have continued to consistently visit the site for a couple of years.

Re: Hacker News Analysis

#50

Post by jbooth »

ignifero wrote: post #32
alanfalcon wrote: post #28

Earlier quoted context omitted.

I like the idea of upvotes and downvotes being multiplied by your avg score. tptacek's votes would be worth 8.85 at the moment, bermanoid's 3.5, and mine 2.92. Newbies would come in at 1.

That would be a violation of the democratic principle behind social news and would incentivize people to game their scores. Members have different interests and expertise, so having a high avg score doesn't mean they are experts in everything (Abuse of power - OMG this is getting more intricate than i thought)

You could take the log of any score > 1 to reduce the incentive to game the system.. that way someone who relentlessly karma whores won't have that much more influence than a mildly uprated commenter, but both will have more influence than a noob.
Post reply on HN