Live data from Hacker News

Hacker News metrics (first rough approach)

mjg59.dreamwidth.org

21–30 of 72 posts

Re: Hacker News metrics (first rough approach)

#21
post #16
post #10

The Hacker News ranking algorithm tends to penalize stories that address social issues. The Hacker News ranking algorithm does not penalize stories that address social issues. It doesn't consider social issues at all, apart from a couple of special cases that we turn off once a major wave subsides [1]. The effects the OP is observing are mostly caused by user flags and the voting ring detector. 1. The intention there…

I would be more fair to address the broader point of the post rather than just one sentence. It's good that the moderator(s) specify why something is penalized[0], but it doesn't really change the overall result. [0] As defined by the post in question.

Stories about hot social and political controversies reliably attract both a lot of upvotes and a lot of user flags. This, together with how the software interprets votes and flags, explains the effects the OP is describing.

On hot social and political issues, many people find it irresistible to conclude that HN is biased against the side they favor and favors the side they oppose. They perceive this bias in all things HN: the community, the software, the mods, etc. But the truth, as far as I can tell, is that on most of these issues the community is divided. That's why the threads on them turn into flamewars.

Users reliably flag threads that turn into flamewars on HN. (By "flamewar" I mean angry arguments over predictable positions.) Moderators do so as well, though not to the same degree. But this isn't a bias against social issues per se, let alone particular social issues—the same pattern holds on threads about, say, basic income, or high-frequency trading, or even something like static vs. dynamic typing in programming. It's a matter of how divided the community is and how strongly people feel. It's more noticeable on any issue where one feels strongly.

What I do think it makes sense to call the bias of the site is a bias against the low-signal/noise discussions that inevitably result when people angrily disagree, especially when they do so through the filter of a pre-existing ideological battle. This is biased in the sense that HN's guidelines are biased: they call for certain types of stories and discussions rather than others.

Sometimes people argue that HN should abandon its anti-flamewar bias, because after all these are important issues and they need to be debated. But this fails to consider systemic effects on the site. The more ragefires burn, the more thoughtful users leave. That selective process, repelling the users HN most wants and attracting those attracted to flames, is a vicious circle that (a) is hard to get out of and (b) would turn the site into scorched earth. For all HN's flaws, it would be nihilistic to just let it burn.

That doesn't mean we aren't open to ways of balancing these things better: they're hard to balance, and if we're missing something we want to know.

Re: Hacker News metrics (first rough approach)

#22

Earlier quoted context omitted.

I'm not assuming either correlation or causation; I'm suggesting a hypothesis that seems worth testing and evaluating. There's a difference in motivation between "check this interesting article out" and "please upvote me", but the ring-voting detector can't necessarily tell the difference.

> There's a difference in motivation between "check this interesting article out" and "please upvote me", but the ring-voting detector can't necessarily tell the difference. ' In the former case, why would they be linking to HN in the first place, instead of the article itself? HN has a basic voting ring heuristic to ignore votes from direct links to HN submissions from a non-HN referrer for this reason.

I can't speak for other people, but personally, when I find a good article, I link to the source I found it through, rather than bypassing the source and linking directly to the article. (I don't do that when submitting stories to HN since HN considers that "blogspam", but personally I like finding new sources of good content, as well as sources of good discussion, rather than just the content itself. And in any case that's a heuristic for what to submit to HN rather than what to link to from elsewhere.)

At a minimum, I would tend to put something like this: interesting article (link to article) via HN (link to HN).

Though I'm certainly less likely to do that now that I know that HN does not like such links.

I don't think it's appropriate to post something like that about one's own story, but for a story you found interesting and want to share, I think it makes sense.

Re: Hacker News metrics (first rough approach)

#23
post #3

> But for now the evidence appears consistent with my innate prejudice - the Hacker News ranking algorithm tends to penalise stories that address social issues. My innate prejudice says something worse: that many stories that have to do with social justice but are oblivious to their potentially adverse effect (intentionally or not) are not penalized and are rather handsomely rewarded. These are stories that seem to b…

You may be including it in "automatically tagged as controversial", but I have the feeling that there are lots of people that are flagging those topics (which fits under "automatic" if you include reflexive flagging of social issues topics, but doesn't if you limit "automatic" to the software running the site). I pretty much don't flag anything, but I can see how that flagging could be driven more out of a feeling th…

Right, flagging is another issue. But as to "those discussions don't accomplish anything", what does any discussion on HN accomplish? On the more technical stories, the discussions often educate. Thing is, HN isn't /r/programming, and there are many stories about company culture, startups raising money, SV superstars etc.. What do the discussions about that kind of stories accomplish? It's just people schmoozing for entertainment. What's wrong with that?

But I have a practical suggestion: Instead of having unpleasant discussions about important stories related to technology, innovation and startups (like women in startups, which is a far more important topic than, say, Uber raising another trillion dollars), simply block comments on these stories, and at least let people read them without having them drop due to too many comments. Sure, people will still flag the stories, but AFAIK, the flagging privilege can be revoked if a user abuses it.

Re: Hacker News metrics (first rough approach)

#24
post #10

The Hacker News ranking algorithm tends to penalize stories that address social issues. The Hacker News ranking algorithm does not penalize stories that address social issues. It doesn't consider social issues at all, apart from a couple of special cases that we turn off once a major wave subsides [1]. The effects the OP is observing are mostly caused by user flags and the voting ring detector. 1. The intention there…

I seem to remember reading that in an attempt to avoid "controversial" subjects, posts are penalized if they generate comments too quickly. Social issue are often hotly debated and occasionally some of the most actively commented on posts on the site until they drop off the front page. So while HN might not actively be penalizing these posts, they might be penalized in indirect but still very real ways.

Re: Hacker News metrics (first rough approach)

#25
post #19

Not sure about penalising algorithms but I make sure to flag pointless social war mongering posts.

Right, because it's the posts that incite social wars rather than the real events they discuss. This is how it happens:

1. Something bad happens due to what we can call "SV culture", at least bad for some people.

2. A brave soul writes about the injustice, and posts the story (or someone else does) to HN.

3. The story is promptly flagged because the discussion of the injustice is "pointless social war mongering".

4. Order is restored.

5. Another brave soul writes about how HN reflects the SV social hegemony by ignoring important issues.

6. That story is promptly flagged.

Re: Hacker News metrics (first rough approach)

#26
post #15
post #11

Earlier quoted context omitted.

it has precisely nothing to do with the content Do certain types of content set off the ring-voting detection more frequently? The type of content likely influences how it's shared and how people respond to it, which would affect how it's voted (and/or flagged). Algorithms are not unbiased, particularly when no one's bothered to study their confounding factors.

> Do certain types of content set off the ring-voting detection more frequently? Sure—people posting their startups, for example, set off the ring detector more frequently than the typical news article. Friends voting for friends' blog posts, too. But is that a bias of the algorithm? That strikes me as a stretch.

Interesting. How do you guard against coincidental rings?

(i.e. a subgroup of people who all love, say, Lisp, and therefore always vote up Lisp stories. They're not an organised group - but they do tend to vote in concert.)

Re: Hacker News metrics (first rough approach)

#27
post #10

The Hacker News ranking algorithm tends to penalize stories that address social issues. The Hacker News ranking algorithm does not penalize stories that address social issues. It doesn't consider social issues at all, apart from a couple of special cases that we turn off once a major wave subsides [1]. The effects the OP is observing are mostly caused by user flags and the voting ring detector. 1. The intention there…

This is insanely disingenuous. User flags and the voting ring detector are part of HN's ranking algorithm, whether they're labeled as such in the code or not.

Re: Hacker News metrics (first rough approach)

#29
post #24
post #10

The Hacker News ranking algorithm tends to penalize stories that address social issues. The Hacker News ranking algorithm does not penalize stories that address social issues. It doesn't consider social issues at all, apart from a couple of special cases that we turn off once a major wave subsides [1]. The effects the OP is observing are mostly caused by user flags and the voting ring detector. 1. The intention there…

I seem to remember reading that in an attempt to avoid "controversial" subjects, posts are penalized if they generate comments too quickly. Social issue are often hotly debated and occasionally some of the most actively commented on posts on the site until they drop off the front page. So while HN might not actively be penalizing these posts, they might be penalized in indirect but still very real ways.

I would hope that HN's "flamewar" detector is somewhat more sophisticated than just "generate comments too quickly".

Re: Hacker News metrics (first rough approach)

#30
post #24
post #10

The Hacker News ranking algorithm tends to penalize stories that address social issues. The Hacker News ranking algorithm does not penalize stories that address social issues. It doesn't consider social issues at all, apart from a couple of special cases that we turn off once a major wave subsides [1]. The effects the OP is observing are mostly caused by user flags and the voting ring detector. 1. The intention there…

I seem to remember reading that in an attempt to avoid "controversial" subjects, posts are penalized if they generate comments too quickly. Social issue are often hotly debated and occasionally some of the most actively commented on posts on the site until they drop off the front page. So while HN might not actively be penalizing these posts, they might be penalized in indirect but still very real ways.

I believe a thread with more comments than votes is penalized as well, suggesting that commenting on a thread without upvoting it first is implicitly flagging it.

And of course there may be other arcane metrics involving thread depth, average comment rate, comment size, thread shape, etc. Who knows?

Post reply on HN