Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf] • Hacker News Board

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#11

Post by Xcelerate » Sun, Apr 23, 2017, 11:23 PM UTC

rosalinekarr wrote: post #4

mrlyc wrote: post #3

That article reminds me of The Auditors in Terry Pratchett's "The Thief of Time" who try to analyse a painting by reducing it to its component molecules. There is no mention of tailoring comments and replies to the target audience. Once I adopted the mindset of a senior high school or college student and posted accordingly, I was able to increase my karma on Reddit from 1,000 per year to 1,000 per month. I still have…

> I still haven't managed to crack Hacker News, though. Tough audience. Just link a bunch of studies to anything you say.[^1] It doesn't actually matter if they're related.[^2] No one really checks the sources anyway.[^3] [^1]: https://arxiv.org/abs/1502.02876 [^2]: https://arxiv.org/abs/1610.02728 [^3]: https://arxiv.org/abs/1611.03799

Haha no way. My highest rated posts have all been some vapid feel-good thing that I posted shortly after an article was submitted to HN. My detailed explanations that link to studies rarely get more than 10 points (same phenomenon occurs on Quora as well actually...)

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#12

Post by booleandilemma » Sun, Apr 23, 2017, 11:23 PM UTC

mrlyc wrote: post #3

That article reminds me of The Auditors in Terry Pratchett's "The Thief of Time" who try to analyse a painting by reducing it to its component molecules. There is no mention of tailoring comments and replies to the target audience. Once I adopted the mindset of a senior high school or college student and posted accordingly, I was able to increase my karma on Reddit from 1,000 per year to 1,000 per month. I still have…

Just post something about React Fiber.

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#13

Post by minimaxir » Sun, Apr 23, 2017, 11:31 PM UTC

derefr wrote: post #10

minimaxir wrote: post #6

Earlier quoted context omitted.

I've written many, many articles/analysis about the karma systems of Reddit/Hacker News. Karma aggregates are meaningless internet points . Neither Reddit nor Hacker News, nor their users, give major impact to how much user a user has (aside for low karma aggregate thresholds for downvoting on HN), nor is there any correlation between the amount of karma and the quality of content that user outputs.

Sure, in aggregate they're meaningless internet points. As a speaker/writer, though, you can use the score on an individual comment you've made, as a rough approximation of how well it managed to communicate its intent: if a comment you've made scores really highly, it can do well to stare at it and attempt to figure out what you did that made it work for people.

Saying that a 20pt comment is better than a 5pt comment because solely of its intrinsic quality is confirmation bias. There are many factors which help determine the score a comment receives, some of which are noted in the paper (time-of-day comment was made, time comment was made relative to the time original post was submitted, the score of the parent, etc.)

For example, this comment made today, which is literally just a quotation, is at 13pts currently: https://news.ycombinator.com/item?id=14178760

But it was a quotation in response to a top rated comment in a top rated thread.

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#14

Post by blazespin » Sun, Apr 23, 2017, 11:45 PM UTC

mrlyc wrote: post #3

That article reminds me of The Auditors in Terry Pratchett's "The Thief of Time" who try to analyse a painting by reducing it to its component molecules. There is no mention of tailoring comments and replies to the target audience. Once I adopted the mindset of a senior high school or college student and posted accordingly, I was able to increase my karma on Reddit from 1,000 per year to 1,000 per month. I still have…

Does HN even respect karma? My guess is no and is just there as a red herring(not necessarily intentionally) that keeps people from cracking the real metrics it's using.

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#15

Post by lacampbell » Sun, Apr 23, 2017, 11:46 PM UTC

I think karma mechanisms have had a net negative effect on online discussion. I don't want to have preconceived notion on how I should feel about a post - I want to make my own decisions. There's no escaping it now though, which is sad.

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#16

Post by derefr » Mon, Apr 24, 2017, 12:03 AM UTC

minimaxir wrote: post #13

derefr wrote: post #10

Earlier quoted context omitted.

Sure, in aggregate they're meaningless internet points. As a speaker/writer, though, you can use the score on an individual comment you've made, as a rough approximation of how well it managed to communicate its intent: if a comment you've made scores really highly, it can do well to stare at it and attempt to figure out what you did that made it work for people.

Saying that a 20pt comment is better than a 5pt comment because solely of its intrinsic quality is confirmation bias. There are many factors which help determine the score a comment receives, some of which are noted in the paper (time-of-day comment was made, time comment was made relative to the time original post was submitted, the score of the parent, etc.) For example, this comment made today, which is literally…

Oh, sure, a factor of 1-10x is noise.

But if you normally write comments that earn 1-5 points, and then one comment earns 300pts, there might be something to the way you wrote it. Or might not. It might just be that you wrote the rebuttal everyone wanted to the stupid comment in a very popular thread.

But you should try to figure that out, because if—after eliminating sources of error—you're still left with an anomalously-high score, then the cause was probably you.

More than just with highly-positive scores, though, I also tend to analyze the type of writing I put into comments that ended up with negative scores. More often than not, it seems that my own problem is that I didn't sprinkle in enough "hedging words" (like the 'more often than not' in this sentence) and then someone came along with a single counterexample and a downvote storm ensued. I wouldn't have figured out that people had that reaction if the points weren't there.

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#17

Post by bcherny » Mon, Apr 24, 2017, 1:05 AM UTC

lacampbell wrote: post #15

I think karma mechanisms have had a net negative effect on online discussion. I don't want to have preconceived notion on how I should feel about a post - I want to make my own decisions. There's no escaping it now though, which is sad.

Given how much content is generated on online forums, it is natural to have a mechanism to surface better/relevant content (different forums define relevance differently). Otherwise you have to wade through a sea of memes and low quality content to get to something that is worth consuming.

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#18

Post by krapp » Mon, Apr 24, 2017, 1:15 AM UTC

bcherny wrote: post #17

lacampbell wrote: post #15

I think karma mechanisms have had a net negative effect on online discussion. I don't want to have preconceived notion on how I should feel about a post - I want to make my own decisions. There's no escaping it now though, which is sad.

Given how much content is generated on online forums, it is natural to have a mechanism to surface better/relevant content (different forums define relevance differently). Otherwise you have to wade through a sea of memes and low quality content to get to something that is worth consuming.

>Otherwise you have to wade through a sea of memes and low quality content to get to something that is worth consuming.

Except now you have to deal with two definitions of quality - what you want to read and what gets voted up, and those don't always correlate.

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#19

Post by lacampbell » Mon, Apr 24, 2017, 1:23 AM UTC

bcherny wrote: post #17

lacampbell wrote: post #15

I think karma mechanisms have had a net negative effect on online discussion. I don't want to have preconceived notion on how I should feel about a post - I want to make my own decisions. There's no escaping it now though, which is sad.

Given how much content is generated on online forums, it is natural to have a mechanism to surface better/relevant content (different forums define relevance differently). Otherwise you have to wade through a sea of memes and low quality content to get to something that is worth consuming.

Given how much content is generated on online forums, it is natural to have a mechanism to surface better/relevant content (different forums define relevance differently).

My key objection is that what the majority of users think is "better/relevant" should have no influence on what I think is better and relevant. I can decide for myself, I don't need the 'hivemind'.

What's wrong with just a "deleting memes" policy?

Re: Hacking the Hivemind: Predicting Comment Karma on Internet Forums (2014) [pdf]

#20

Post by stil » Mon, Apr 24, 2017, 1:43 AM UTC

lacampbell wrote: post #19

bcherny wrote: post #17

Earlier quoted context omitted.

Given how much content is generated on online forums, it is natural to have a mechanism to surface better/relevant content (different forums define relevance differently). Otherwise you have to wade through a sea of memes and low quality content to get to something that is worth consuming.

Given how much content is generated on online forums, it is natural to have a mechanism to surface better/relevant content (different forums define relevance differently). My key objection is that what the majority of users think is "better/relevant" should have no influence on what I think is better and relevant. I can decide for myself, I don't need the 'hivemind'. What's wrong with just a "deleting memes" policy?

bcherny is saying that karma's primary function is content curation. sure, you might like a few downvoted submissions from time to time, but for most content karma scores will be saving you time sifting through a sea of content. removing content by hand requires an active and vigilant moderation team, user votes help mitigate that burden. Also, the power to 'delete' is most likely a slippery slope...