Live data from Hacker News

Ask HN: How would you sort HN users by quality of comments?

news.ycombinator.com

11–20 of 28 posts

Re: Ask HN: How would you sort HN users by quality of comments?

#11
Something along the lines of:

Comment scores for all comments added together (with negative scores taken off) / number of comments. A bit simplistic, and prone to echo chamber effects (high rated may just mean they say things they know everyone will agree with), but it's probably the most objective setup here, given the lack of miraculous super smart AI to 'judge' quality on any deeper level.

Re: Ask HN: How would you sort HN users by quality of comments?

#13
"High quality comments" for a user in general is completely subjective. They might have "high quality comments" on C programming, and "low quality comments" on marriage equality. You'd need a custom filter for every possible topic of conversation, and even then you'd have to filter based on what you find to be "high quality".

It's easier to just filter based on popular opinions or popular people, which is what karma is for.

Re: Ask HN: How would you sort HN users by quality of comments?

#14
post #10

"Squawk to Talk" ratio Squawk = number of words in your hacker news comments Talk = number of words in children of your hacker news comments

I don't agree with this. Some people write comments that are completely false or outdated, which is followed by a series of corrections and tangential discussions.

[deleted]

Re: Ask HN: How would you sort HN users by quality of comments?

#15
post #11

Something along the lines of: Comment scores for all comments added together (with negative scores taken off) / number of comments. A bit simplistic, and prone to echo chamber effects (high rated may just mean they say things they know everyone will agree with), but it's probably the most objective setup here, given the lack of miraculous super smart AI to 'judge' quality on any deeper level.

In addition to echo chamber effects, this would strongly favor comments in popular threads. This may or may not be appropriate, depending on how you define "high-quality."

Re: Ask HN: How would you sort HN users by quality of comments?

#16
post #11

Something along the lines of: Comment scores for all comments added together (with negative scores taken off) / number of comments. A bit simplistic, and prone to echo chamber effects (high rated may just mean they say things they know everyone will agree with), but it's probably the most objective setup here, given the lack of miraculous super smart AI to 'judge' quality on any deeper level.

In addition to echo chamber effects, this would strongly favor comments in popular threads. This may or may not be appropriate, depending on how you define "high-quality."

> this would strongly favor comments in popular threads

Sum the total number of upvotes and score based on that. Track an independent score for downvotes. Display the two scores separately, in addition to the ratio between them. The MVP could be as simple as `votes_for_user_in_post/votes_in_post`. Scores would be intentionally low, you're looking for a track-record of greatness and not a once-off jackpot (or crackpot) opinion.

There could be further improvements with population distributions: if most of the comments are receiving between 2 and 10 upvotes, a comment with 20 is truly exceptional (for that post).

The problem that follows is that early comments that are mediocre will likely get more upvotes than late comments that are great. You're strangely aiming for the inverse of the HN comment ranking algorithm. StackOverflow has this problem: quick answers that are barely good enough typically receive more upvotes, and S/O hasn't solved that problem.

Re: Ask HN: How would you sort HN users by quality of comments?

#17
Whatever metric you use, since HN as a community is effectively training the metric, you'll simply capture what HN users consider as "high-quality" comments.

HN users do have their biases (in my opinion):

- Anti big company (GOOG, APPL, ... Lees so for APPL thouhg)

- Mostly left leaning

- Value political correctness over accuracy

- Anti defense and government (loathe 3 letter agencies, despite being fully cognizant of their mission statements)

I personally do not consider any comment falling squarely in the above silos to be high-quality, but you'll see them being consistently up-voted by the community.

Re: Ask HN: How would you sort HN users by quality of comments?

#19
I don't think it's possible.

A high-voted comment isn't necessarily high-quality, just really popular. I recently had a command get to 36 points [0]. I'm honestly not convinced of it's quality, even though it's my own comment. It was as a strongly worded opinion based on a high level of cynicism, not research. But people liked it, so it got a decent number of votes.

[0] https://news.ycombinator.com/item?id=19060284

Re: Ask HN: How would you sort HN users by quality of comments?

#20
Hand select comments and produce a training set. Train some model, pump every comment through it. Make a histogram of all the values you test and any user that is a statistical outlier from that dataset (towards higher values) will be added to a list of "high quality" posters.
Post reply on HN