Show HN: Quality News – Towards a fairer ranking algorithm for Hacker News

TLDR;

- Quality News is a Hacker News client that provides additional data and insights on submissions, notably, the upvoteRate metric.

- We propose that this metric could be used to improve the Hacker News ranking score.

- In-depth explanation: https://github.com/social-protocols/news#readme

The Hacker News ranking score is directly proportional to upvotes, which is a problem because it creates a feedback loop: higher rank leads to more upvotes leads to higher rank, and so on...

                →
             ↗     ↘
    Higher Rank   More Upvotes
             ↖     ↙
                ←

Our proposed solution is to use upvoteRate instead of upvotes in the ranking formula. upvoteRate is an estimate of how much more or less likely users are to upvote a story compared to the average story, taking account how much attention the story as received, based on a history of the ranks and times at which it has been shown. You can read about how we calculate this metric in more detail here: https://github.com/social-protocols/news#readme

About 1.5 years ago, we published an article with this basic idea of counteracting the rank-upvotes feedback loop by using attention as negative feedback. We received very valuable input from the HN community (https://news.ycombinator.com/item?id=28391659). Quality News has been created based largely on this feedback.

Currently, Quality News shows the upvoteRate metric for live Hacker News data, as well as charts of the rank and upvote history of each story. We have not yet implemented an alternative ranking algorithm, because we don't have access to data on flags and moderator actions, which are a major component of the HN ranking score.

We'd love to see the Hacker News team experiment with the new formula, perhaps on an alternative front page. This will allow the community to evaluate whether the new ranking formula is an improvement over the current one.

We look forward discussing our approach with you!

Links:

Site: https://news.social-protocols.org/

Readme: https://github.com/social-protocols/news#readme

Previous Blog Post: https://felx.me/2021/08/29/improving-the-hacker-news-ranking...

Previous Discussion: https://news.ycombinator.com/item?id=28391659

Show HN: Quality News – Towards a fairer ranking algorithm for Hacker News
news.social-protocols.org

This is 2023 and text classification problems that I struggled with at a startup 5 years ago are now easy and the power of transformer models is obscured by the ChatGPT hype. It is time that we turn our back in the collaborative filtering algorithms that made social media a hellscape and embrace content-based filtering.

I have a model that predicts if an article will front page or get a high ratio of comments/votes. It has a terrible ROCAUC because it is such a fuzzy problem but it is well calibrated and just today my RSS reader told me a story I thought was a nothingburger would succeed on both metrics and… It did!

I did make an attempt to take into account the factors you’re concerned about and I was surprised that the AUC didn’t go up. Probably I did it wrong though.

Look up my profile, I’d love to chat about it.

I was originally going to joke that maybe you should turn your script on to the stock market, but I'm guessing with your background you may have some experience in that regard!

One difference is that upvoteRate formula adjusts for where the submission appears on the page (the rank). It also adjusts for how many site-wide upvotes occurred during that time period.

You are right, since we don't know the number of users who saw the page + the vote icon, so we can't calculate the probability Pr(upvote|saw submission with upvote button). But the upvoteRate formula would be proportional to this probability, times additional factors for rank and time.

We talked about this in our original blog article here: https://felx.me/2021/08/29/improving-the-hacker-news-ranking...