I'm interested to know how they deal with massive amounts of data and still deliver blazing fast results.
When I go to https://news.ycombinator.com/, are the threads sorted on-the-fly or pre-computed every x minutes?
I found lots of information regarding the actual algorithm, but not how it is implemented.
Does Hacker News sort trending topics on-the-fly or as a scheduled task?
1–7 of 7 posts
Re: Does Hacker News sort trending topics on-the-fly or as a scheduled task?
#2It reranks items incrementally.
I'd love to get to the point where we can sort everything on every request.
Re: Does Hacker News sort trending topics on-the-fly or as a scheduled task?
#3It reranks items incrementally. I'd love to get to the point where we can sort everything on every request.
Do you have any source for this? I'd love to know more about how they re rank items incrementally.
Do they actually update every single row in the database every 5 minutes? It seems too slow.
Re: Does Hacker News sort trending topics on-the-fly or as a scheduled task?
#4It reranks items incrementally. I'd love to get to the point where we can sort everything on every request.
Do you have any source for this? I'd love to know more about how they re rank items incrementally. Do they actually update every single row in the database every 5 minutes? It seems too slow.
Source: dang is one of the employees of Y Combinator who help run HN.
Dang--might be good to have some way to indicate that so new folks understand your special role in the community here
Re: Does Hacker News sort trending topics on-the-fly or as a scheduled task?
#5It reranks items incrementally. I'd love to get to the point where we can sort everything on every request.
Do you have any source for this? I'd love to know more about how they re rank items incrementally. Do they actually update every single row in the database every 5 minutes? It seems too slow.
HN doesn't use a database. There are no rows. Just one process on one server with everything in memory. Data is lazy loaded from flat files on-demand.
Re: Does Hacker News sort trending topics on-the-fly or as a scheduled task?
#6Earlier quoted context omitted.
Do you have any source for this? I'd love to know more about how they re rank items incrementally. Do they actually update every single row in the database every 5 minutes? It seems too slow.
HN doesn't use a database. There are no rows. Just one process on one server with everything in memory. Data is lazy loaded from flat files on-demand.
I assumed that in memory they had something structurally resembling a relational database but they might not. It would never even occur to me to not use a database for something like this.
Re: Does Hacker News sort trending topics on-the-fly or as a scheduled task?
#7Earlier quoted context omitted.
Do you have any source for this? I'd love to know more about how they re rank items incrementally. Do they actually update every single row in the database every 5 minutes? It seems too slow.
HN doesn't use a database. There are no rows. Just one process on one server with everything in memory. Data is lazy loaded from flat files on-demand.
Thank you. I did not expect that!