Live data from Hacker News

Ask HN: Why and how is Hacker News so fast?

news.ycombinator.com

1–10 of 110 posts

Ask HN: Why and how is Hacker News so fast?

#1
I'm currently using an artificially-shaped 256k Internet connection (it resets in a couple days), and over the past few weeks browsing the web, I've very consistently found HN to be one of the fastest loading sites I've come across.

I understand that HN is built on top of Arc (http://arclanguage.org), a Lisp variant, and that the code (https://github.com/wting/hackernews/blob/master/news.arc) uses flat files to store post and vote data. This is generally speaking quite an unusual/unexpected architecture for a site presumed to serve a consistently moderate load, but HN manages to do so remarkably well, presumably because the site itself is so incredibly lightweight. I don't think the minimal HTML is the only factor though, although it probably counts for the majority.

I'm interested to understand as much as possible/practicable/relevant about the server(s') configuration, uplink bandwidth, other tunables, etc, so I can get an idea of what makes HN so exceptionally responsive and "a cut above 99% of everything else". Does CloudFlare really make that much of a difference? :P

I know there's some "secret sauce" in there somewhere, because virtually-verbatim clones of HN such as http://firespotting.com/ seem almost as fast... but just not as instantaneous as HN.

(PS: On the occasions I get hit by it, my ISP's shaping config seems quite involved/nuanced, and I actually want to profile it because it's so catalyzing and would be very useful to apply to my own projects for "worst-case" testing. For the interested, the details I have thus far can be found here: http://serverfault.com/questions/709529/how-can-i-profile-my-isps-bandwidth-shaping-settings)

Re: Ask HN: Why and how is Hacker News so fast?

#2
PPS: I just clicked the serverfault link (I pulled it out of my address bar history without loading it, because bandwidth) and realized it was [on hold]. I don't seem to have "gotten" the Stack Exchange thing, this happens to most of my questions, heh.

So, feel free to reply here instead of the SF question if you have a response for that.

Re: Ask HN: Why and how is Hacker News so fast?

#4
Without looking I can tell you it's two reasons:

1) It's text based and, therefore, can keep its total response small.

2) The page is cached on the backend so database lookups are kept infrequent and small.

Those two reasons alone make it faster than, say, 80% of most web sites.

Re: Ask HN: Why and how is Hacker News so fast?

#5
Loading this webpage I downloaded ~12KB. For comparison, I went to the NYT homepage and downloaded ~1.2 MB. For people with shitty downlinks like you and I, the absence of exorbitant quantities of Javascript and images makes a lot of difference - a lot more than time-to-first-byte.

Re: Ask HN: Why and how is Hacker News so fast?

#6
It's because it's only text with little to no JavaScript, and I haven't looked at the source behind HN, but presumably, the HTML for every URL is cached upon each change (comment, upvote) and requires very little processing time per user customization.

Re: Ask HN: Why and how is Hacker News so fast?

#7
I haven't done any real analysis of it, but I've always assumed it's because it renders up a single small HTML document rather than large core content and dozens of external references (ads, analysis, etc.).

Looking at this page in my browser's network analyzer, I see a request for the page itself (3.8k), followed by requests for the css and a few images which don't impact layout. On top of that, JS isn't used for layout, so the HTML renderer can do its job.

Re: Ask HN: Why and how is Hacker News so fast?

#8
post #3

Server-side is a different beast than the whole server->client part. HN does not load particularly fast for me, 300-500ms is fairly slow.

Compared to most newspapers and other sites using megabytes of Javascript, HN is blazingly fast.

Whenever I find myself on a shitty connection with high response time and (moderate) packet loss, trying to read a newspaper online is just painful, whereas HN mostly loads fine anyay.

Re: Ask HN: Why and how is Hacker News so fast?

#10
In addition to being so small (text-based, minimal JS), HN is cached by CloudFlare, which grabs copies of dynamic pages and pushes those copies out to their CDN edge servers around the globe so they're closer to end users.

When your browser asks for the IP to news.ycombinator.com, you get the address of a cloudflare CDN server. Which web server the DNS server will send you is based on a GeoIP lookup of your own IP address so that they can give you the address to the server physically closest to you, and therefore the lowest possible latency.

Edit: Apparently this is not how CF does things (see below explanation by benjojo12) but is not uncommon in load-balancing and distribution.

    news.ycombinator.com.	296	IN	CNAME	news.ycombinator.com.cdn.cloudflare.net.
    news.ycombinator.com.cdn.cloudflare.net. 299 IN	A 198.41.190.47
    news.ycombinator.com.cdn.cloudflare.net. 299 IN	A 198.41.191.47
Post reply on HN