Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small things add up: 4chan's migration to a cookieless domain

chrishateswriting.com

Post reply on HN
Print view View original

31–40 of 139 posts

Re: Small things add up: 4chan's migration to a cookieless domain

#31

Post by moot »

willvarfar wrote: post #26
moot wrote: post #12

Earlier quoted context omitted.

You're absolutely right -- it is negligible. When you're serving upwards of a petabyte per month, 23 GB isn't exactly a lot! > On the other hand, if saving single bytes was significant, there would be a lot more potential in the source by shortening CSS class names etc. rather than picking a short domain name. Also spot on, but the point I was trying to make is I was given the choice of choosing a longer domain and a…

What is your caching like, and do you use precompression? In fact, how do you serve that many pages full-stop? We (HN) need to know!

We write pages to disk as compressed HTML, and make use of nginx's gzip_static and gunzip modules to serve them. So every time a person posts, we regenerate the applicable reply and index HTML. The high postrate boards are rebuilt with a daemon on a timer, since past a certain point (~1 post per second) it's wasteful to regenerate on demand given how long the script takes to run.

We essentially think of flat files on disk as a cache for the database, and don't employ a proxy cache or any other common HTTP proxies. It's a little unorthodox, but it works well for us.

Re: Small things add up: 4chan's migration to a cookieless domain

#33

Post by callesgg »

largehotcoffee wrote: post #28

A quick look shows that you could minify the Javascript quite a bit more (which is relatively easy, and would save a lot more than 50 bytes). You might also look at inlining all of the Javascript into the HTML directly (this is what Google does), which saves the extra HTTP requests. You could also do this with some of the persistent images on the page (logo) and inline them with base64 (but I don't think it actually…

    which saves the extra HTTP requests
The reason google does it is due to them wanting a fast single page load.

4chan reloads the the page on each click meaning you save a shitload of bandwidth by not having to send the scripts and css each time the page loads.

Re: Small things add up: 4chan's migration to a cookieless domain

#34

Post by iriche »

sliverstorm wrote: post #3
quchen wrote: post #2

> 50 bytes may not seem like a lot, but when you’re serving 500 million pageviews per month, it adds up. That's a void argument in the article. If they were serving pages with 500 bytes each this would indeed be a huge improvement, but no page is 500 bytes. I just opened 4chan.org, and the markup before is already 1,836 bytes. The entire frontpage of /b/ is 114,428 bytes, and saving 50 is absolutely negligible. On th…

That's still 25GB/month of bandwidth they save. Who knows, maybe that nudges them out of a high-cost data tier.

You often pay less per GB the higher you get

Re: Small things add up: 4chan's migration to a cookieless domain

#35

Post by cheald »

gambler wrote: post #32

Maybe it's just me, but I hate how current web technologies force people to register separate domains for static content. This is not how domains are supposed to work.

They don't. You could serve your site off of www.domain.com and your CDN off of cdn.domain.com easily.

Re: Small things add up: 4chan's migration to a cookieless domain

#36

Post by gabriel34 »

I'm a bit paranoid, don't trust google and never really liked the idea of recaptcha on 4chan because of the illusion of anonymity found there. The cherry on the top is finding out uses Google analytics (mind you I haven't been there in a while. Back then I wasn't nearly this concerned with privacy)

Re: Small things add up: 4chan's migration to a cookieless domain

#37

Post by moot »

cheald wrote: post #35
gambler wrote: post #32

Maybe it's just me, but I hate how current web technologies force people to register separate domains for static content. This is not how domains are supposed to work.

They don't. You could serve your site off of www.domain.com and your CDN off of cdn.domain.com easily.

Correct, but only if you don't set any *.domain.com cookies, which as it turns out most do.

Re: Small things add up: 4chan's migration to a cookieless domain

#38

Post by bjt »

cheald wrote: post #35
gambler wrote: post #32

Maybe it's just me, but I hate how current web technologies force people to register separate domains for static content. This is not how domains are supposed to work.

They don't. You could serve your site off of www.domain.com and your CDN off of cdn.domain.com easily.

But then you still have to remain vigilant against a clueless dev or random JS lib on www.domain.com setting a cookie for .domain.com, which your browser will helpfully include with requests to cdn.domain.com. With the completely separate root you're protected from that.

http://en.wikipedia.org/wiki/HTTP_cookie#Domain_and_Path

Re: Small things add up: 4chan's migration to a cookieless domain

#39

Post by moot »

Killswitch wrote: post #24
quchen wrote: post #16

Earlier quoted context omitted.

Right, if you use a short domain name "because why not" for something new that's alright. It was just the "it adds up" part that I was commenting on. Do you have (well, you're moot, so -- want to share) some more data of 4chan's current size? I'm sure lots of people would be interested in hearing about that. (This would probably make a great individual post.)

I'd be very interested in this... I used to visit 4chan all the time back in the day, not so much now, but I do enjoy reading statistics about the site. Thanks for this article moot.

I've been meaning to write a post about all of the weird stuff we do in the interest of maximizing our limited resources. We've always had to stretch things as far as possible given server, financial, and time constraints, which has led to some interesting/unorthodox "solutions."

Re: Small things add up: 4chan's migration to a cookieless domain

#40

Post by largehotcoffee »

callesgg wrote: post #33
largehotcoffee wrote: post #28

A quick look shows that you could minify the Javascript quite a bit more (which is relatively easy, and would save a lot more than 50 bytes). You might also look at inlining all of the Javascript into the HTML directly (this is what Google does), which saves the extra HTTP requests. You could also do this with some of the persistent images on the page (logo) and inline them with base64 (but I don't think it actually…

which saves the extra HTTP requests The reason google does it is due to them wanting a fast single page load. 4chan reloads the the page on each click meaning you save a shitload of bandwidth by not having to send the scripts and css each time the page loads.

You're absolutely right, I was just sharing some other options. Given the way 4chan is currently built inlining all of the Javascript would not be beneficial over keeping it external and allowing the browser to cache it.
Post reply on HN