Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small things add up: 4chan's migration to a cookieless domain

chrishateswriting.com

Post reply on HN
Print view View original

41–50 of 139 posts

Re: Small things add up: 4chan's migration to a cookieless domain

#41

Post by moot »

lnanek2 wrote: post #29
largehotcoffee wrote: post #28

A quick look shows that you could minify the Javascript quite a bit more (which is relatively easy, and would save a lot more than 50 bytes). You might also look at inlining all of the Javascript into the HTML directly (this is what Google does), which saves the extra HTTP requests. You could also do this with some of the persistent images on the page (logo) and inline them with base64 (but I don't think it actually…

Each page of the site isn't all that different from the rest. They could just load the page once and update the page content with a JS AJAX call and DOM manipulation, honestly. You wouldn't even need to send full URLs for the images that way, just the unique part of the URL such that JS could reconstruct it from a constant prefix.

We actually do this with reply loading already, but not by default. We have a read-only JSON API and use that to append new replies when you're browsing in a thread (click [Update] or [x Auto] at the top/bottom of a thread). We also have a de-pagination feature that grabs all of the OPs from a board and let's you scroll through the indexes as one giant page.

Re: Small things add up: 4chan's migration to a cookieless domain

#42

Post by icebraining »

psykovsky wrote: post #23
moot wrote: post #8

Earlier quoted context omitted.

I think you're confusing the two benefits. What the thread OP is talking about is the decrease in page size from the URLs included in the page source. Choosing a shorter URL for the static domain versus a longer one resulted in a rough savings of 50 bytes per page (size), compressed. You're referring to request cookie size, which was also decreased significantly (CloudFlare still sets a single cookie, unfortunately),…

Kind of off-topic, but how do you feel about doing a conscious MITM to your users(by using Cloudflare), being 4chan home of Anonymous and all that?

4chan logs IPs and provides them to the cops if the ask, so it's not like users who want to remain completely anonymous can do so without being behind seven proxies anyway.

Re: Small things add up: 4chan's migration to a cookieless domain

#43

Post by gokhan »

tomp wrote: post #14

> If you’ve been linked directly to a Facebook photo, you may have noticed the domain wasn’t facebook.com, but instead something like fbcdn-x-x.akamaihd.net. Large sites load static content from special domains for a few reasons, but primarily to reduce request overhead, and sometimes security. This works for 4chan, but in case of Facebook, it actually reduces security - since cookies cannot be checked for photos on…

If you give the url of the photo you are allowed to see to someone else, you can give the actual photo as well. I don't get the security risk here.

Re: Small things add up: 4chan's migration to a cookieless domain

#44

Post by moot »

largehotcoffee wrote: post #28

A quick look shows that you could minify the Javascript quite a bit more (which is relatively easy, and would save a lot more than 50 bytes). You might also look at inlining all of the Javascript into the HTML directly (this is what Google does), which saves the extra HTTP requests. You could also do this with some of the persistent images on the page (logo) and inline them with base64 (but I don't think it actually…

Thanks for pointing that out. We minify production JS using Closure Compiler but sometimes that leaves room for improvement.

There's definitely a tradeoff between inlining JS and small images, but I think in our case it makes more sense to leave them external to leverage browser (and since we use a CDN for static content -- edge) caching.

I believe I tried to get ngx_pagespeed up and running when it was announced, but couldn't get it to compile from source. Sometimes (read: often) it sucks to be a FreeBSD user.

Re: Small things add up: 4chan's migration to a cookieless domain

#46

Post by Kudos »

moot wrote: post #44
largehotcoffee wrote: post #28

A quick look shows that you could minify the Javascript quite a bit more (which is relatively easy, and would save a lot more than 50 bytes). You might also look at inlining all of the Javascript into the HTML directly (this is what Google does), which saves the extra HTTP requests. You could also do this with some of the persistent images on the page (logo) and inline them with base64 (but I don't think it actually…

Thanks for pointing that out. We minify production JS using Closure Compiler but sometimes that leaves room for improvement. There's definitely a tradeoff between inlining JS and small images, but I think in our case it makes more sense to leave them external to leverage browser (and since we use a CDN for static content -- edge) caching. I believe I tried to get ngx_pagespeed up and running when it was announced, bu…

I've found that uglifyjs does about as good a job as Closure Compiler, but waaay faster. Could be a nice speed up in deploy times for you.

Re: Small things add up: 4chan's migration to a cookieless domain

#47

Post by dangayle »

moot wrote: post #31
willvarfar wrote: post #26

Earlier quoted context omitted.

What is your caching like, and do you use precompression? In fact, how do you serve that many pages full-stop? We (HN) need to know!

We write pages to disk as compressed HTML, and make use of nginx's gzip_static and gunzip modules to serve them. So every time a person posts, we regenerate the applicable reply and index HTML. The high postrate boards are rebuilt with a daemon on a timer, since past a certain point (~1 post per second) it's wasteful to regenerate on demand given how long the script takes to run. We essentially think of flat files on…

Flat files on a disk == awesome

Re: Small things add up: 4chan's migration to a cookieless domain

#48

Post by lucb1e »

quchen wrote: post #2

> 50 bytes may not seem like a lot, but when you’re serving 500 million pageviews per month, it adds up. That's a void argument in the article. If they were serving pages with 500 bytes each this would indeed be a huge improvement, but no page is 500 bytes. I just opened 4chan.org, and the markup before is already 1,836 bytes. The entire frontpage of /b/ is 114,428 bytes, and saving 50 is absolutely negligible. On th…

> > 50 bytes may not seem like a lot, but when you’re serving 500 million pageviews per month, it adds up.

I thought you were quoting me here, I wrote the same text yesterday only with a figure of 100 million pageviews. That was weird to see.

Re: Small things add up: 4chan's migration to a cookieless domain

#49

Post by mapgrep »

I'm very curious why an anonymous site that doesn't even allow registered users gets 100k worth of cookies on a typical connection.

Not saying you're doing anything wrong, just curious. I assume some of it is for ad tracking, but that's still a hell of a lot of data!

Re: Small things add up: 4chan's migration to a cookieless domain

#50

Post by moot »

dangayle wrote: post #47
moot wrote: post #31

Earlier quoted context omitted.

We write pages to disk as compressed HTML, and make use of nginx's gzip_static and gunzip modules to serve them. So every time a person posts, we regenerate the applicable reply and index HTML. The high postrate boards are rebuilt with a daemon on a timer, since past a certain point (~1 post per second) it's wasteful to regenerate on demand given how long the script takes to run. We essentially think of flat files on…

Flat files on a disk == awesome

We've had a surprising amount of difficulty maintaining this over the years, since FreeBSD's NFSv4 implementation kind of sucked for a while, and it doesn't support mounting a memory partition (tmpfs). But you can trick it by mounting the memory partition using nullfs, and then nfsmount-ing that.

We used that memory-partition-over-network thing for a while, but actually switched to SSD-over-network because it was faster than the memory partition. I spoke with a FreeBSD maintainer about it and he said what we were doing was so unsupported/unoptimized that he wasn't surprised.

We run into weird FreeBSD edge cases pretty often where there are few people, if anyone, who can answer our questions. Sometimes I wish we'd gone with Linux, but after ten years the hassle of switching doesn't seem worth it. Thankfully 9.2-RELEASE has been pretty good to us.

Post reply on HN