System loads web pages 34 percent faster by fetching files more effectively • Hacker News Board

Re: System loads web pages 34 percent faster by fetching files more effectively

#101

Post by leeoniya » Mon, Dec 19, 2016, 5:01 PM UTC

supergreg wrote: post #82

leeoniya wrote: post #30

Earlier quoted context omitted.

this has been true in my own testing (with plain SSL/TLS, not even SPDY/HTTP2/QUIC), but you can read another analysis that reaches the same conclusion here: https://thethemefoundry.com/blog/why-we-dont-use-a-cdn-spdy-... another important reason i personally don't use CDNs is the privacy of my users.

>another important reason i personally don't use CDNs is the privacy of my users. How high up the privacy concern is using a CDN vs having cloudflare in front of you or everything being in AWS or using a browser by an ad company?

> having cloudflare in front of you or everything being in AWS

sure, sometimes it's unavoidable when there is real value added in using these platforms. at the end of the day most of us have to entrust our infrastructure to some hosting provider. I trust a provider I pay (with a known privacy policy) more than a public CDN (with a privacy policy that allows them to make money in unspecified ways)

but yes, centralized ssl termination for distributed systems is an issue that's difficult to mitigate WRT privacy. maybe via https://en.wikipedia.org/wiki/Multipath_TCP ? i don't know enough here.

> using a browser by an ad company

this is up to the user

Re: System loads web pages 34 percent faster by fetching files more effectively

#102

Post by leeoniya » Mon, Dec 19, 2016, 5:16 PM UTC

manarth wrote: post #92

leeoniya wrote: post #68

Earlier quoted context omitted.

the bandwidth isnt the problem. your browser still has to make the https connection to the cdn and request the file with a Last-Modified or Etag header so the server can return a 304 Not Modified response. This is the real cost, not the download size itself. I'm not sure how often browsers choose to do this, but if you refresh a page, they all will.

In general, browsers shouldn't even revalidate the file as long as it's younger than the max-age lifespan. They may choose to "evict" it from the cache, but that's generally based on a policy of recent usage and needing to free up/prioritise disk space for more recent assets. I'm now interested to see how different browsers handle refresh/force-refresh - whether they revalidate a file that's still considered fresh, a…

last time i tested this was a couple years ago.

i could not find a reliable way to make the 304 requests disappear using any available cache-related headers :/

Re: System loads web pages 34 percent faster by fetching files more effectively

#103

Post by saurik » Mon, Dec 19, 2016, 7:05 PM UTC

Thiez wrote: post #66

saurik wrote: post #11

Earlier quoted context omitted.

This still smells NP Hard. I mean, in practice for simple dependencies it is probably quite tractable, but this is a combinatorial optimization problem that seems pretty similar to an online modification to Job Shop Scheduling, where the material requirements map loosely to machine-task pairings that would be unblocked by orders, acting to make the problem more complex, not easier.

Your sense of smell is off. Assuming a directed acyclic graph (the usual shape of a dependency tree), assume we write the result order to a list L. Walk over all vertices once, create a mapping M of each vertex to its number of incoming edges, and add all vertices without incoming edges to a list R. This is O(|V| + |E|). Now, pick the first item of R and append it to L. For each outgoing edge in the item we chose, de…

Yes, I know that topological sort is O(|V| + |E|). What I am claiming is that the problem of buying building materials in an optimal order isn't topological sort: there are numerous topological orders of a graph, but some will be much much slower than others to build at your construction site. To determine which of the many possible orders is the fastest one, you have to take into account how long various tasks will take and how many workers you have available for those tasks. When you get some materials, that unlocks certain parts of the gantt chart of what sounds like Job Shop Scheduling. To me, this is the same form of complaint as pointing out that Traveling Salesman isn't Topological Sort.

Re: System loads web pages 34 percent faster by fetching files more effectively

#104

Post by Pete_D » Mon, Dec 19, 2016, 7:08 PM UTC

amelius wrote: post #64

leeoniya wrote: post #19

but we can already make web pages load 500% faster by not shoveling a ton of shit, not loading scripts from 60 third-party domains (yes stop using CDNs for jQuery/js libs, those https connections aren't free - they're much more expensive than just serving the same script from your existing connection), reducing total requests to 1.22MB embedded youtube players [1], 500kb of other js bloat, 200kb of webfonts, 150kb of…

Actually I don't even read the articles anymore when on mobile. I just use HN, and hope somebody posts a TL;DR, or some relevant comment that gives some more information about the article. Only if this is not the case will I consider clicking on the article link. It's pretty sad actually. I secretly wish there was some way that allows us (as a community) to collaboratively "pirate" articles, perhaps as a torrent (IPF…

I usually only see lynx and friends mentioned as jokes, but 'elinks --dump | less' really can be a huge readability improvement for some articles.

Re: System loads web pages 34 percent faster by fetching files more effectively

#105

Post by Eridrus » Tue, Dec 20, 2016, 6:52 PM UTC

richmarr wrote: post #52

bartread wrote: post #36

Earlier quoted context omitted.

Using a CDN for common libraries may help, but doesn't always, and is something you should measure rather than just assume. The situation where a CDN actually hurts performance is one I've seen periodically at different clients. When people talk about serving jQuery, or J. Random JavaScript library, from a CDN it means the specific version of jQuery (or whatever) that they're using. There's literally no guarantee tha…

> ... something you should measure rather than just assume How do people do this? My obstacle with this point is each CDN would have a different impact in each locale due to the locations of their points of presence, and CDN-ing each resource would a different impact based on the sites that particuar individual had visited. Measuring it in any useful way in advance of a change would be really hard unless I'm missing…

You can use JavaScript timers to measure how long things take to load and infer how many of the resources come from cache, etc.