I realize there's no publicly available way to tell that yahoo.com and yimg.com are the same entity, but it would be good to at least note this as an issue with the analysis.
Some analysis of the 1M most popular sites on the web
101–110 of 130 posts
Re: Some analysis of the 1M most popular sites on the web
#102This isn't quite out yet: it's in Chrome trunk [2] and still under review in Firefox [3].
[1] https://w3c.github.io/webappsec/specs/subresourceintegrity/
[2] https://code.google.com/p/chromium/issues/detail?id=355467
Re: Some analysis of the 1M most popular sites on the web
#103Earlier quoted context omitted.
Explain to me like I'm five what features a website that hosts it's own javascript can't have versus one that loads those same javascripts from remote source?
It can't have the features that would have been built, in the time spent learning about and implementing security. I regard nearly all security for startup-class, low-user, and low-value companies to be premature optimization, which is deadly to a new project's potential.
Re: Some analysis of the 1M most popular sites on the web
#104It seems like you're marking sites down for using a cookiless domain for resources, even though that's faster and no less secure? For example, you'd mark google down for referencing gstatic.com or facebook down for referencing fbcdn.com. I realize there's no publicly available way to tell that yahoo.com and yimg.com are the same entity, but it would be good to at least note this as an issue with the analysis.
Edit: done.
Re: Some analysis of the 1M most popular sites on the web
#105Earlier quoted context omitted.
Chrome, Firefox, and Opera all support it, as do Google, Twitter, Akamai, Jetty, Apache, and several others: https://github.com/http2/http2-spec/wiki/Implementations https://en.wikipedia.org/wiki/HTTP/2 That's a big chunk of the Internet right there. IE 11 and Safari 9 both support it, so once their respective betas go public that's the rest of the client-side support. Nginx is supposed to support it by the end of th…
Only for TLS or only in beta versions. It's still going to be awhile before it's worth it to sabotage older browser performance, even once sites update their servers. And in the end, that'll just embolden sites to crust their pages with more analytics and trackers until the performance isn't any better.
However it's definitely not just beta versions. Check it: http://caniuse.com/#search=http2
If you broaden that out to http2 and its very similar predicessor spdy then the browser support graph looks even better, including the latest versions of Safari, Mobile Safari, and IE: http://caniuse.com/#feat=spdy
>82% of US traffic supports SPDY or better.
Re: Some analysis of the 1M most popular sites on the web
#106Earlier quoted context omitted.
Chrome, Firefox, and Opera all support it, as do Google, Twitter, Akamai, Jetty, Apache, and several others: https://github.com/http2/http2-spec/wiki/Implementations https://en.wikipedia.org/wiki/HTTP/2 That's a big chunk of the Internet right there. IE 11 and Safari 9 both support it, so once their respective betas go public that's the rest of the client-side support. Nginx is supposed to support it by the end of th…
Only for TLS or only in beta versions. It's still going to be awhile before it's worth it to sabotage older browser performance, even once sites update their servers. And in the end, that'll just embolden sites to crust their pages with more analytics and trackers until the performance isn't any better.
Actually no. If you advertise in your headers that you support SPDY/HTTP2, they'll use it even if they are not using encrypted http in the first request. Anyone who hasn't updated their servers to support it can't honestly claim they care a lot about performance.
> It's still going to be awhile before it's worth it to sabotage older browser performance, even once sites update their servers.
It is already worth it really, particularly when you factor in mobile where performance is a bigger.
> And in the end, that'll just embolden sites to crust their pages with more analytics and trackers until the performance isn't any better.
Well, there is a natural equilibrium that we tend to arrive at, but at least 1st party trackers are so lightweight with SPDY as to be irrelevant. If you have one 16K image somewhere on the page, the overhead of loading 100 trackers will seem negligible (JavaScript might be another matter though ;-).
Re: Some analysis of the 1M most popular sites on the web
#107Earlier quoted context omitted.
Eh, I can fit one copy of libc in L2 cache, but not 30 copies.
What if we add kernel same-page merging to the mix? Might still be a little less efficient at run time than the optimal use of shared libraries. But shared libraries make packaging more complex, especially if one does it Debian-style, with each shared library in its own package, a separate -dev package, etc.
Might be a whole lot less efficient than even sub-optimal use of shared libraries. A optimizing linker pulling together static libraries is going to make page-merging the executable almost impossible.
Re: Some analysis of the 1M most popular sites on the web
#108Likely culprits are "performance analyzers" that grade a website and report an "F" (failing) grade for not using CDN-hosted common libraries. This is a red herring: this idea that the user will already have a cached copy of CDN-hosted jQuery is bogus. Even for a common library like jQuery: the number of versions of jQuery that are in use is likely above 50, and the number of popular CDNs that host jQuery is surely ab…
Minifying your JS and CSS files is a very good practice as it's not only secure, but also is compact. Grunt is a very powerful tool that does this.
If you think you gain much in terms of compactness, you might not understand how the subsequent gzip compression works. ;-) There may well still be a gain, but it won't be significant.
Re: Some analysis of the 1M most popular sites on the web
#109I agree re not using externally-hosted Javascript. In fact I seem to remember a year ago Google Code having connectivity issues and jQuery all over the place failing to load. I was glad on that day that I always host my own jQuery. Re tracking, I ran into this embedded in some webfonts CSS a project was using (downloaded from one of those font websites): /* @import must be at top of file, otherwise CSS will not work…
Re: Some analysis of the 1M most popular sites on the web
#110some relatively serious questions on the methodology: - how did you define third party assets vs domain-managed assets? Is anything not hosted under example.com automatically third party? What about Twitter.com and t.co? I know this one is picky but would like a feel for the figures. - how deep did you scrape the (million!) sites? If it's front page or similar Inwould not be surprised to see figures revised upwards s…
I will release code + data for bootstrapping but until then here are my answers to your questions: > how did you define third party assets vs domain-managed assets? Is anything not hosted under example.com automatically third party? What about Twitter.com and t.co? I know this one is picky but would like a feel for the figures. That's based on the hosting domain being the same or a superset of the domain that the pag…
Two things leap out. Firstly I love the way you chose to do 1 million sites. I would have gone, hmm, maybe top thousand, and called it a representative sample :-) The scale of the modern world is still something I am grappling with.
Secondly, is that 200 Mbps down / 20 mbps up? I think the UK has some broadband access lessons to learn if that's true. My wet piece of string is getting threadbare.