Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some analysis of the 1M most popular sites on the web

jacquesmattheij.com

Post reply on HN
Print view View original

31–40 of 130 posts

Re: Some analysis of the 1M most popular sites on the web

#31

Post by anantzoid »

jwr wrote: post #13

Likely culprits are "performance analyzers" that grade a website and report an "F" (failing) grade for not using CDN-hosted common libraries. This is a red herring: this idea that the user will already have a cached copy of CDN-hosted jQuery is bogus. Even for a common library like jQuery: the number of versions of jQuery that are in use is likely above 50, and the number of popular CDNs that host jQuery is surely ab…

Minifying your JS and CSS files is a very good practice as it's not only secure, but also is compact. Grunt is a very powerful tool that does this.

Re: Some analysis of the 1M most popular sites on the web

#32

Post by jacquesm »

theg2 wrote: post #22

We offload a ton of our scripts to S3 buckets on random unrelated domains and it's a pretty common practice. Did this take that into account?

No, it did not. It would have to tie in the whois data to make that match (and even then it might not). The analysis is URL based, but I don't think changing that to account for those sites that use random domains to store chunks of their site would make a huge difference, but it's a valid criticism.

Re: Some analysis of the 1M most popular sites on the web

#33

Post by rhblake »

> The request for the code contains a referring url which tells the entity hosting the script who is visiting your pages and which pages they are visiting (this goes for all externally hosted content (fonts, images etc), not just javascript)

This can now be mitigated thanks to Referrer Policy [0]:

"The simplest policy is No Referrer, which specifies that no referrer information is to be sent along with requests made from a particular settings object to any origin. The header will be omitted entirely."

Voilà:
It's a W3C draft, but it's supported by latest FF/Chrome/Safari, and Microsoft Edge [1], although currently, with Edge, you'll want to use the legacy keyword "never" instead. (AFAIK "never" works with all the aforementioned browsers.)

> Google analytics junkies in particular will have to weigh whether they feel their users privacy is more important to them than their ability to analyze their users movements on the site.

There's a nice alternative - Piwik [2]. It's very much like GA, but GPL and self-hosted, and with various options for privacy [3]. You can even use it without cookies, if you don't mind the somewhat reduced accuracy and functionality.

Regarding fonts from Google Fonts, it's super-easy to host them yourself. There's a nice bash script [4] that downloads the font you want in all its formats/weights and generates the proper CSS. There's also the google-webfonts-helper service [5], and Font Squirrel has a webfont generator [6].

[0] https://w3c.github.io/webappsec/specs/referrer-policy/

[1] https://msdn.microsoft.com/en-us/library/dn904194%28v=vs.85%...

[2] https://piwik.org/

[3] https://piwik.org/docs/privacy/

[4] https://github.com/neverpanic/google-font-download

[5] https://github.com/majodev/google-webfonts-helper

[6] http://www.fontsquirrel.com/tools/webfont-generator

Re: Some analysis of the 1M most popular sites on the web

#34

Post by jacquesm »

aw3c2 wrote: post #30

> Flash seems to be very rapidly on the way out, less than 1% of the domains I looked at still contained flash content What exactly did you look at? Homepages?

Yes, homepages and all the content subsequently loaded (directly or indirectly through multiple layers of scripting or iframes). Essentially what you'd get if you were to visit each and every homepage on the top list and logged the urls that were loaded as a consequence of that.

Re: Some analysis of the 1M most popular sites on the web

#35

Post by lifeisstillgood »

So, as someone who has never really bothered with blockers of any sort, what would be the ideal blocker to install / write? (I am thinking iOS as sadly that is my primary medium these days)

- able to prevent download of any third party hosted assets - able to hash the above assets and allow user to approve their use (ie can approve jquery v1.5 from cdn.google.com) - is this whitelist approach going to work? Does ghostery or similar already do this?

I vastly prefer a whitelist approach - but if 2/3 of the web will break I am at a loss ...

Re: Some analysis of the 1M most popular sites on the web

#36

Post by pradn »

jwr wrote: post #13

Likely culprits are "performance analyzers" that grade a website and report an "F" (failing) grade for not using CDN-hosted common libraries. This is a red herring: this idea that the user will already have a cached copy of CDN-hosted jQuery is bogus. Even for a common library like jQuery: the number of versions of jQuery that are in use is likely above 50, and the number of popular CDNs that host jQuery is surely ab…

Can you please elaborate how using share libraries is "pretty much pointless and incurs a performance penalty"? That goes against my intuition of how they work.

Re: Some analysis of the 1M most popular sites on the web

#37

Post by jacquesm »

gingerlime wrote: post #21

I wonder what's considered external though. If I compile / minify my javascript and CSS and then use a CDN to cache or host it -- is this considered external?? If so, how is it different from trusting my hosting provider to host my site in the first place, or my domain provider for resolving for me? How can this analysis know whether or not the resource is external? based on dns records alone? because I can still use…

If the url used to fetch the file is not related to the domain the original html comes from then that would be counted as external.

You can point *.my-domain.com to an external resource but it would see that resource as still under your control.

I will post the code soon.

Re: Some analysis of the 1M most popular sites on the web

#38

Post by jcr »

jacquesm, maybe you didn't want to wade into the details too much, but you didn't mention a major attack vector on third party scripts, namely, the transparent caches run by nearly all ISPs. Also, unless a third party script is served over HTTPS to users, regularly verifying the scripts is useless since _your_ ISP will give you _their_ cached copy, and similar is true for all site users. Transparent CDN's are another consideration for related caching problem.

Re: Some analysis of the 1M most popular sites on the web

#39

Post by aw3c2 »

jacquesm wrote: post #34
aw3c2 wrote: post #30

> Flash seems to be very rapidly on the way out, less than 1% of the domains I looked at still contained flash content What exactly did you look at? Homepages?

Yes, homepages and all the content subsequently loaded (directly or indirectly through multiple layers of scripting or iframes). Essentially what you'd get if you were to visit each and every homepage on the top list and logged the urls that were loaded as a consequence of that.

As much as I hate Flash, I don't think you can infer that then. Does your analytics consider https://www.youtube.com/ using Flash? I see no Flash on it here.

Re: Some analysis of the 1M most popular sites on the web

#40

Post by jfb »

enum_write3 wrote: post #16

Like, security is hard, comp sci was hard, so the websites are all built by social science grads and "designers" who know fuck all about security.

As opposed to all that sterling, highly-secure software built by REAL MEN with COMPUTER DEGREES, right?
Post reply on HN