Some analysis of the 1M most popular sites on the web

i have always been irked by the fact that my bank and most websites behind an https connection (especially after login) have ANY external resources. this is a major flaw on numerous levels; how such a thing is still allowed by browser vendors is IMO borderline voluntary negligence. all such sites, if they must host external resources, should do so only within an iframe w/sandbox [1]. the fact that the external resource is also "https" [on some foreign property] is completely and utterly meaningless.

[1] http://www.w3schools.com/tags/att_iframe_sandbox.asp

Re: Some analysis of the 1M most popular sites on the web

#5

Post by jacquesm » Fri, Jul 24, 2015, 4:37 PM UTC

leeoniya wrote: post #4

i have always been irked by the fact that my bank and most websites behind an https connection (especially after login) have ANY external resources. this is a major flaw on numerous levels; how such a thing is still allowed by browser vendors is IMO borderline voluntary negligence. all such sites, if they must host external resources, should do so only within an iframe w/sandbox [1]. the fact that the external resour…

My bank (ABN/Amro in NL) is even worse, they not only include external resources, they include external resources that are critical for their site to function, in other words, if I disable the various trackers and analytics elements on their page the site simply no longer works. You'd expect the opposite!

Re: Some analysis of the 1M most popular sites on the web

#6

Post by leeoniya » Fri, Jul 24, 2015, 4:54 PM UTC

jacquesm wrote: post #5

leeoniya wrote: post #4

i have always been irked by the fact that my bank and most websites behind an https connection (especially after login) have ANY external resources. this is a major flaw on numerous levels; how such a thing is still allowed by browser vendors is IMO borderline voluntary negligence. all such sites, if they must host external resources, should do so only within an iframe w/sandbox [1]. the fact that the external resour…

My bank (ABN/Amro in NL) is even worse, they not only include external resources, they include external resources that are critical for their site to function, in other words, if I disable the various trackers and analytics elements on their page the site simply no longer works. You'd expect the opposite!

wow! i think there should be a place on the internet for publicly naming and shaming such practices. like a Darwin Award or Razzies [1] of webdev.

[1] https://en.wikipedia.org/wiki/Golden_Raspberry_Awards

Re: Some analysis of the 1M most popular sites on the web

#7

Post by heyalexej » Fri, Jul 24, 2015, 4:55 PM UTC

This is very interesting. Will you release the data and code at some point?

Re: Some analysis of the 1M most popular sites on the web

#8

Post by ktusznio » Fri, Jul 24, 2015, 4:58 PM UTC

What about services like npm that distribute code? Are these analogous or do they have additional security in place?

Re: Some analysis of the 1M most popular sites on the web

#9

Post by jacquesm » Fri, Jul 24, 2015, 5:01 PM UTC

heyalexej wrote: post #7

This is very interesting. Will you release the data and code at some point?

Yes, I will definitely release the code and the dataset required to bootstrap the rest. It takes a long long time to run and you'll need a good bit of bandwidth. I won't be releasing the raw data because there is simply too much of it.

Re: Some analysis of the 1M most popular sites on the web

#10

Post by jacquesm » Fri, Jul 24, 2015, 5:04 PM UTC

leeoniya wrote: post #6

jacquesm wrote: post #5

Earlier quoted context omitted.

My bank (ABN/Amro in NL) is even worse, they not only include external resources, they include external resources that are critical for their site to function, in other words, if I disable the various trackers and analytics elements on their page the site simply no longer works. You'd expect the opposite!

wow! i think there should be a place on the internet for publicly naming and shaming such practices. like a Darwin Award or Razzies [1] of webdev. [1] https://en.wikipedia.org/wiki/Golden_Raspberry_Awards

That's one of the things I'm considering right now. To re-write the top 1000 or so with annotations and then to sort them by category as well as an example of a site that is 'clean' in the same category.

There are a ton of offenders and some of them are very well known.

One of the interesting things you find when you look at this data is that the bigger sites really do have their stuff set up better (for instance, by using in-house analytics) but there are lots and lots of exceptions and some of them are quite shocking. For instance, I've found two major car brands that include evercookies on their corporate websites in Eastern Europe, a thing that no respectable company should ever do. I suspect an ad agency is the cause of this so I'm still digging away at that.