What to do when your site is "too large" for Google Analytics? • Hacker News Board

Re: What to do when your site is "too large" for Google Analytics?

#31

Post by liedra » Thu, Aug 12, 2010, 8:42 PM UTC

124816 wrote: post #27

liedra wrote: post #23

Earlier quoted context omitted.

That's curious, because I was just at a talk by Alma Whitten who said that Google didn't use any of the GA data for profiling or ad-related stuff, and simply provided it as a service. Did I misunderstand her, or, possibly, you?

Alma Whitten might be uninformed. (Was this talk recorded? Do you have a link to it?) There are some options in the account UI that, if enabled, allow Google to do some analysis of your data. Link directly to their help center: http://www.google.com/support/analytics/bin/answer.py?hl=en&...

Ooh, okay, I think I see where the misunderstanding is: I was asking Alma (in questioning time at a talk at the IFIP/PrimeLife summer school ( http://www.cs.kau.se/IFIP-summerschool/program.html ), which was unfortunately not recorded) about website users' information, not the data of the website itself. Though it's a pretty fine line there, isn't it!

I told her that I blocked google analytics because I didn't want it to find out things about me for advertising etc. She responded saying that Google doesn't do that, but now that I think about it she was probably saying that Google doesn't profile me specifically, nothing about anonymised data... yeah, now I'm not so sure. :)

Re: What to do when your site is "too large" for Google Analytics?

#32

Post by loup-vaillant » Thu, Aug 12, 2010, 9:47 PM UTC

124816 wrote: post #29

jasonkester wrote: post #26

Earlier quoted context omitted.

The reason that Google is choking on this 30M pageview/month site is because analytics for a 30M pageview/month site is hard . Things like unique visitor, unique referer, unique whatever tracking just explode when you have to compare each IP address you get with 20M-entry list. If you try to do it on your webserver, you'll have a 3 hour window each day where a significant portion of your CPU and Memory is not availab…

Most stats providers compute unique visitors using client-side cookies; not entirely accurate, but quite cheap. You're right though, it's a lot easier to use an existing (and free or nearly free) tool than to build your own.

I agree it's a lot easier. But Google Analytics has a drawback I can't accept: several sites I visited make me process scripts from Analitycs, and that takes a noticeable amount of time and bandwidth. When someone visits my site, she's giving me her time. I'd rather not waste it.

That, and the fact that I prefer to process my logs on my computers. As a matter of principle (decentralization, privacy, freedom, blah blah).

Now, I barely know Analytics. But it looks cool, so I'll check, and see if I can reap most of their benefits myself. (If I can't, I'll have a tough choice indeed.)

Re: What to do when your site is "too large" for Google Analytics?

#33

Post by 124816 » Fri, Aug 13, 2010, 3:02 AM UTC

loup-vaillant wrote: post #32

124816 wrote: post #29

Earlier quoted context omitted.

Most stats providers compute unique visitors using client-side cookies; not entirely accurate, but quite cheap. You're right though, it's a lot easier to use an existing (and free or nearly free) tool than to build your own.

I agree it's a lot easier. But Google Analytics has a drawback I can't accept: several sites I visited make me process scripts from Analitycs, and that takes a noticeable amount of time and bandwidth. When someone visits my site, she's giving me her time. I'd rather not waste it. That, and the fact that I prefer to process my logs on my computers. As a matter of principle (decentralization, privacy, freedom, blah bla…

Yep, GA used to have the occasional problem with that, though I've been told some of it was due to a bug in the browser (Firefox's) code that decided what to put in the "Waiting for xyz..." spot -- if that's what you're referring to.

(Something about that bar not being updated until the first byte of the stream was read -- so if a static.foo.com was hanging connections, it would get stuck blaming whatever loaded right before it.)

GA now has asynchronous tracking though, which solves this pretty nicely. Though I'm not a JS guy, my understanding of it is that instead of sourcing the script and then issuing the tracking calls, you put some instructions into a global variable, and then source the script. Nothing blocks on the script's loading, so even if GA is down the rest of the page is supposed to load correctly. Once the GA script loads, it looks at the global and runs the tracking commands.

Re: What to do when your site is "too large" for Google Analytics?

#34

Post by loup-vaillant » Fri, Aug 13, 2010, 7:15 AM UTC

124816 wrote: post #33

loup-vaillant wrote: post #32

Earlier quoted context omitted.

I agree it's a lot easier. But Google Analytics has a drawback I can't accept: several sites I visited make me process scripts from Analitycs, and that takes a noticeable amount of time and bandwidth. When someone visits my site, she's giving me her time. I'd rather not waste it. That, and the fact that I prefer to process my logs on my computers. As a matter of principle (decentralization, privacy, freedom, blah bla…

Yep, GA used to have the occasional problem with that, though I've been told some of it was due to a bug in the browser (Firefox's) code that decided what to put in the "Waiting for xyz..." spot -- if that's what you're referring to. (Something about that bar not being updated until the first byte of the stream was read -- so if a static.foo.com was hanging connections, it would get stuck blaming whatever loaded righ…

I do use Firefox, so I probably hit that bug. Also, with NoScript, forbidding GA causes no problem.

But still, I would have liked the analysis process to be completely invisible from the client.