I mean as a POC its not bad, but google analytics is not the same as analyzing server logs (contrary to what most people would suggest). Most of the value of ga comes from session and user level metrics, which are 1000x more difficult to implement than showing pageviews. Unless you are planning on building a device graph that rivals google, you can't clone ga.
Show HN: Serverless Analytics Built from Scratch
21–30 of 50 posts
Re: Show HN: Serverless Analytics Built from Scratch
#22"serverless" is really the misnomer of the year.
"I implemented this using some technology that abstracted away the server configuration, but it is running on a server... hmmm"
Re: Show HN: Serverless Analytics Built from Scratch
#23"serverless" is really the misnomer of the year.
"I implemented this using some technology that abstracted away the server configuration, but it is running on a server... hmmm"
Nonono, no server:
"[..] write serverless code which runs in the fabric of the Internet itself" - https://blog.cloudflare.com/building-with-workers-kv/
Re: Show HN: Serverless Analytics Built from Scratch
#24I mean as a POC its not bad, but google analytics is not the same as analyzing server logs (contrary to what most people would suggest). Most of the value of ga comes from session and user level metrics, which are 1000x more difficult to implement than showing pageviews. Unless you are planning on building a device graph that rivals google, you can't clone ga.
> google analytics is not the same as analyzing server logs
This is what most people don’t get with ga.
Google Analytics does the heavy lift by removing incoherent , corrupted or malicious data insertion.
Let’s say I use Puppeteer i can scrap this page a million time with completely wrong headers like « Netscape 8.1 ». GA purify this type of malicious attempts , it will probably look my IP Adress and figure out that it’s actually coming from only one IP and « Netscape » is too rare to be considered as an actual browser so it would probably ignore it.
All others « free google analytics alternatives » that exists today don’t have this type of mechanism to prevent from data corruption.
In general they just get an Http Request and acknowledge it as a legitimate visit.
Logging an Http request from a browser is not even a tenth of the work GA does under the hood.
Re: Show HN: Serverless Analytics Built from Scratch
#25Interesting. I once built a ga clone using appengine, cloud dataflow, and big query. I guess that would count as serverless? Benchmarked it against the official dumps to big quey too and it was pretty spot on for every metric we could lookup!
Bigquery takes minimum 2-3 seconds for every query. Google Analytics is much faster, responds in a few hundreads milliseconds. What did you use dataflow for? How did you get data from end points and insert them into bigquery? Using streaming inserts?
> Google Analytics is much faster, responds in a few hundreads milliseconds.
GA stores summary tables for each day for the basic values. If you have a large site and request segments or anything that's not in the summary tables, it can be quite slow.
Also, BigQuery is multi-tenant. GA would have dedicated instances.
> What did you use dataflow for? How did you get data from end points and insert them into bigquery? Using streaming inserts?
cosmie pretty much got it. AppEgnine collected. DataFlow sessionized and some other processing (geoip lookup, filtering, &c). BigQuery stored.
I actually had AppEngine dumping into Cloud Datastore, but I also experimented with PubSub and also using Cloud Storage access logs.
Re: Show HN: Serverless Analytics Built from Scratch
#26I mean as a POC its not bad, but google analytics is not the same as analyzing server logs (contrary to what most people would suggest). Most of the value of ga comes from session and user level metrics, which are 1000x more difficult to implement than showing pageviews. Unless you are planning on building a device graph that rivals google, you can't clone ga.
> google analytics is not the same as analyzing server logs This is what most people don’t get with ga. Google Analytics does the heavy lift by removing incoherent , corrupted or malicious data insertion. Let’s say I use Puppeteer i can scrap this page a million time with completely wrong headers like « Netscape 8.1 ». GA purify this type of malicious attempts , it will probably look my IP Adress and figure out that…
I'm not sure how well that filtering works in practice. I think most of it is just that it only tracks clients that load javascript.
Re: Show HN: Serverless Analytics Built from Scratch
#27I mean as a POC its not bad, but google analytics is not the same as analyzing server logs (contrary to what most people would suggest). Most of the value of ga comes from session and user level metrics, which are 1000x more difficult to implement than showing pageviews. Unless you are planning on building a device graph that rivals google, you can't clone ga.
> google analytics is not the same as analyzing server logs This is what most people don’t get with ga. Google Analytics does the heavy lift by removing incoherent , corrupted or malicious data insertion. Let’s say I use Puppeteer i can scrap this page a million time with completely wrong headers like « Netscape 8.1 ». GA purify this type of malicious attempts , it will probably look my IP Adress and figure out that…
> Google Analytics does the heavy lift by removing incoherent , corrupted or malicious data insertion.
Unless it's referrer spam...that somehow still sticks around (at least last time I checked, which was several months ago).
Re: Show HN: Serverless Analytics Built from Scratch
#28"serverless" is really the misnomer of the year.
[deleted]
Re: Show HN: Serverless Analytics Built from Scratch
#29"serverless" is really the misnomer of the year.
"Still on a server, but not your problem" just doesn't have the same ring to it though