Building the world’s fastest website analytics (2021)
1–10 of 40 posts
Re: Building the world’s fastest website analytics (2021)
#2At some point, isn't it better to build your own database? And choose the trade offs that fit your project. Or pay/hire someone to do it.
Re: Building the world’s fastest website analytics (2021)
#3At some point, isn't it better to build your own database? And choose the trade offs that fit your project. Or pay/hire someone to do it.
And that point is when you can’t find someone to host it for you.
Though, I’d be sorely tempted to spin up a bfc of clickhouse on bare metal.
Re: Building the world’s fastest website analytics (2021)
#4At some point, isn't it better to build your own database? And choose the trade offs that fit your project. Or pay/hire someone to do it.
That sounds like a really bad solution, actually.
Building a reliable DBMS with any kind of decent performance can easily take years. And by then, it's still not sure that you're doing any better than any of the existing solutions.
Re: Building the world’s fastest website analytics (2021)
#5At some point, isn't it better to build your own database? And choose the trade offs that fit your project. Or pay/hire someone to do it.
Definitely not. No regrets after making the move. March next year will be the two year mark.
Re: Building the world’s fastest website analytics (2021)
#6At some point, isn't it better to build your own database? And choose the trade offs that fit your project. Or pay/hire someone to do it.
And that point is when you can’t find someone to host it for you. Though, I’d be sorely tempted to spin up a bfc of clickhouse on bare metal.
That one was tried, among many others covered in the article.
Re: Building the world’s fastest website analytics (2021)
#7Earlier quoted context omitted.
And that point is when you can’t find someone to host it for you. Though, I’d be sorely tempted to spin up a bfc of clickhouse on bare metal.
That one was tried, among many others covered in the article.
Clickhouse isn’t really relevant for a lot of us. SingleStore outperforms clickhouse and the latter isn’t MySQL wire compatible. Far more features come with SingleStore too. And if you speak to people running Clickhouse, they’re also maintaining a Postgres set-up. With SingleStore, you get your OLTP and OLAP in one database. So our users & sites table sit in memory (backed up by disk), meaning ultra fast read/write speeds (comparable to clustered/high availability Redis). And then we put our pageviews/events in columnstore (disk) which offers rapid performance for analytical queries.
Re: Building the world’s fastest website analytics (2021)
#8At some point, isn't it better to build your own database? And choose the trade offs that fit your project. Or pay/hire someone to do it.
I honestly can't think of any time where "build your own database" is a good solution, unless you're a database company or maybe Google.
Re: Building the world’s fastest website analytics (2021)
#9Sounds pretty cool. It's interesting to see how the landscape for this has changed over the last 15 years, since I worked at Splunk.
I just started working at FeatureBase a few weeks ago, here in Austin. It's Open Source and we have a 5 minute guide to get it running here: https://docs.featurebase.com/. We also have a cloud/serverless option.
FeatureBase is a B-tree database which uses Roaring Bitmaps. This makes it suitable for doing analytical queries on massive data sets immediately after ingestion. You do have to model the data properly, however.
Re: Building the world’s fastest website analytics (2021)
#10[deleted]