Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building the world’s fastest website analytics (2021)

usefathom.com

Post reply on HN
Print view View original

31–40 of 40 posts

Re: Building the world’s fastest website analytics (2021)

#31

Post by dravita »

hodgesrm wrote: post #23
dravita wrote: post #19

Earlier quoted context omitted.

If you're really confident that your SaaS application will only have that single workload (single table group-by) into perpetuity, it's a workable choice. But, that's a big bet against the reality of the diverse workloads required today in scale-up SaaS applications. Sure, you can stitch together specialized stores to make your own accidental distributed database for your app, but the simpler, more straight-forward a…

> Sure, you can stitch together specialized stores to make your own accidental distributed database for your app, but the simpler, more straight-forward and lower-latency alternative available. Organizations like Facebook, Amazon, Netflix, Cloudflare, and many, many others have been combining specialized data stores for many years. Stonebraker, Madden, and colleagues summarized the argument for this approach in 2007.…

So, you think that the 99.99% of businesses around the world who are not in the cloud hyperscaler business should all build their systems as if they had the engineering resources of cloud hyperscalers? I've seen first-hand how enterprises with Web 2.0 envy made costly, multi-year mistakes in technology choices and architecture with that approach. Be inspired by the engineering prowess of FB, AWS, Netflix, etc., for sure, but don't make the mistake of taking the approach of a $470B (AWS) or $86B (FB) company directly when your company's situation is nothing like that. Stated another way, you can't coach exactly like Pep Guardiola if you don't have his resources and his players. Existing enterprises have a totally different starting point. Enterprises simply can't follow the hyperscaler hegemony for their digital transformation strategies and expect to win. Also, think about who is served by building inefficient, accidental distributed databases underneath your SaaS applications when that can be avoided by using a proven technology alternative already exists. It's the CSPs you're paying in the monthly cloud bill for every byte stored, every byte computed and every byte moved.

Re: Building the world’s fastest website analytics (2021)

#32

Post by hodgesrm »

dravita wrote: post #31
hodgesrm wrote: post #23

Earlier quoted context omitted.

> Sure, you can stitch together specialized stores to make your own accidental distributed database for your app, but the simpler, more straight-forward and lower-latency alternative available. Organizations like Facebook, Amazon, Netflix, Cloudflare, and many, many others have been combining specialized data stores for many years. Stonebraker, Madden, and colleagues summarized the argument for this approach in 2007.…

So, you think that the 99.99% of businesses around the world who are not in the cloud hyperscaler business should all build their systems as if they had the engineering resources of cloud hyperscalers? I've seen first-hand how enterprises with Web 2.0 envy made costly, multi-year mistakes in technology choices and architecture with that approach. Be inspired by the engineering prowess of FB, AWS, Netflix, etc., for s…

MySQL, ClickHouse, and Kafka are all proven technologies. You can run them yourself or choose any of a number of excellent cloud services to run them for you. It's not that hard to combine them and let each part serve the purpose for which it is designed. I've seen companies from small startups to organizations the size of Facebook take this approach. It's a reasonable architecture with many thousands of successful examples.

My own company is not especially large, but we run MySQL, ClickHouse, and Prometheus to manage our own SaaS platform for ClickHouse. We are happy with the result. (Though it would be nice to use ClickHouse as a Prometheus backend.)

Re: Building the world’s fastest website analytics (2021)

#33

Post by JackWritesCode »

nnopepe wrote: post #13
SmoothBrain12 wrote: post #2

At some point, isn't it better to build your own database? And choose the trade offs that fit your project. Or pay/hire someone to do it.

These people don't have the inhouse skills to maintain a simple SQL server, let alone optimize their schema (MySQL being too slow to sum/group 11m records in a timely manner? Please.) Attempting to make your own db while being ignorant of competing products would be an absolute disaster

I completely agree! For the people following back home, who don’t want to move away from MySQL, can you let us know how to achieve a The query was:

SELECT SUM(pageviews) as total, pathname FROM pageviews GROUP BY pathname ORDER BY total DESC LIMIT 10.

If we don’t hear an answer from you, I’ll be really upset. Otherwise, we may have to add your ideas to the article!

Re: Building the world’s fastest website analytics (2021)

#34

Post by nojvek »

SingleStore is pretty amazing since it supports UPDATE DELETE and joins with other tables.

Clickhouse updates are painful. I looked on postgres with citus extension and it only supports appends.

Their paper on how they organize the columnstore and row store buffer is pretty neat. https://images.go.singlestore.com/Web/MemSQL/%7Bd1e77ba1-0e3...

It’s really cool to see usefathom take off doing millions in ARR with two engineers. SingleStore has to been a high leverage choice so they can focus on product. Power to them.

Meanwhile we’ve been grappling with a mixture of Mongo, Snowflake and Postgres for more than an year with a decent sized team. The curse of VC funded startups is that with more money and more engineers, a likely outcome is more complexity. The infra becomes a complex beast that takes many quarters/years to build because there isn’t that market pressure to be time and money efficient.

Re: Building the world’s fastest website analytics (2021)

#35

Post by begonealong »

datalopers wrote: post #27
AdamProut wrote: post #26

Earlier quoted context omitted.

Because half truths like your statements bother me? Yandex is a large investor in clickhouse Inc. (at last as far as public record shows). Have they forced Yandex to divest their holdings?

It's literally in the article I linked ( https://clickhouse.com/blog/we-stand-with-ukraine ). As you're incapable of reading let me highlight it for you: > We have no operations in Russia, no Russian investors, and no Russian members of our Board of Directors. It's disappointing after 11+ yrs of failing to find traction for MemSQL, this is the hill you die on? You're running your mouth on some random comments thread…

This article makes a lot more sense when you know that Singlestore/MemSQL gifted Jack Ellis shares.

Re: Building the world’s fastest website analytics (2021)

#36

Post by datalopers »

begonealong wrote: post #35
datalopers wrote: post #27

Earlier quoted context omitted.

It's literally in the article I linked ( https://clickhouse.com/blog/we-stand-with-ukraine ). As you're incapable of reading let me highlight it for you: > We have no operations in Russia, no Russian investors, and no Russian members of our Board of Directors. It's disappointing after 11+ yrs of failing to find traction for MemSQL, this is the hill you die on? You're running your mouth on some random comments thread…

This article makes a lot more sense when you know that Singlestore/MemSQL gifted Jack Ellis shares.

Hah. What a joke. So it's all astroturfing and fake evangelism.

Re: Building the world’s fastest website analytics (2021)

#37

Post by JackWritesCode »

begonealong wrote: post #35
datalopers wrote: post #27

Earlier quoted context omitted.

It's literally in the article I linked ( https://clickhouse.com/blog/we-stand-with-ukraine ). As you're incapable of reading let me highlight it for you: > We have no operations in Russia, no Russian investors, and no Russian members of our Board of Directors. It's disappointing after 11+ yrs of failing to find traction for MemSQL, this is the hill you die on? You're running your mouth on some random comments thread…

This article makes a lot more sense when you know that Singlestore/MemSQL gifted Jack Ellis shares.

Lmao. Hey, if I wasn't running Fathom, Dev Rel @ SingleStore is the only role I'd consider in tech right now. And I would definitely try to get some of those sweet shares as part of compensation. Alas, I don't own any SingleStore shares.

Re: Building the world’s fastest website analytics (2021)

#38

Post by bizofeet »

datalopers wrote: post #16

They would have saved a whole lot of time just using Clickhouse which is a US company and has completely severed ties with Russia [1]. Altinity has offered managed clickhouse since 2020 [2], and now Clickhouse Cloud does too [3] [1] https://clickhouse.com/blog/we-stand-with-ukraine [2] https://altinity.com/cloud-database/ [3] https://clickhouse.com/cloud

Clickhouse is a great technology for a niche scenario. A lack of a good QO is one big thing missing but not the only one unfortunately.

I know a company that forked Clickhouse but had to rebuild virtually everything to be a proper database.

There are already plenty of technologies, more mature than Clickhouse for addressing such scenarios.

Re: Building the world’s fastest website analytics (2021)

#39

Post by qxip »

hodgesrm wrote: post #32
dravita wrote: post #31

Earlier quoted context omitted.

So, you think that the 99.99% of businesses around the world who are not in the cloud hyperscaler business should all build their systems as if they had the engineering resources of cloud hyperscalers? I've seen first-hand how enterprises with Web 2.0 envy made costly, multi-year mistakes in technology choices and architecture with that approach. Be inspired by the engineering prowess of FB, AWS, Netflix, etc., for s…

MySQL, ClickHouse, and Kafka are all proven technologies. You can run them yourself or choose any of a number of excellent cloud services to run them for you. It's not that hard to combine them and let each part serve the purpose for which it is designed. I've seen companies from small startups to organizations the size of Facebook take this approach. It's a reasonable architecture with many thousands of successful e…

> it would be nice to use ClickHouse as a Prometheus backend

Well... that's already possible and it works great! As you might know https://qryn.dev turns ClickHouse into a powerful Prometheus *remote_write* backend and the GO/cloud version supports full PromQL queries off ClickHouse transparently (the JS/Node version transpiles to LogQL instead) and from a performance point of view its well on par with Prometheus, Mimir and Victoriametrics in our internal benchmarks (including Clickhouse as part of the resource set) with millions of inserts/s and broad client compatibility. Same for Logs (LogQL) and Traces (Tempo)

Disclaimer: I work on qryn

Post reply on HN