ClickHouse cost-efficiency in action: analyzing 500B rows on an Intel NUC

Heavily optimized column store, incl use of SSE instructions, etc. Moreover, some architectural tradeoffs mentioned at https://clickhouse.yandex/docs/en/introduction/features_cons...

Re: ClickHouse cost-efficiency in action: analyzing 500B rows on an Intel NUC

#7

Post by SloopJon » Mon, Jan 06, 2020, 6:34 PM UTC

kresten wrote: post #3

What’s the tldr on why it is fast?

I'd say the materialized view is the main thing:

> Thus, we add the following materialized view ... At the end we should have 1440 times fewer rows in the aggregate than the source table.

The cost of populating that view is amortized over the 17.5 hours it took to load the data.

Re: ClickHouse cost-efficiency in action: analyzing 500B rows on an Intel NUC

#8

Post by chupasaurus » Mon, Jan 06, 2020, 6:50 PM UTC

jgrahamc wrote: post #2

We use ClickHouse extensively and it's been great: https://blog.cloudflare.com/http-analytics-for-6m-requests-p...

You've lost a Russian IT meme in translation.

We use the verb "to use brakes" to describe that something works slowly. There was a long story of threads about Java performance, until the meme was solidified after the news from 2005 DARPA Grand Challenge (racing competition for autonomous robotic self-driven cars): a car named Tommy by Jefferson Team which was running Java under Linux haven't used breaks before a turn and crashed to the wall at 70 mph, hence Java runs fast and Java doesn't use brakes were both described perfectly by the same sentence.

Yandex used the meme to advertise Clickhouse for engineers.

edit: formatting

edit2: brakes instead of breaks, wtf

Re: ClickHouse cost-efficiency in action: analyzing 500B rows on an Intel NUC

#9

Post by [deleted] » Mon, Jan 06, 2020, 6:58 PM UTC

chupasaurus wrote: post #8

jgrahamc wrote: post #2

We use ClickHouse extensively and it's been great: https://blog.cloudflare.com/http-analytics-for-6m-requests-p...

You've lost a Russian IT meme in translation. We use the verb " to use brakes " to describe that something works slowly. There was a long story of threads about Java performance, until the meme was solidified after the news from 2005 DARPA Grand Challenge (racing competition for autonomous robotic self-driven cars): a car named Tommy by Jefferson Team which was running Java under Linux haven't used breaks before a tu…

[deleted]

Re: ClickHouse cost-efficiency in action: analyzing 500B rows on an Intel NUC

#10

Post by nkrisc » Mon, Jan 06, 2020, 6:59 PM UTC

chupasaurus wrote: post #8

jgrahamc wrote: post #2

We use ClickHouse extensively and it's been great: https://blog.cloudflare.com/http-analytics-for-6m-requests-p...

You've lost a Russian IT meme in translation. We use the verb " to use brakes " to describe that something works slowly. There was a long story of threads about Java performance, until the meme was solidified after the news from 2005 DARPA Grand Challenge (racing competition for autonomous robotic self-driven cars): a car named Tommy by Jefferson Team which was running Java under Linux haven't used breaks before a tu…

I think you mean "brakes," as I thought you meant "breaks" like in an iterator. Not that familiar with Java though, just assuming based on the reference to a vehicle failing to brake before a turn.

I thought maybe it meant using break statements in your loops slows things down or something.