Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizing ClickHouse for Intel's ultra-high core count processors

clickhouse.com

Post reply on HN
Print view View original

11–20 of 53 posts

Re: Optimizing ClickHouse for Intel's ultra-high core count processors

#11

Post by nivertech »

pixelpoet wrote: post #3

This post looks like excellent low-level optimisation writing just in the first sections, and (I know this is kinda petty, but...) my heart absolutely sings at their use of my preferred C++ coding convention where & (ref) neither belongs to the type nor the variable name!

I think it belongs to type, but since they use “auto” it looks standalone and can be confused with the “&” operator. I personally always used * and & as a prefix of the variable name, not as a suffix in the type name, except when used to specify types in templates.

Re: Optimizing ClickHouse for Intel's ultra-high core count processors

#13

Post by sdairs »

hobo_in_library wrote: post #8
jiehong wrote: post #7

Great work! I like duckdb, but clickhouse seems more focused on large scale performance. I just thought that the article is written from the point of view of a single person, but has multiple authors, which is a bit weird. Did I misunderstood something?

Not sure what happened here, but it's not uncommon for a post to have one primary author and then multiple reviewers/supporters also credited

Yep that's pretty much the case here!

Re: Optimizing ClickHouse for Intel's ultra-high core count processors

#14

Post by pclmulqdq »

bee_rider wrote: post #4

288 cores is an absurd number of cores. Do these things have AVX512? It looks like some of the Sierra Forest chips do have AVX512 with 2xFMA… That’s pretty wide. Wonder if they should put that thing on a card and sell it as a GPU (a totally original idea that has never been tried, sure…).

AVX-512 is on the P-cores only (along with AMX now). The E-cores only support 256-bit vectors.

If you're doing a lot of loading and storing, these E-core chips are probably going to outperform the chips with huge cores because they will be idling a lot. For CPU-bound tasks, the P-cores will win hands down.

Re: Optimizing ClickHouse for Intel's ultra-high core count processors

#15

Post by sdairs »

jiehong wrote: post #7

Great work! I like duckdb, but clickhouse seems more focused on large scale performance. I just thought that the article is written from the point of view of a single person, but has multiple authors, which is a bit weird. Did I misunderstood something?

ClickHouse works in-process and on the CLI just like DuckDB, but also scales to hundreds of nodes - so it's really not limited to just large scale. Handling those smaller cases with a great experience is still a big focus for us

Re: Optimizing ClickHouse for Intel's ultra-high core count processors

#16

Post by sdairs »

bee_rider wrote: post #4

288 cores is an absurd number of cores. Do these things have AVX512? It looks like some of the Sierra Forest chips do have AVX512 with 2xFMA… That’s pretty wide. Wonder if they should put that thing on a card and sell it as a GPU (a totally original idea that has never been tried, sure…).

how long until I have 288 cores under my desk I wonder?

Re: Optimizing ClickHouse for Intel's ultra-high core count processors

#19

Post by vlovich123 »

I'm generally surprised they're still using the unmaintained old version of jemalloc instead of a newer allocator like the Bazel-based TCMalloc or mimalloc which have significantly better techniques due to better OS primitives & about a decade or so of R&D behind them.

Re: Optimizing ClickHouse for Intel's ultra-high core count processors

#20

Post by zokier »

sdairs wrote: post #16
bee_rider wrote: post #4

288 cores is an absurd number of cores. Do these things have AVX512? It looks like some of the Sierra Forest chips do have AVX512 with 2xFMA… That’s pretty wide. Wonder if they should put that thing on a card and sell it as a GPU (a totally original idea that has never been tried, sure…).

how long until I have 288 cores under my desk I wonder?

Does 2x160 cores count?

https://www.titancomputers.com/Titan-A900-Octane-Dual-AMD-EP...

Post reply on HN