Viewing profile — ople

"Point at any ClickHouse table – no schema migrations/no OTEL requirement" I haven't tried it yet but I definitely will because of this. Pretty much every other CH observability pr…

This is a cool and underrated new feature: It basically enables simple joins of large and dynamically updated tables efficiently. This is something that has been a bit of an Achill…

Why not just use SQL? With LLMs evolving to do sophisticated text-to-SQL, the case for a custom language for the sake of simplicity is diminishing. I think that expressiveness, per…

This is one of the things I appreciate about ClickHouse: Contributors willing and able to dive in to performance tune even low-level core stuff like this and getting merged quickly…

True. My bet is that arbitrary text queries converted to SQL via LLM will become an increasingly popular alternative. Not sure if it will completely replace the custom query langua…

Recently there seems to be an bunch of SPL (Splunk) -like query languages popping up: PQL, PRQL, Grafana Explore Logs syntax, Kusto.. Probably others as well. Does yet another simi…

Very interesting observations! Merge performance tuning seems often overlooked even though it's a key aspect of sustained ClickHouse performance. I also like that the blog is quite…

I have had the same experience. I was constantly bumping into unexpected limitations. Moving to CH felt like the opposite, with many more ”Wow, I didn’t expect this to be possible …

I have pretty much exactly the same experience. However, I do feel that they are trying to really do the right thing with the new 3.0 architecture, addressing the deficiencies (mos…

Unsurprisingly IBM pioneered this with "Capacity on Demand" https://www.ibm.com/docs/en/power9?topic=environment-capacit... I'm guessing it has had some popularity among customers …

Hehe.. In retrospect the whole team was in fairly good spirit although the situation was stressful. A lot of this was due to the top management giving the time and space for the sp…

The reading of the metadata required quite a lot of random acces. We were fairly sure that if a high-end array and controller with fast disks is struggling with it, then a traditio…

We copied the raw image file of the corrupted metadata filesystem (MDT in Lustre lingo) to the ramdisk. Then we mounted it via loopback and copied the files to tarballs. The bit th…

The metadata target (MDT) in the MDS is actually "ldiskfs" which is an enhanced version of ext4. One possibility may be to use ZFS in the future as the support in Lustre seems to b…

It was filesystem-level corruption in Lustre. The underlying disk arrays and other hardware have comprehensive redundancy.

Author here. We have had an automated deletion policy on our previous filesystems but opted out this time: There are users that have temporary files that they want to persist on th…

Author here: We considered that but as the access pattern was likely pretty much random, the performance would have been terrible. Due to the break we had nearly a 1000 clustered s…