How does CH compare with the recent announcements made by Databricks Reyden...
Ten years of ClickHouse in open source
71–80 of 110 posts
Re: Ten years of ClickHouse in open source
#72CHDB deserves a lot of credit too, wish they would unmark the rust version as experimental. The python version works very well!
Re: Ten years of ClickHouse in open source
#73It is sad they are afraid to mention on the page that "data processing for a web analytics system ... similar to Google Analytics" was actually something used in Yandex.
Elsewhere on the page, they avoid mentioning Yandex. In fact, do they ever mention Yandex? That’s probably not to advertise for that company. I don’t see why it’s sad?
Or could it be because hecklers might call them "un-American" if they did list Yandex? The two aren't the same.
Re: Ten years of ClickHouse in open source
#74Earlier quoted context omitted.
What do you mean? Postrgesql is a relational and row based db, ClickHouse is columnar Clickhouse doesn’t replace postgresql:
This is a extremely common issue that happens in growing firms. You start off with everything in Postgres, it makes the most sense. Soon you realize some tables are growing really huge - usually some sort of time-series or log data reaching 10TB+. You can no longer fit it in one node. You can try you luck with some sharding extensions, but they add complexity to upgrades. In that case it makes total sense to move the…
I did this exact thing this year. I moved about >5 TB of event and log data into Clickhouse. I went from having to increase Postgres storage capacity every few months ($$$) to never thinking about it. Migrating to Clickhouse took me from constant timeouts on analytics queries to all queries returning in well under 100ms, and automated data retention using TTLs going from nightly pruning jobs that failed constantly (Postgres hates DELETEs on large tables) to again, never thinking about it. Because of this, I was able to increase retention per-account from a mere 30 days like I had -- just to keep performance and storage costs under control -- to much, much longer. Huge fan.
Re: Ten years of ClickHouse in open source
#75Love clickhouse it is just super performant. I've had to tweak a few queries for performance here and there but it has been more than stellar. I'd initially set up a real time pipeline ingest for us to handle our larger incremental ingests when i started (used redshift in the past, very expensive and quite slow in comparison), it has been unnecessary so far, as clickhouse has just been able to chew through tons of data and large transformations without breaking a sweat.
Only issue i had was that the default configuration had some quite heavy handed tracing turned on which tanked performance on the relatively little machine we had for it, it has since been scaled up and is the core of our data stack.
I can't recommend clickhouse enough. If you were truly large I'd probably choose something else, but as long as we're staying on a few nodes it is manageable complexity and we enjoy using it.
Re: Ten years of ClickHouse in open source
#76ClickHouse recently has been a breath of fresh air compared to using timescaledb for a long time. Although psql is the greatest there is and I really enjoyed the fact that I could rely on a single database system to run everything, when it came to migration maintenance and deployment it's really a pain and it also feels like development on timescaledb is a bit wishy washy with all the structural changes from version…
There is significant work happening in the PostgreSQL ecosystem to make "use a single system to run everything" possible. ParadeDB is one such system pushing on full-text/vector search, with some light aggregations, at the index level. There is also work being done by DuckDB folks with pg_duckdb and others like Xata. (disclaimer: I work at ParadeDB)
Re: Ten years of ClickHouse in open source
#77ClickHouse recently has been a breath of fresh air compared to using timescaledb for a long time. Although psql is the greatest there is and I really enjoyed the fact that I could rely on a single database system to run everything, when it came to migration maintenance and deployment it's really a pain and it also feels like development on timescaledb is a bit wishy washy with all the structural changes from version…
I was using TimescaleDB some very long time ago, things have changed quite a lot since (it's now even named differently). In my current setup I was thinking on doing both: upgrading postgresql to timescaledb (to archive old data etc.), and to deploy ClickHouse in parallel. I'm still considering whether to go big on PeerDB to get ClickHouse mirror or just deploy it separately without additional fragility layer. Would…
We at PeerDB/CickPipes tried to make that mirroring as frictionless as possible. It is validated at scale across 1000s of customer moving over half a PB of a data per month from Postgres to ClickHouse. You should give it a shot and you might be surprised.
Side note: May be there is way even more native than CDC/logical decoding that you never have to worry about keeping PG and CH in sync. Stay tuned for some updates from us on that end!
- Sai from ClickHouse
Re: Ten years of ClickHouse in open source
#78I discovered ClickHouse around 2017-18 and built a PoC to replace Elasticsearch: 5x better storage and qps, in a couple of weeks. Managers rejected it because it wasn't well known and was seen as "some database made by Russians." On a personal level, it's quite sad to have seen that train coming so early and not been able to get on board.
[flagged]
My country literally has sanctions on your country. If I do any projects with you, I’ll pay a fine of $200K and go to jail for up to 10 years.
It’s not racism, it’s just caution.
That’s sad because I’ve always wanted to visit Russia and from the outside it looks like a really interesting place.
Now, if you could change your government to one that listens to the European Court of Human Rights (namely: don’t start random wars, respect freedom of religion and respect people’s right not to participate in useless wars they don’t believe in) then we’d be best friends.
Re: Ten years of ClickHouse in open source
#79I discovered ClickHouse around 2017-18 and built a PoC to replace Elasticsearch: 5x better storage and qps, in a couple of weeks. Managers rejected it because it wasn't well known and was seen as "some database made by Russians." On a personal level, it's quite sad to have seen that train coming so early and not been able to get on board.
[flagged]
There is an awesome amount of software written by Russians - nginx, clickhouse, 7zip, Far Manager, WinRAR, even Kotlin (his creator) - I grew up, as bulgarian, with Russian books (especially math) that were indispensible.
Do I like the Russian goverment, or any of their goverments (current, soviet or royal ones) - Hell no! We were liberated "two" or was it "three" times by then only to screw things for us badly (Bulgaria).
The people, the culture, the language, the music, art, science are great...
I still don't understand why so many bulgarians still like Putin... but hey!
Re: Ten years of ClickHouse in open source
#80ClickHouse replacing Loki finally made our observability stack feel 'right'. It really is a powerhouse for logs and general analytical queries.
How do you use it for visualization? Do you use ClickStack? or something else?
There is HyperDX - search is not fastest, but it could be something that we do too - haven't checked deeply if high-cardinality is big issue with ClickHouse, but seeing some high cardinality data with what we post.