Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ClickHouse, a column-oriented DBMS to generate analytical reports in real time

github.com

Post reply on HN
Print view View original

21–30 of 33 posts

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#21

Post by bicubic »

eldargab wrote: post #17
dorfsmay wrote: post #5

Can it run on a cluster? Or single server only? Can you update specific rows? How fast are updates? How does it compare to monetDB.

monetDB is a sort of drop-in replacement for a regular database with all expected features and good compatibility. On other hand ClickHouse will be incompatible with most of existing tools and it's better to learn well its limitations and workaround technics in advance. But once you dump into it substantial amounts of time series data you'll find it 10+ times faster and 2-3 times smaller than monet.

Is there a way to handle scenarios where the data does update? Like backfill of latent events within a 1-2 day window.

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#22

Post by tadkar »

lykr0n wrote: post #3

Clickhouse has some weird quirks when you think of it as a SQL Database, but its astounding to use it. It's faster than one would think, it can do some really cool data modeling, and provides a wealth of features for the average user out of the box. The most important thing, and the thing that makes it attractive to me is that it is almost stupidly simple to setup and get running. It's quite simple (when you wrap you…

Second the “stupidly simple to setup and get running”. My company works with billion row datasets on client sites where we get super locked down accounts. Clickhouse is a single binary that you can run with no actual “install” needed.

Also echo the comments in the rest of the discussion about it being blazing fast. On our beefier machines we get querying in the 100s of millions of rows per second when the data is not in cache.

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#23

Post by drej »

I remember playing with it a while ago and here are my 2c:

- It's ridiculously fast, like, I didn't know where the performance was coming from.

- Getting it up and running was a bit clunky (Docker saved me), but I hear it's better now.

- It has non-standard (I mean, everyone is non-standard, but this is way off ANSI) and case-sensitive (???) SQL syntax. This annoyed me again and again.

- It seemed (and still seems) like a project that lives and dies with one developer - and no matter how brilliant he may be, I'm not willing to invest in a technology that has this risk and it's so hard to migrate off of (because of the non-standard SQL).

I'm sad about the last two points, because the database is rather brilliant otherwise.

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#24

Post by dschuler »

drej wrote: post #23

I remember playing with it a while ago and here are my 2c: - It's ridiculously fast, like, I didn't know where the performance was coming from. - Getting it up and running was a bit clunky (Docker saved me), but I hear it's better now. - It has non-standard (I mean, everyone is non-standard, but this is way off ANSI) and case-sensitive (???) SQL syntax. This annoyed me again and again. - It seemed (and still seems) l…

By one developer you mean Yandex or that most commits are made by a couple of users? Being backed by a large company (the Russian Google apparently) that has an independent revenue stream seems like a large plus, but maybe not enough to cancel out.

I'm wary of investing effort into a potentially unsupported project as well, but I wonder if ClickHouse only seems "out there" because we're not aware of the Russian tech ecosystem (at least I'm not).

People don't seem concerned about building anything with Firebase, but Google doesn't have a good track record of changing its mind about priorities or service pricing.

What would you recommend instead for a column-oriented db that you can self-host (commercial or open source)?

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#25

Post by Grue3 »

drej wrote: post #23

I remember playing with it a while ago and here are my 2c: - It's ridiculously fast, like, I didn't know where the performance was coming from. - Getting it up and running was a bit clunky (Docker saved me), but I hear it's better now. - It has non-standard (I mean, everyone is non-standard, but this is way off ANSI) and case-sensitive (???) SQL syntax. This annoyed me again and again. - It seemed (and still seems) l…

>It seemed (and still seems) like a project that lives and dies with one developer

This project is developed by Yandex and they have a team working on it.

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#26

Post by bsg75 »

georgewfraser wrote: post #16
manigandham wrote: post #11

Earlier quoted context omitted.

Redshift doesn't separate compute from storage either unless you're using Spectrum. Presto isn't a database at all and can read from many data stores. The rest are all cloud-hosted with lots of moving parts. MemSQL, Vertica, Actian, Greenplum, and SQL Server are better comparisons. ClickHouse is a column-oriented db and actually one of the most advanced, focusing on performance at all costs with lots of table storage…

My point is there’s just no advantage to ClickHouse. The things that make it fast are in every column store. There’s other options that do everything it does and more.

By this definition we should all be running Oracle.

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#27

Post by drej »

dschuler wrote: post #24
drej wrote: post #23

I remember playing with it a while ago and here are my 2c: - It's ridiculously fast, like, I didn't know where the performance was coming from. - Getting it up and running was a bit clunky (Docker saved me), but I hear it's better now. - It has non-standard (I mean, everyone is non-standard, but this is way off ANSI) and case-sensitive (???) SQL syntax. This annoyed me again and again. - It seemed (and still seems) l…

By one developer you mean Yandex or that most commits are made by a couple of users? Being backed by a large company (the Russian Google apparently) that has an independent revenue stream seems like a large plus, but maybe not enough to cancel out. I'm wary of investing effort into a potentially unsupported project as well, but I wonder if ClickHouse only seems "out there" because we're not aware of the Russian tech…

It's in the Yandex namespace and it is used by said company, which is a huge plus. But if you looked at the development history just a while ago, it was highly dependent on Alexey.

It reminds me of Grumpy (https://github.com/google/grumpy), which was released by Google, but was later basically abandoned when the lead left Google.

That being said, the situation is better than last time I checked this, there is a handful of somewhat active developers. https://github.com/yandex/ClickHouse/graphs/contributors

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#28

Post by bsg75 »

drej wrote: post #23

I remember playing with it a while ago and here are my 2c: - It's ridiculously fast, like, I didn't know where the performance was coming from. - Getting it up and running was a bit clunky (Docker saved me), but I hear it's better now. - It has non-standard (I mean, everyone is non-standard, but this is way off ANSI) and case-sensitive (???) SQL syntax. This annoyed me again and again. - It seemed (and still seems) l…

> It seemed (and still seems) like a project that lives and dies with one developer

One major contributor (who may be project lead at Yandex?) and a lot of active contributors: https://github.com/yandex/ClickHouse/graphs/contributors

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#29

Post by PeterZaitsev »

drej wrote: post #23

I remember playing with it a while ago and here are my 2c: - It's ridiculously fast, like, I didn't know where the performance was coming from. - Getting it up and running was a bit clunky (Docker saved me), but I hear it's better now. - It has non-standard (I mean, everyone is non-standard, but this is way off ANSI) and case-sensitive (???) SQL syntax. This annoyed me again and again. - It seemed (and still seems) l…

If you need Advanced SQL Support ClickHouse is not there (Yet) but if you need high performance for relatively basic queries ClickHouse is great.

It is developed mostly by ClickHouse staff but there is at least one company https://www.altinity.com/ which offers Commercial Support, Consulting, Trainin for ClickHouse

Re: ClickHouse, a column-oriented DBMS to generate analytical reports in real time

#30

Post by PeterZaitsev »

georgewfraser wrote: post #6

The basic techniques for implementing a fast column-store data warehouse have been well-known for 10 years. There are several excellent commercial and open-source implementations of these techniques: - BigQuery - Snowflake - Redshift - Presto ClickHouse is not one of them. It doesn't have: - Transactions - Distributed joins - Separate compute from storage - UPDATE - User management I don't mean to be a jerk, I'm just…

There is quite a difference between theoretical technologies and stable high-performance implementation. Majority of things ClickHouse does are very well known it just does them

Here is example Performance comparison we did at Percona https://www.percona.com/blog/2017/02/13/clickhouse-new-opens...

Post reply on HN