what is the difference with http://github.com/apache/doris
Druid: A Real-Time Analytical Data Store
11–20 of 30 posts
Re: Druid: A Real-Time Analytical Data Store
#12My sense from the few projects I've seen attempt to use Druid is that there is quite a lot of infrastructure overhead / DevOps support required to manage a cluster at scale, and that fairly complex ingestion pipelines are required to load the data in the right format. Anecdotally, I've heard that ClickHouse is easier to deploy from this perspective with similar performance, but would love to get others views / experi…
Several years back, I was running it on a single server along with Kafka ( I was requested to keep everything related to analytics to a single huge server ) and it started with a fight for zookeeper between these two. While it worked quite well thereafter, keeping it up was a battle.
Maybe situation is better with clusters and Kafka moving away from zookeeper.
Re: Druid: A Real-Time Analytical Data Store
#13What other options are there of this type? ClickHouse — anything else?
Re: Druid: A Real-Time Analytical Data Store
#14My sense from the few projects I've seen attempt to use Druid is that there is quite a lot of infrastructure overhead / DevOps support required to manage a cluster at scale, and that fairly complex ingestion pipelines are required to load the data in the right format. Anecdotally, I've heard that ClickHouse is easier to deploy from this perspective with similar performance, but would love to get others views / experi…
We found the opposite. Setting up Druid clusters is the easiest compared to its competitors: Clickhouse and Pinot.
Especially since ingestion goes straight to S3. We don’t really worry about backups (just deal with PG backups).
Just make sure your ZK is happy and all will be well.
The hard part about Druid is tuning:
- the ingestions: Spec definition, compaction, sharding strategy, RAM consumptions, etc.
- and query performance: RAM consumptions, number of threads, timeouts, etc.
Re: Druid: A Real-Time Analytical Data Store
#15What other options are there of this type? ClickHouse — anything else?
maybe doris is good （http://github.com/apache/doris)
join support is bad for clickhouse. For apache doris, performance and join is better than clickhouse.
Re: Druid: A Real-Time Analytical Data Store
#16My sense from the few projects I've seen attempt to use Druid is that there is quite a lot of infrastructure overhead / DevOps support required to manage a cluster at scale, and that fairly complex ingestion pipelines are required to load the data in the right format. Anecdotally, I've heard that ClickHouse is easier to deploy from this perspective with similar performance, but would love to get others views / experi…
We found the opposite. Setting up Druid clusters is the easiest compared to its competitors: Clickhouse and Pinot. Especially since ingestion goes straight to S3. We don’t really worry about backups (just deal with PG backups). Just make sure your ZK is happy and all will be well. The hard part about Druid is tuning: - the ingestions: Spec definition, compaction, sharding strategy, RAM consumptions, etc. - and query…
How is the documentation for this deployment and tuning? Last time I checked I had the impression that anything about the dozen of different node types weren't very clear, not to mention details about the ingestion were all over the place.
And it was hard to find examples of configuration, ingestion other than basic tutorials
Re: Druid: A Real-Time Analytical Data Store
#17My sense from the few projects I've seen attempt to use Druid is that there is quite a lot of infrastructure overhead / DevOps support required to manage a cluster at scale, and that fairly complex ingestion pipelines are required to load the data in the right format. Anecdotally, I've heard that ClickHouse is easier to deploy from this perspective with similar performance, but would love to get others views / experi…
I feel like it shares a lot of the complexity with the rest of Hadoop-adjacent products. If your company already manages standard Hadoop infra, it's probably not too different, otherwise it seems a quite bumpy road.
Re: Druid: A Real-Time Analytical Data Store
#18Earlier quoted context omitted.
We found the opposite. Setting up Druid clusters is the easiest compared to its competitors: Clickhouse and Pinot. Especially since ingestion goes straight to S3. We don’t really worry about backups (just deal with PG backups). Just make sure your ZK is happy and all will be well. The hard part about Druid is tuning: - the ingestions: Spec definition, compaction, sharding strategy, RAM consumptions, etc. - and query…
How is the documentation for this deployment and tuning? Last time I checked I had the impression that anything about the dozen of different node types weren't very clear, not to mention details about the ingestion were all over the place. And it was hard to find examples of configuration, ingestion other than basic tutorials
Docs definitely have rooms for improvement.
Architecturally, It is easier to visualize this two big group:
- query serving: coordinator, historical, broker
- ingestion: overlord, middlemanager
router unifies all of Druid API together.
I would start with the Helm chart to get some basic idea on tunings.
Re: Druid: A Real-Time Analytical Data Store
#19My sense from the few projects I've seen attempt to use Druid is that there is quite a lot of infrastructure overhead / DevOps support required to manage a cluster at scale, and that fairly complex ingestion pipelines are required to load the data in the right format. Anecdotally, I've heard that ClickHouse is easier to deploy from this perspective with similar performance, but would love to get others views / experi…
We found the opposite. Setting up Druid clusters is the easiest compared to its competitors: Clickhouse and Pinot. Especially since ingestion goes straight to S3. We don’t really worry about backups (just deal with PG backups). Just make sure your ZK is happy and all will be well. The hard part about Druid is tuning: - the ingestions: Spec definition, compaction, sharding strategy, RAM consumptions, etc. - and query…
[deleted]
Re: Druid: A Real-Time Analytical Data Store
#20My sense from the few projects I've seen attempt to use Druid is that there is quite a lot of infrastructure overhead / DevOps support required to manage a cluster at scale, and that fairly complex ingestion pipelines are required to load the data in the right format. Anecdotally, I've heard that ClickHouse is easier to deploy from this perspective with similar performance, but would love to get others views / experi…
We found the opposite. Setting up Druid clusters is the easiest compared to its competitors: Clickhouse and Pinot. Especially since ingestion goes straight to S3. We don’t really worry about backups (just deal with PG backups). Just make sure your ZK is happy and all will be well. The hard part about Druid is tuning: - the ingestions: Spec definition, compaction, sharding strategy, RAM consumptions, etc. - and query…
> Just make sure your ZK is happy and all will be well.
That sounds like the opposite of easy to setup (and maintain).