Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

github.com

Post reply on HN
Print view View original

21–30 of 88 posts

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#21

Post by mikeshi42 »

hosh wrote: post #7

I liked Otel for traces and maybe logging -- but I think the Otel metrics is over-engineered. Does ClickStack have a way to ingest statsd data, preferably with Datadog extensions (which adds tagging)? Does ClickStack offer correlations across traces, logging, and metrics via unified service tagging? Does the UI offer the ability to link to related traces, logging, and metrics? Why does the Elixir sdk use the hyperdx…

Great questions!

OTel Metrics: I get it, it's specified as almost a superset of everyone's favorite metric standards with config for push/pull, monotonic vs delta, exponential/"native" histograms, etc. I have my preferences as well which would be a subset of the standard but I get why a unifying standard needed to be flexible.

Statsd: The great thing about the OTel collector is that it allows ingesting a variety of different data formats, so you can take in statsd and output OTel or write directly to ClickHouse: https://github.com/open-telemetry/opentelemetry-collector-co...

We correlate across trace/span id as well as resource attributes. The correlation across logs/traces with span/trace id is a pretty well worn path across our product. Metrics to the rest is natively done via resource attributes and we primarily expose correlation for K8s-based workloads with more to come. We don't do exemplars _yet_ to solve the more generic correlation case for metrics (though I don't think statsd can transmit exemplars)

Elixir: We try to do our best to support wherever our users are, the OTel SDK and ours have continued to change in parallel over time - we'll want to likely re-evaluate if we should start pointing towards the base OTel SDK for Elixir. We've been pretty early on the OTel SDK side across the board so things continue to evolve, for example our Deno OTel integration came out I think over a year before Deno officially launched one with native HyperDX documentation Notebooks: Yes, it should land in an experimental state shortly, stay tuned :) There's a lot of exciting workflows we're looking to unlock with notebooks as well. If you have any thoughts in this direction, please let me know. I'd love to get more user input ahead of the first release.

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#22

Post by mikeshi42 »

readdit wrote: post #2

I like and use HyperDX in production and like it a lot. So kudos to the team for building and merging with Clickhouse. I found a lot of monetary value switching over to HyperDX considering it's significantly more cost efficient for our needs. Should we be starting to prepare for the original HyperDX product to be deprecated and potentially move over to ClickStack?

First off, always really excited to hear from our production users - glad to hear you're getting good value out of the platform!

HyperDX isn't being deprecated, you can probably see on the marketing page it's still really prominently featured as an integral part of the stack - so nothing changing there.

We do of course want to get users onto HyperDX v2 and the overall ClickStack pattern. This doesn't mean HyperDX is going away by any means - just that HyperDX is focused a lot more on the end-user experience, and we get to leverage the flexibility, learnings and performance of a more exposed ClickHouse-powered core which is the intent of ClickStack. On the engineering side, we're working on making sure it's a smooth path for both open source and cloud.

side note: weird I thought I replied to this one already but I've been dealing with spotty wifi today :)

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#23

Post by mikeshi42 »

Immortalin wrote: post #11

I remember back in the day Mike was building Huggingface before Huggingface was a thing. He was ahead of his time. It's a pity model depot is no longer around.

Wow this is an incredible throwback! Can't believe your memory is this good. It's quite funny and I totally agree - I met the Gradio founders in an accelerator (when they were just getting started) after we shut down ModelDepot - and they of course ended up getting acquired into Hugging Face. It's funny how things end up sometimes :)

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#24

Post by cbhl »

ksec wrote: post #5

It would have even much better if the link was pointing to https://github.com/hyperdxio/hyperdx the actual source code. Because right now without the message on HN here, I wouldn't know what "open source observability stack" meant when the webpage does not explain what HyperDX is, nor does it provide a link to it or its code. I was expecting the whole thing "Open Source Datadog" to be ClickStack Repo inside Clickhous…

Looks like it is pointing there now; old link was https://clickhouse.com/use-cases/observability for posterity

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#26

Post by secondcoming »

user3939382 wrote: post #9

There’s so many of these log aggregators I’ve completely lost track. I used Datadog extensively and found it overpriced and a very confusing UI.

Everyone has found Datadog to be overpriced!

So they switch to Prometheus and Grafana and now have to manage a Prometheus cluster. Far cheaper, but far more annoying.

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#27

Post by mike_heffner »

mikeshi42 wrote: post #14
bilalq wrote: post #12

This is really interesting. Is Clickhouse the only stateful part of this stack? Would love to see compatbility with Rotel[0], a Rust implementation of the OTEL collector, so that this becomes usable for serverless runtime environments. One key thing Datadog has is their own proprietary alternative to the OTEL collector that is much more performant. [0]: https://github.com/streamfold/rotel

I agree - rotel seems like a really good fit for a lightweight lambda integration for OTel, it of course should work already since we stand up an OTel ingest endpoint so it should be seamless to send data over! (Kind of the beauty of OTel of course) I've also been in touch with Mike & Ray for a bit, who've told me they've added ClickHouse support recently which makes the story even better :)

Hi all — one of the authors of Rotel here. Thanks for the kind words, Bilal and Michael.

We're excited to test our Clickhouse integration with Clickstack, as we believe OTel and Clickhouse make for a powerful observability stack. Our open-source Rust OpenTelemetry collector is designed for high-performance, resource-constrained environments. We'd love for you to check it out!

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#28

Post by atombender »

I'm looking for a new logging solution to replace Kibana. I have very good experience with ClickHouse, and HyperDX looks like a decent UI for it.

I'm primarily interested in logs, though, and the existing log shipping pipeline is around Vector on Kubernetes. Admittedly Vector has an OTel sink in beta, but I'm curious if that's the best/fastest way to ship logs, especially given that the original data comes out of apps as plain JSON rather than OTel.

The current system is processing several TB/day and needs fairly serious throughput to keep up.

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#30

Post by mikeshi42 »

codegeek wrote: post #4

How are you different than Signoz, another YC company that also does Observability using clickhouse ?

Echoing the comment below, I guess one obvious thing is that we are a team at ClickHouse and an official first-party product on top. That translates into:

- We're flexible on top of any ClickHouse instance, you can use virtually any schema in ClickHouse and things will still work. Custom schemas are pretty important for either tuned high performance or once you're at a scale like Anthropic. This makes it also incredibly easy to get started (especially if you already have data in ClickHouse). - The above also means you don't need to buy into OTel. I love OTel but some companies choose to use Vector, Cribl, S3, a custom writing script, etc for good reasons. All of that is supported natively due to the various ClickHouse integrations, and naturally means you can use ClickStack/HyperDX in that scenario as well. - We also have some cool tools around wrangling telemetry at scale, from Event Deltas (high cardinality correlation between slow spans and normal spans to root cause issues) to Event Patterns (clustering similar logs or spans together automatically with ML) - all of these help users dive into their data in easier ways than just searching & charting. - We also have session replay capability - to truly unify everything from click to infra metrics.

We're built to work at the 100PB+ scale we run internally here for monitoring ClickHouse Cloud, but flexible enough to pin point specific user issues that get brought up once in a support case in an end-to-end manner.

There's probably a lot more I'm missing. Ultimately from a product philosophy standpoint, we aren't big believers in the "3 pillars" concept, which tends to manifest as 3 silos/tabs for "logs", "metrics", "traces" (this isn't just Signoz - but across the industry). I'm a big believer that we're building tools to unify and centralize signals/clues in one place and giving the right datapoint at the right time to the engineer. During an incident I just think about what's the next clue I can get to root cause an issue, not if I'm in the logging product or the tracing product.

Post reply on HN