Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

github.com

Post reply on HN
Print view View original

31–40 of 88 posts

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#31

Post by mikeshi42 »

atombender wrote: post #28

I'm looking for a new logging solution to replace Kibana. I have very good experience with ClickHouse, and HyperDX looks like a decent UI for it. I'm primarily interested in logs, though, and the existing log shipping pipeline is around Vector on Kubernetes. Admittedly Vector has an OTel sink in beta, but I'm curious if that's the best/fastest way to ship logs, especially given that the original data comes out of app…

Luckily ClickHouse and serious throughput are pretty synonymous. Internally we're at 100+PB of telemetry stored in our own monitoring system.

Vector supports directly writing into ClickHouse - several companies use this at scale (iirc Anthropic does exactly this, they spoke about this recently at our user conference).

Please give it a try and let us know how it goes! Happy to help :)

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#32

Post by atombender »

mikeshi42 wrote: post #31
atombender wrote: post #28

I'm looking for a new logging solution to replace Kibana. I have very good experience with ClickHouse, and HyperDX looks like a decent UI for it. I'm primarily interested in logs, though, and the existing log shipping pipeline is around Vector on Kubernetes. Admittedly Vector has an OTel sink in beta, but I'm curious if that's the best/fastest way to ship logs, especially given that the original data comes out of app…

Luckily ClickHouse and serious throughput are pretty synonymous. Internally we're at 100+PB of telemetry stored in our own monitoring system. Vector supports directly writing into ClickHouse - several companies use this at scale (iirc Anthropic does exactly this, they spoke about this recently at our user conference). Please give it a try and let us know how it goes! Happy to help :)

Thanks! Very familiar with ClickHouse, but can logs then be ingested into CH without going through HyperDX? Doesn't HyperDX require a specific schema that the Vector pipeline would have to adapt the payloads to?

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#33

Post by mikeshi42 »

ensignavenger wrote: post #29

Really interesting, Unfortunately, it looks like HyperDX depends on Mongo? I wonder if there are any open source document stores (possibly a mongo compatible one)( that could work with it?

In theory you should be able to try using FerretDB for example.

We have this on the medium term roadmap to investigate proper support for a compatibilty layer such as ferret or more likely just using ClickHouse itself as the operational data store.

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#34

Post by mikeshi42 »

atombender wrote: post #32
mikeshi42 wrote: post #31

Earlier quoted context omitted.

Luckily ClickHouse and serious throughput are pretty synonymous. Internally we're at 100+PB of telemetry stored in our own monitoring system. Vector supports directly writing into ClickHouse - several companies use this at scale (iirc Anthropic does exactly this, they spoke about this recently at our user conference). Please give it a try and let us know how it goes! Happy to help :)

Thanks! Very familiar with ClickHouse, but can logs then be ingested into CH without going through HyperDX? Doesn't HyperDX require a specific schema that the Vector pipeline would have to adapt the payloads to?

Nope! We're virtually schema agnostic, you can map your custom schema to observability concepts (ex. the SQL expression for TraceID, either a column or a full function/expression will work).

We don't have any lock in to our ingestion pipeline or schema. Of course we optimize a lot for the OTel path, but it works perfectly fine without it too.

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#35

Post by atombender »

mikeshi42 wrote: post #34
atombender wrote: post #32

Earlier quoted context omitted.

Thanks! Very familiar with ClickHouse, but can logs then be ingested into CH without going through HyperDX? Doesn't HyperDX require a specific schema that the Vector pipeline would have to adapt the payloads to?

Nope! We're virtually schema agnostic, you can map your custom schema to observability concepts (ex. the SQL expression for TraceID, either a column or a full function/expression will work). We don't have any lock in to our ingestion pipeline or schema. Of course we optimize a lot for the OTel path, but it works perfectly fine without it too.

That's great to hear. I will take a closer look ASAP.

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#36

Post by ptrfarkas »

mikeshi42 wrote: post #33
ensignavenger wrote: post #29

Really interesting, Unfortunately, it looks like HyperDX depends on Mongo? I wonder if there are any open source document stores (possibly a mongo compatible one)( that could work with it?

In theory you should be able to try using FerretDB for example. We have this on the medium term roadmap to investigate proper support for a compatibilty layer such as ferret or more likely just using ClickHouse itself as the operational data store.

FerretDB maintainer here - we'll be looking at this

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#37

Post by mikeshi42 »

ptrfarkas wrote: post #36
mikeshi42 wrote: post #33

Earlier quoted context omitted.

In theory you should be able to try using FerretDB for example. We have this on the medium term roadmap to investigate proper support for a compatibilty layer such as ferret or more likely just using ClickHouse itself as the operational data store.

FerretDB maintainer here - we'll be looking at this

That'd be awesome! Ferret has been on my radar for a while now :) If you want to chat with us on Discord: https://hyperdx.io/discord

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#38

Post by HatchedLake721 »

mikeshi42 wrote: post #22
readdit wrote: post #2

I like and use HyperDX in production and like it a lot. So kudos to the team for building and merging with Clickhouse. I found a lot of monetary value switching over to HyperDX considering it's significantly more cost efficient for our needs. Should we be starting to prepare for the original HyperDX product to be deprecated and potentially move over to ClickStack?

First off, always really excited to hear from our production users - glad to hear you're getting good value out of the platform! HyperDX isn't being deprecated, you can probably see on the marketing page it's still really prominently featured as an integral part of the stack - so nothing changing there. We do of course want to get users onto HyperDX v2 and the overall ClickStack pattern. This doesn't mean HyperDX is…

Still confused where HyperDX ends and where ClickStack starts.

Is HyperDX === ClickStack?

Is ClickStack = HyperDX + something closed source?

Is ClickStack just a cloud version of HyperDX?

Is it same thing, HyperDX, rebranded as ClickStack?

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#39

Post by cyberax »

phillipcarter wrote: post #13
hosh wrote: post #7

I liked Otel for traces and maybe logging -- but I think the Otel metrics is over-engineered. Does ClickStack have a way to ingest statsd data, preferably with Datadog extensions (which adds tagging)? Does ClickStack offer correlations across traces, logging, and metrics via unified service tagging? Does the UI offer the ability to link to related traces, logging, and metrics? Why does the Elixir sdk use the hyperdx…

> but I think the Otel metrics is over-engineered. What about OTel metrics is difficult? You can set up receivers for other metrics sources like stasd or even the DD agent, so there's no need to immediately replace your metrics stack.

One critical problem for me: no support for raw metrics.

Sometimes, you just want to export ALL of your metrics to the server and let it deal with histograms, aggregation, etc.

Another annoyance is the API, you can't just put "metrics.AddMeasurement('my_metric', 11)", you have to create a `Meter` (which also requires a library name), and then use it.

Re: Show HN: ClickStack – Open-source Datadog alternative by ClickHouse and HyperDX

#40

Post by ensignavenger »

ensignavenger wrote: post #29

Really interesting, Unfortunately, it looks like HyperDX depends on Mongo? I wonder if there are any open source document stores (possibly a mongo compatible one)( that could work with it?

FerretDB looks like a great alternative, thanks! I'll be keeping Ferret and ClickStack on my radar!
Post reply on HN