Live data from Hacker News

M3DB, a distributed timeseries database

m3db.io

61–70 of 138 posts

Re: M3DB, a distributed timeseries database

#61
post #38

Earlier quoted context omitted.

Clickhouse is an analytic column-based RDBMS. It's not a timeseries database. Each class of product is used to solve different problems.

Clickhouse works exceptionally well as a TSDB.

That is also my experience. Also in bencmark it is almost as fast as GPU analytical DBs or KDB+.

Re: M3DB, a distributed timeseries database

#62

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

Huh? The last commit to the H3 github repo was 3 days ago. In what sense is it abandoned? Genuinely interested as we are considering using it as a core library.

Re: M3DB, a distributed timeseries database

#63

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

We're definitely still working on H3. We just got a nice, new domain: h3geo.org

We're also basically done with a new Python wrapper written in Cython. https://github.com/uber/h3-py/tree/cython

We could probably use some help with the last step of packaging, if anyone is interested!

Re: M3DB, a distributed timeseries database

#64

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

https://github.com/facebookarchive - 11 Pages of unsupported open source software.

Beringei, a TSDB, (https://github.com/facebookarchive/beringei) in particular with what you are saying was a PR piece (https://engineering.fb.com/core-data/beringei-a-high-perform...) since it was never really used by anyone outside of FB.

I really don't see the negative part of free code that you can learn from and/or incorporate at all.

Re: M3DB, a distributed timeseries database

#65

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

https://github.com/facebookarchive - 11 Pages of unsupported open source software. Beringei, a TSDB, ( https://github.com/facebookarchive/beringei ) in particular with what you are saying was a PR piece ( https://engineering.fb.com/core-data/beringei-a-high-perform... ) since it was never really used by anyone outside of FB. I really don't see the negative part of free code that you can learn from and/or incorporate…

Free code is good, yes.

When you rely on the systems themselves, and do so with expectation of support from the originating company, your expectations will almost certainly be broken.

I think that's the simplest takeaway - not to run away from any open-sourced project, but to take into proper consideration if/how they plan on supporting the tool, and how much you would be capable of adapting and owning yourself if the worst happened.

Re: M3DB, a distributed timeseries database

#66

Earlier quoted context omitted.

https://github.com/facebookarchive - 11 Pages of unsupported open source software. Beringei, a TSDB, ( https://github.com/facebookarchive/beringei ) in particular with what you are saying was a PR piece ( https://engineering.fb.com/core-data/beringei-a-high-perform... ) since it was never really used by anyone outside of FB. I really don't see the negative part of free code that you can learn from and/or incorporate…

Free code is good, yes. When you rely on the systems themselves, and do so with expectation of support from the originating company, your expectations will almost certainly be broken. I think that's the simplest takeaway - not to run away from any open-sourced project, but to take into proper consideration if/how they plan on supporting the tool, and how much you would be capable of adapting and owning yourself if th…

Yeah exactly. If one wants support they can pay for it i.e. SaaS if available. If an open source project has not created a contract with any user then there is no guarantee of support. I don't believe any company creates a contract with users automatically because they made source code available. That is unsustainable.

Chronosphere is the SaaS part for M3DB in this case. The negativity around someone open sourcing code for PR is nuts especially when all of the code is available. I love reading the code and getting ideas about how things work.

Re: M3DB, a distributed timeseries database

#67

Earlier quoted context omitted.

Uber is in the process of ditching uChat and moving to Slack

I accidentally left this point out. In retrospect it's easy to say Uber made the wrong decision to make uChat but it was one of few options at the time.

Seems like a huge point to leave out.

Are you affiliated with Uber?

Re: M3DB, a distributed timeseries database

#69
post #45

How does it compare to TimescaleDB [1] ? [1] https://www.timescale.com

TimescaleDB co-founder here.

TimescaleDB is a more versatile time-series database. It supports a variety of datatypes (text, ints, floats, arrays, json), allows for out-of-order writes and backfilling of old data, supports full SQL, JOINs between tables (eg for metadata), flexible continuous aggregates, native compression, and is backed by the reliability of Postgres. [0]

M3DB seems much more limited in scope [1]:

"Current Limitations

Due to the nature of the requirements for the project, which are primarily to reduce the cost of ingesting and storing billions of timeseries and providing fast scalable reads, there are a few limitations currently that make M3DB not suitable for use as a general purpose time series database.

The project has aimed to avoid compactions when at all possible, currently the only compactions M3DB performs are in-memory for the mutable compressed time series window (default configured at 2 hours). As such out of order writes are limited to the size of a single compressed time series window. Consequently backfilling large amounts of data is not currently possible.

The project has also optimized the storage and retrieval of float64 values, as such there is no way to use it as a general time series database of arbitrary data structures just yet."

[0] https://www.timescale.com/

[1] https://m3db.github.io/m3/m3db/#current-limitations

Re: M3DB, a distributed timeseries database

#70
post #38

Earlier quoted context omitted.

Clickhouse is an analytic column-based RDBMS. It's not a timeseries database. Each class of product is used to solve different problems.

Clickhouse works exceptionally well as a TSDB.

While this is true, for a metrics workload it does not work great I have both seen and heard from others, mainly due to the fact it does not have an inverted index - so finding a small subset of metrics in a dataset of billions of metrics ends up taking significant time due to the scan required to find the timeseries matching the arbitrary number of dimensions specified to find the timeseries you're looking for.

If you're building it with a specific application and a concrete schema you can create which will result in fast queries and don't have requirements for arbitrary dimensions being specified for lookup, then yes it's great as a TSDB.

Prometheus, M3DB, etc all use an inverted index alongside the column store TSDB to help with metrics workloads.

Post reply on HN