Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

M3DB, a distributed timeseries database

m3db.io

Post reply on HN
Print view View original

101–110 of 138 posts

Re: M3DB, a distributed timeseries database

#101

Post by akulkarni »

manigandham wrote: post #92
bdcravens wrote: post #33

Earlier quoted context omitted.

Clickhouse is an analytic column-based RDBMS. It's not a timeseries database. Each class of product is used to solve different problems.

Time-series data is just data where every record has a "time" field. That's it. Any database can handle it, and columnstore RDBMS are designed to store and query trillion-row tables with full SQL functionality. The only advantage a "time-series" database gives you is some time-based query operators (like gap filling, last value, smoothing, etc). Those are now being added to SQL support for RDBMS so there's really not…

"Time-series data is just data where every record has a "time" field. That's it."

That's a pretty big simplification. It's like saying one could go running in dress shoes. (Yes, it's possible, but don't you want to use the right tool for the job?)

As one time-series database example, because TimescaleDB [0] is focused on time-series data, its users benefit from [1]:

* 40-50x compression for metrics data (so storage costs for compressed data are 2-2.5% what they would normally be)

* Versatile continuous aggregate policies

* Variable data retention policies

* Overall much more efficient compute and memory utilization (because of faster insert and query rates)

* And yes, also time-based query operators for gap filling, first/last, LOCF, etc

(And I'm sure roskilli could describe M3DB's own advantages over non-time-series DBs.)

[0] Disclaimer to other readers, I'm a co-founder (although OP already knows this, as we've jousted on HN before :-) )

[1] All of our benchmarks and other engineering notes are published here: https://blog.timescale.com/tag/engineering/

Re: M3DB, a distributed timeseries database

#102

Post by manigandham »

akulkarni wrote: post #101
manigandham wrote: post #92

Earlier quoted context omitted.

Time-series data is just data where every record has a "time" field. That's it. Any database can handle it, and columnstore RDBMS are designed to store and query trillion-row tables with full SQL functionality. The only advantage a "time-series" database gives you is some time-based query operators (like gap filling, last value, smoothing, etc). Those are now being added to SQL support for RDBMS so there's really not…

"Time-series data is just data where every record has a "time" field. That's it." That's a pretty big simplification. It's like saying one could go running in dress shoes. (Yes, it's possible, but don't you want to use the right tool for the job?) As one time-series database example, because TimescaleDB [0] is focused on time-series data, its users benefit from [1]: * 40-50x compression for metrics data (so storage c…

I'd say you started with a standard RDBMS and added the other features like sharding, column-oriented storage, time-series helper functions to SQL, and more to it to end up similar to the other native column stores but with a more flexible and popular Postgres frontend.

At the very least, I think we both agree that the relational/SQL options work just fine compared to limited time-series databases like Influx. And for the record, we do use timescale so you've won me over on the PG usability front.

Re: M3DB, a distributed timeseries database

#103

Post by roskilli »

sciurus wrote: post #100
cube2222 wrote: post #82

Earlier quoted context omitted.

Here you have the specifics: https://m3db.github.io/m3/m3db/architecture/engine/ I admit I’ve exaggerated a bit as Prometheus doesn’t support downsampling, in m3db I only keep 2 weeks of data at full resolution, 2 months at lower, and 5 years at even lower.

Downsampling might not save as much space as you think depending on how m3db works. https://github.com/thanos-io/thanos/issues/813 goes in to why for a similar project.

Whether it saves space or not, looking at metrics over period of months or years when the data is raw is far more slow/expensive than looking at downsampled data.

If you still want to be able to quickly graph and view old data, downsampling is the only way to keep your queries interactive.

Take for instance 30s data vs 10min data. 20x more computation, network exchange and everything else of that nature needs to happen.

Also if you want to keep only a subset of your data for a very long time, you need to have retention policies - otherwise you end up storing all that extra data forever.

At large numbers (terabytes to petabytes) this stuff is impactful, at smaller numbers (gigabytes) my points here are far less relevant.

Re: M3DB, a distributed timeseries database

#104

Post by akulkarni »

manigandham wrote: post #102
akulkarni wrote: post #101

Earlier quoted context omitted.

"Time-series data is just data where every record has a "time" field. That's it." That's a pretty big simplification. It's like saying one could go running in dress shoes. (Yes, it's possible, but don't you want to use the right tool for the job?) As one time-series database example, because TimescaleDB [0] is focused on time-series data, its users benefit from [1]: * 40-50x compression for metrics data (so storage c…

I'd say you started with a standard RDBMS and added the other features like sharding, column-oriented storage, time-series helper functions to SQL, and more to it to end up similar to the other native column stores but with a more flexible and popular Postgres frontend. At the very least, I think we both agree that the relational/SQL options work just fine compared to limited time-series databases like Influx. And fo…

"And for the record, we do use timescale so you've won me over on the PG usability front."

Great! Didn't realize that.

And yes, I would agree that a relational/SQL time-series database like TimescaleDB can work quite well. :-)

Re: M3DB, a distributed timeseries database

#105

Post by roskilli »

linuxhansl wrote: post #99
buro9 wrote: post #12

Earlier quoted context omitted.

It's a database for a metric platform. Think of OpenTSDB and Prometheus. Or for a better comparison think of Thanos https://thanos.io/ As to whether they could fulfil Uber's needs, the thing about scale (real massive scale - I work at Cloudflare) is that everything breaks in weird ways according to your specific uses of a technology. The things listed above work for companies, until they don't. There's few things tha…

We run OpenTSDB (which stores its data in Apache HBase) at scale. 150m-200m events/minute and about 20-30 trillion (10^12) events stored. Doubling about every 12-18 months or so. While it's true that things start to creak at scale, this has worked remarkably well for us so far. I doubt M3DB is somehow magical in this regard.

M3DB ingested 30 million datapoints per second (so 1.8 billion per minute) with each node writing hundreds of thousands of writes per second. The dataset was in the petabytes.

For us the cost savings vs OpenTSDB (millions of dollars of hardware), the faster query time and the reduction in oncall overhead was worthwhile.

Re: M3DB, a distributed timeseries database

#106

Post by bfung »

richieartoul wrote: post #4
missosoup wrote: post #2

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

Chronosphere, a startup founded by two of the early M3 engineers, just raised 11 million dollars to build a monitoring platform based around M3DB: https://techcrunch.com/2019/11/05/chronosphere-launches-with... Uber also uses M3DB extensively internally and the project is nowhere near being abandoned or on life support: https://github.com/m3db/m3/commits/master

Yep, another way to pre-market/pre-signal to investors.

A pretty common story these days:

1. I built an X to solve Unicorn U’s problem.

2. Open source it, give talks on it

3. Leave Unicorn U and start a company based on X

4. ...

5. Profit(???)

Also, debatable whether or not Unicorn U actually needed a freshly built X instead of using existing tools/tech.

Re: M3DB, a distributed timeseries database

#107

Post by rixed »

idjango wrote: post #73
aeyes wrote: post #38

Earlier quoted context omitted.

Clickhouse works exceptionally well as a TSDB.

I also confirm that. Several companies have successfully transitioned their monitoring stack from graphite initial python implementation to a clickhouse based backend.

Not to bad-mouth Clickhouse but the original python implementation of graphite + carbon was setting the bar very low, though, and transitioning from there to anything would have increasing performances by orders of magnitude.

Re: M3DB, a distributed timeseries database

#108

Post by shaklee3 »

roskilli wrote: post #105
linuxhansl wrote: post #99

Earlier quoted context omitted.

We run OpenTSDB (which stores its data in Apache HBase) at scale. 150m-200m events/minute and about 20-30 trillion (10^12) events stored. Doubling about every 12-18 months or so. While it's true that things start to creak at scale, this has worked remarkably well for us so far. I doubt M3DB is somehow magical in this regard.

M3DB ingested 30 million datapoints per second (so 1.8 billion per minute) with each node writing hundreds of thousands of writes per second. The dataset was in the petabytes. For us the cost savings vs OpenTSDB (millions of dollars of hardware), the faster query time and the reduction in oncall overhead was worthwhile.

Hundreds of thousands per second isn't very high when you compare that to clickhouse or kdb+.

Re: M3DB, a distributed timeseries database

#109

Post by shaklee3 »

akulkarni wrote: post #101
manigandham wrote: post #92

Earlier quoted context omitted.

Time-series data is just data where every record has a "time" field. That's it. Any database can handle it, and columnstore RDBMS are designed to store and query trillion-row tables with full SQL functionality. The only advantage a "time-series" database gives you is some time-based query operators (like gap filling, last value, smoothing, etc). Those are now being added to SQL support for RDBMS so there's really not…

"Time-series data is just data where every record has a "time" field. That's it." That's a pretty big simplification. It's like saying one could go running in dress shoes. (Yes, it's possible, but don't you want to use the right tool for the job?) As one time-series database example, because TimescaleDB [0] is focused on time-series data, its users benefit from [1]: * 40-50x compression for metrics data (so storage c…

It seems clickhouse also has most of those features, but is not considered a time series database. Is that wrong?

Re: M3DB, a distributed timeseries database

#110

Post by rixed »

bostik wrote: post #48
clircle wrote: post #9

Does "Time Series Database" mean anything technical, or is this just some Uber marketing? In statistics, time series has a technical meaning.

TSDBs are a special case of databases. And oh boy, time-series is hard . Your regular RDBMS is going to be either write-heavy or read-heavy. You can pretty easily[ß] optimise the database for one of these utilisation patterns. But a TSDB basically combines the worst of both worlds: telemetry at any scale is important, and monitoring reliability in an always-online system is not optional. TSDBs are written to very fre…

> But because they are also used for system-wide monitoring, they are read from all the time.

And this is the billion dollars mistake of the current devop culture. I believe we are doing monitoring wrong. Real time monitoring need no persistent storage. Troubleshooting does need persistent storage, but not monitoring, and unless your infrastructure is broken all the time then querying past data must occur only rarely.

From what i have seen, this mistake seems to stem from the web culture that tends to favor designs centered on a database. Whereas you want your real-time monitoring to be centered on stream processing, with one output to the persistent store for later retrieval.

There is no good reason for your dashboards nor your alerts to hit a database every few minutes, this design is just wrong.

I'm actually working on the prototype of a stream processor tailored at small scale network monitoring and would welcome any discussion/criticism on this topic. Notice that "small scale" for a stream processor is much larger than "small scale" for a database, and that i believe a good stream processor capable of running arbitrary persistent queries for monitoring on the order of a million data points per second should fit a single server and be more than enough to monitor an above the average sized business infrastructure.

Post reply on HN