Live data from Hacker News

M3DB, a distributed timeseries database

m3db.io

41–50 of 138 posts

Re: M3DB, a distributed timeseries database

#41
post #5

Earlier quoted context omitted.

It's opensource. Why should Uber give any guarantees? They are not in the business of selling software. Unless Uber is actively blocking contributions, it's not Uber's fault if no community formed around something they opensourced. As for this being a PR piece, they could have achieved the same with just a detailed blog post and no code. It looks like a expensive PR piece if they have to opensource work that took pro…

I agree with everything you say. But without any certainty around the roadmap, support, and longterm commitment by Uber to maintain these projects, they're nothing more than interesting repos amongst a sea of interesting repos. The way Uber brands them suggests that they're suitable for use in production environments, but so far that hasn't been the case with anything they open sourced outside a narrow envelope that…

Ignore the project and move on? If you expect every open source project to be served by all your entitlements, you will be repeatedly disappointed.

Re: M3DB, a distributed timeseries database

#42
post #35

Earlier quoted context omitted.

A time series database is specialized for use cases where the data and query patterns are solely temporal in nature and must show the latest data in real-time (performance metrics/monitoring and stock prices come to mind). Relational and NoSQL databases tend to degrade rapidly with these query patterns at scale (think of the complexity of SQL queries to bucket rows by timestamp). https://en.m.wikipedia.org/wiki/Time_…

Note that temporal databases are also a thing, so it's probably wise to avoid using the word "temporal" when discussing time series databases. As far as I know kdb+ is the only technology that has a foot in both camps. https://en.m.wikipedia.org/wiki/Temporal_database

> As far as I know kdb+ is the only technology that has a foot in both camps.

Teradata Vantage also supports both. And you're absolutely right, it's important not to conflate "temporal" and "time series" support.

Re: M3DB, a distributed timeseries database

#43
post #5

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

It's opensource. Why should Uber give any guarantees? They are not in the business of selling software. Unless Uber is actively blocking contributions, it's not Uber's fault if no community formed around something they opensourced. As for this being a PR piece, they could have achieved the same with just a detailed blog post and no code. It looks like a expensive PR piece if they have to opensource work that took pro…

Everything you say is true, but tossing useless code releases over the wall isn't really participating in the open source community, either.

It looks to me like maybe their engineers internally are fans of the idea of "open source", and the PR department is happy to try and get some good press out of it, but the company culture isn't really set up to develop in public or maintain these things they've nominally "released".

Sadly, this isn't unusual among tech companies, but it'd be more obvious what's happening if they just put up a bare-bones FTP with a README: "Here's some code under . Use it at your own risk."

Re: M3DB, a distributed timeseries database

#44

Earlier quoted context omitted.

I agree with everything you say. But without any certainty around the roadmap, support, and longterm commitment by Uber to maintain these projects, they're nothing more than interesting repos amongst a sea of interesting repos. The way Uber brands them suggests that they're suitable for use in production environments, but so far that hasn't been the case with anything they open sourced outside a narrow envelope that…

>It seems like Uber had too big of an engineering department with too little work to do, so they started reinventing wheels. Which is cool if they're willing to support them in the long term, but so far that hasn't proven to be the case. Former Uber engineer here. I can assure you that while our engineering team was massive, there was anything but too little work. If anything most engineers were massively overtaxed.…

Uber is in the process of ditching uChat and moving to Slack

Re: M3DB, a distributed timeseries database

#46
post #5

Uber has started many projects that ended up getting open sourced. And many of them are now either abandoned or on life support. H3 comes to mind as something we almost ended up using but luckily avoided. These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.

It's opensource. Why should Uber give any guarantees? They are not in the business of selling software. Unless Uber is actively blocking contributions, it's not Uber's fault if no community formed around something they opensourced. As for this being a PR piece, they could have achieved the same with just a detailed blog post and no code. It looks like a expensive PR piece if they have to opensource work that took pro…

Even I have half a mind about releasing something that I cannot commit to for a little bit - the initial bugfix stage, at least. I would say a giant like Uber hurts themselves more in terms of PR when not being conservative enough about putting source out there. People inevitably gravitate towards big player "open sauce" as it implies some commitment.

A company should first put their own system through hell and decide that "yeah, this is good, we are sticking with this", before luring people to use it.

Re: M3DB, a distributed timeseries database

#47
post #5

Earlier quoted context omitted.

It's opensource. Why should Uber give any guarantees? They are not in the business of selling software. Unless Uber is actively blocking contributions, it's not Uber's fault if no community formed around something they opensourced. As for this being a PR piece, they could have achieved the same with just a detailed blog post and no code. It looks like a expensive PR piece if they have to opensource work that took pro…

I agree with everything you say. But without any certainty around the roadmap, support, and longterm commitment by Uber to maintain these projects, they're nothing more than interesting repos amongst a sea of interesting repos. The way Uber brands them suggests that they're suitable for use in production environments, but so far that hasn't been the case with anything they open sourced outside a narrow envelope that…

> The way Uber brands them suggests that they're suitable for use in production environments, but so far that hasn't been the case with anything they open sourced outside a narrow envelope that resembles their own operating model.

Not a contradiction. Many of these tools are suitable for use in production, almost by definition, since they are being used in production, at Uber. They might or might not work in your environment out of the box. But they are certainly often likely a better starting point than an empty editor, even when they do not. Most of the ones I am familiar with, are happy to get PRs generalizing them to more varied environments.

> they're nothing more than interesting repos amongst a sea of interesting repos.

As someone who has open-sourced on GitHub: research prototypes hacked together for a research paper deadline in grad school, class projects, for-fun hacks, and also production tooling I built as a paid engineer, I'd say there is a big difference! :) And there would still be a big difference even if the later were somehow never touched again after the first "we are open-sourcing this!" commit.

That said, we do try to maintain the things we open-source. Standards of support vary because individuals maintaining these projects, and their situations, vary. This is true for non-OSS internal tools too. In my experience, having gone through the Uber OSS process twice, and having started it a third time and decided against releasing (yet?), Uber does try to make reasonably sure that it's open-sourcing stuff that will be useful and is planned to be maintained. At the same time, they have to balance it with making it easy to open-source tools, otherwise too many useful things would remain internal only.

Also, note, some of these tools have exactly one developer internally as the maintainer, and not even as their full time job. For example, I am the sole internal maintainer[1] for https://github.com/uber/NullAway and also have 3-4 other projects internally on my plate, most of which are in earlier stages and need more frequent attention[2]. If and when said developer leaves, effort is made to find a new owner. This is not always successful, particularly if the tool has become non-critical internally. Sometimes, leaving owners retain admin rights on the repos and keep working on the tool (Manu, NullAway's original author, co-maintains it), but I don't think anyone is suggesting that that should be an obligation.

Finally, obviously, nothing here is the official Uber position on anything, just my own personal observations. This doesn't represent my employer, and so on. I am also pretty sure most of this is not even Uber specific :)

[1] Not the only internal contributor! Also, there is one external maintainer, as mentioned a few sentences later. But in terms of this being anyone's actual responsibility...

[2] Just to clarify, I think between Manu's interest, my own, and it being relatively critical tooling at Uber, NullAway is pretty well maintained. But I can understand why that isn't always a given for all projects.

Re: M3DB, a distributed timeseries database

#48
post #9

Does "Time Series Database" mean anything technical, or is this just some Uber marketing? In statistics, time series has a technical meaning.

TSDBs are a special case of databases. And oh boy, time-series is hard.

Your regular RDBMS is going to be either write-heavy or read-heavy. You can pretty easily[ß] optimise the database for one of these utilisation patterns. But a TSDB basically combines the worst of both worlds: telemetry at any scale is important, and monitoring reliability in an always-online system is not optional.

TSDBs are written to very frequently; even at a reasonably low scale we could be talking about couple of hundred thousand writes every few seconds. But because they are also used for system-wide monitoring, they are read from all the time.

ß: a read-heavy regular DB has the ratio of reads:writes in thousands, perhaps millions; a write-heavy DB can be read from a couple of times every few seconds, but can be written to at a rate of tens of thousands of entries per second. You - or your expensive DBA - can optimise the DB for one of these patterns, but not for both. TSDBs have to support both patterns at the same time, so their internals have been geared to this one specific domain.

Re: M3DB, a distributed timeseries database

#49

Earlier quoted context omitted.

>It seems like Uber had too big of an engineering department with too little work to do, so they started reinventing wheels. Which is cool if they're willing to support them in the long term, but so far that hasn't proven to be the case. Former Uber engineer here. I can assure you that while our engineering team was massive, there was anything but too little work. If anything most engineers were massively overtaxed.…

Uber is in the process of ditching uChat and moving to Slack

I accidentally left this point out. In retrospect it's easy to say Uber made the wrong decision to make uChat but it was one of few options at the time.

Re: M3DB, a distributed timeseries database

#50

admins/mods: this needs an apostrophe to turn it into "Uber's M3DB". For anyone who's never heard of M3DB, and lives in a place where Uber doesn't operatore or is even banned (and so isn't part of daily life or conversation) "Ubers" might just as easily be some db researcher affiliated with the university of who knows where showing off something they came up with last summer and got a grant for.

Fixed, thanks
Post reply on HN