Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on Uber’s List of Postgres Limitations

blog.2ndquadrant.com

Post reply on HN
Print view View original

61–70 of 108 posts

Re: Thoughts on Uber’s List of Postgres Limitations

#61

Post by [deleted] »

Illniyar wrote: post #18
leftnode wrote: post #14

Earlier quoted context omitted.

Can you expand on what you mean by a single table with manual indexes? How does a frequently used table with indexes not resemble an RDBMS?

the secondary indexes are saved in a different table and (I can only assume) the application layer (or one layer above mySql) is responsible for keeping that table up to date. If you use only 1 table, then it's by no means "relational", so I don't see why you'll need a database system designed from start to finish to support a relational model.

[deleted]

Re: Thoughts on Uber’s List of Postgres Limitations

#62

Post by MustardTiger »

paulddraper wrote: post #57
majewsky wrote: post #30

Earlier quoted context omitted.

> static page cache behind cdn I wouldn't want my page behind a CDN. CDNs make users much more trackable across sites. My point isn't that CDNs are bad for everyone. My point is, once more, that most questions are not as simple as they may appear.

> CDNs make users much more trackable across sites. How does has CDN do this in a way that a "regular" web deployment wouldn't?

The same CDN is serving thousands of sites, so they can track a ton of what users do. CDNs are perfectly placed to capture and sell user stats.

Re: Thoughts on Uber’s List of Postgres Limitations

#63

Post by brianwawok »

ahachete wrote: post #32
threeseed wrote: post #26

Earlier quoted context omitted.

> Which is really just what every company at these scales do No they aren't. You need to understand that databases are being used in a whole range of different ways than you think. Batch/streaming analytics are typically done with HDFS, Cassandra, HBase, MongoDB etc. Event aggregation often with a time series database like Druid, InfluxDB etc. Web API serving layers are still the domain of lightweight SQL databases l…

While I mostly agree with your comment, some NoSQL databases are not suitable for analytics. Indeed, they are terrible at them. Unstructured information is, surprise, unstructured, which means it is harder to query / analyze, since the structure needs to be fully scanned and parsed to perform the computation. I have seen many cases where MongoDB is used for this and suffers from really bad performance.

I think the trap is simple POCS.

Do a simple POC where you ram 1 million rows in a MongoDB and then make a webapp to do some basic analytics. Look, it works and get responses within a second. Cool!

Then real data comes in at 12 billion rows and your analytics take 3 hours to run.

So you try to do the sharding thing, and realize that it works for a while, except now every analytics query needs to hit every shard....

Re: Thoughts on Uber’s List of Postgres Limitations

#64

Post by Illniyar »

laszlokorte wrote: post #51
Illniyar wrote: post #18

Earlier quoted context omitted.

the secondary indexes are saved in a different table and (I can only assume) the application layer (or one layer above mySql) is responsible for keeping that table up to date. If you use only 1 table, then it's by no means "relational", so I don't see why you'll need a database system designed from start to finish to support a relational model.

> If you use only 1 table, then it's by no means "relational" [...] FYI The term "relational" in "relational database" does not refer to the relation between tables but to the mathematical concept of a relation which is some set of tuples ie a single table.

Well, the relational model does not describe or contain a notion of table really, so I guess you could model it in any way you want, even within a single table.

But I think that's a mute point, I doubt they kept any part of the relational model inside their schemaless database.

Re: Thoughts on Uber’s List of Postgres Limitations

#65

Post by Illniyar »

pas wrote: post #28
Illniyar wrote: post #18

Earlier quoted context omitted.

the secondary indexes are saved in a different table and (I can only assume) the application layer (or one layer above mySql) is responsible for keeping that table up to date. If you use only 1 table, then it's by no means "relational", so I don't see why you'll need a database system designed from start to finish to support a relational model.

The "relationalness" is/was not really important here. It's all about MVCC and how storage engines handle it. Postgres is lacking in these scenarios, whereas a particular fine tuned version (or fork) of InnoDB (or MyRocks or whatever they end up chosing) handles this better. See Facebook's "mysql-5.6" branch, that has hundreds of patches piled on to support especially these taxing workloads.

Well, for that matter, as I understood it, they also have no transactions nor atomicity and are basically eventually consistant , though these I'm simply inferring from their posts, it was not stated as far as I can remember.

So MVCC also has almost zero bearing on what they are doing.

Re: Thoughts on Uber’s List of Postgres Limitations

#66

Post by arnarbi »

late2part wrote: post #52
arnarbi wrote: post #34

Earlier quoted context omitted.

That second point is a really unnecessarily belittling straw man, and I think such comments are counterproductive to the discussion.

I disagree. When you put yourself on stage to criticize another, you open yourself up to criticism. And in this case, his little blog didn't scale, and it was rightfully pointed out. It's not terribly nice, but the comment is true, and relevant.

Criticism is good. Straw men are not ("your opinions and arguments on dbs are irrelevant because your blog fell over"), and being holier-than-thou about it only makes people stop listening to your actual criticism. That's why I think it's unproductive.

Re: Thoughts on Uber’s List of Postgres Limitations

#67

Post by duaneb »

fauigerzigerk wrote: post #17
Illniyar wrote: post #7

I think its worth mentioning again that what Uber ended up using has no resemblence to an RDBMS (single table, manual indexes). So regardless of whether their complaints are justified or not, it should not be taken as an endorsment of mySQL over postgres, but rather of an endorsement of NoSQL over RDBMS . Which is really just what every company at these scales do (except for google and f5 if whitepapers are to be con…

If it was just about NoSQL vs RDBMS they could have used Postgres as a key-value store instead of mySQL. So yes they absolutely did endorse mySQL over Postgres. Also, what is the scale of Uber that you think is unsuitable for RDBMS? Uber does a million rides per day but these rides are very predictably local. I don't know enough to make any firm claims here but at first sight this doesn't look like the sort of scale…

They endorsed MySQL over PostgreSQL for a type of structure and access very few people use. I don't see how the query language plays into the recommendation at all.

Re: Thoughts on Uber’s List of Postgres Limitations

#68

Post by rch »

asah wrote: post #54

To me, the real news is that Uber ($50B company) didn't bother to engage the postgres community before migrating - they'd have jumped to support Uber.

I don't get the sense that Uber has had much stability in technical leadership over its lifetime. Big moves like this can be as much cultural as technical.

Re: Thoughts on Uber’s List of Postgres Limitations

#70

Post by slavik81 »

Illniyar wrote: post #64
laszlokorte wrote: post #51

Earlier quoted context omitted.

> If you use only 1 table, then it's by no means "relational" [...] FYI The term "relational" in "relational database" does not refer to the relation between tables but to the mathematical concept of a relation which is some set of tuples ie a single table.

Well, the relational model does not describe or contain a notion of table really, so I guess you could model it in any way you want, even within a single table. But I think that's a mute point, I doubt they kept any part of the relational model inside their schemaless database.

Probably just an auto-correct mistake, but you mean "that's a moot point".
Post reply on HN