Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on Uber’s List of Postgres Limitations

blog.2ndquadrant.com

Post reply on HN
Print view View original

21–30 of 108 posts

Re: Thoughts on Uber’s List of Postgres Limitations

#21

Post by harshal »

Potentially the most useful part of this post to me was this part

> 2ndQuadrant is working on highly efficient upgrades from earlier major releases, starting with 9.1 → 9.5/9.6.

I hadn't heard of that before. Anybody know more about this? I'm currently babysitting a 9.1 deployment which we desperately want to get upgraded. The amount of downtime this can tolerate a very limited and I was currently tasked with coming up with a plan. Its going to get hairy. If such a tool is really on its way, I could make a case for holding off on the upgrade for a few more months and save quite a bit of work.

Re: Thoughts on Uber’s List of Postgres Limitations

#22

Post by pilif »

harshal wrote: post #21

Potentially the most useful part of this post to me was this part > 2ndQuadrant is working on highly efficient upgrades from earlier major releases, starting with 9.1 → 9.5/9.6. I hadn't heard of that before. Anybody know more about this? I'm currently babysitting a 9.1 deployment which we desperately want to get upgraded. The amount of downtime this can tolerate a very limited and I was currently tasked with coming…

you can use pg_upgrade with -k - it will complete within seconds. Afterwards, things will be slow until a complete analyze updates the statistics, but the update itself can be done in seconds.

I have updated ~2TB of database from 9.0 all the way to 9.5 over the years.

Re: Thoughts on Uber’s List of Postgres Limitations

#24

Post by Illniyar »

fauigerzigerk wrote: post #17
Illniyar wrote: post #7

I think its worth mentioning again that what Uber ended up using has no resemblence to an RDBMS (single table, manual indexes). So regardless of whether their complaints are justified or not, it should not be taken as an endorsment of mySQL over postgres, but rather of an endorsement of NoSQL over RDBMS . Which is really just what every company at these scales do (except for google and f5 if whitepapers are to be con…

If it was just about NoSQL vs RDBMS they could have used Postgres as a key-value store instead of mySQL. So yes they absolutely did endorse mySQL over Postgres. Also, what is the scale of Uber that you think is unsuitable for RDBMS? Uber does a million rides per day but these rides are very predictably local. I don't know enough to make any firm claims here but at first sight this doesn't look like the sort of scale…

"Also, what is the scale of Uber that you think is unsuitable for RDBMS"

Never said it's unsuitable, I have no idea. Just said the move from relational to NoSQL is what most billion dollar companies with mass-market apps or websites do.

Many of them also do sharding on normal RDBMS.

Re: Thoughts on Uber’s List of Postgres Limitations

#25

Post by threeseed »

ungzd wrote: post #23

It seems that after lots of hyped NoSQL systems companies are still using MySQL or Postgresql as simple storage backend with lots of custom crutches over it, like it's 2007. So all these cassandaras and riaks failed expectations?

This is one of the more ridiculous posts I've seen on HN.

"Companies" are taking a whole range of approaches to storing data. With a combination of NoSQL, SQL and Filesystem e.g. HDFS and everything else in between. Cassandra in particular is killing it right now under the stewardship of Datastax which is why they've grown from 1 person up to 400+ employees.

Re: Thoughts on Uber’s List of Postgres Limitations

#26

Post by threeseed »

Illniyar wrote: post #7

I think its worth mentioning again that what Uber ended up using has no resemblence to an RDBMS (single table, manual indexes). So regardless of whether their complaints are justified or not, it should not be taken as an endorsment of mySQL over postgres, but rather of an endorsement of NoSQL over RDBMS . Which is really just what every company at these scales do (except for google and f5 if whitepapers are to be con…

> Which is really just what every company at these scales do

No they aren't. You need to understand that databases are being used in a whole range of different ways than you think.

Batch/streaming analytics are typically done with HDFS, Cassandra, HBase, MongoDB etc. Event aggregation often with a time series database like Druid, InfluxDB etc. Web API serving layers are still the domain of lightweight SQL databases like MySQL, PostgreSQL. Customer data warehouses are still dominated by Teradata, Oracle, SAP etc.

What you misconstrue as a single platform is often a multi headed beast with various architectural components. Each comprising their own technology stacks.

Re: Thoughts on Uber’s List of Postgres Limitations

#28

Post by pas »

Illniyar wrote: post #18
leftnode wrote: post #14

Earlier quoted context omitted.

Can you expand on what you mean by a single table with manual indexes? How does a frequently used table with indexes not resemble an RDBMS?

the secondary indexes are saved in a different table and (I can only assume) the application layer (or one layer above mySql) is responsible for keeping that table up to date. If you use only 1 table, then it's by no means "relational", so I don't see why you'll need a database system designed from start to finish to support a relational model.

The "relationalness" is/was not really important here. It's all about MVCC and how storage engines handle it.

Postgres is lacking in these scenarios, whereas a particular fine tuned version (or fork) of InnoDB (or MyRocks or whatever they end up chosing) handles this better.

See Facebook's "mysql-5.6" branch, that has hundreds of patches piled on to support especially these taxing workloads.

Re: Thoughts on Uber’s List of Postgres Limitations

#29

Post by dkersten »

f055 wrote: post #10

I came here to write a snarky comment, but now I can write two ;) first: if you think any particular db platform is clearly a winner in "db wars", you are naive. there are so many factors involved in configuring the db, the backend, the frontend etc. that you can always find a case where: the supposedly winning db is failing, or the supposedly worse db is performing perfectly fine. and from my experience, you should…

re: your second point, this isn't necessarily anything to do with database scalability or tuning.

Re: Thoughts on Uber’s List of Postgres Limitations

#30

Post by majewsky »

f055 wrote: post #10

I came here to write a snarky comment, but now I can write two ;) first: if you think any particular db platform is clearly a winner in "db wars", you are naive. there are so many factors involved in configuring the db, the backend, the frontend etc. that you can always find a case where: the supposedly winning db is failing, or the supposedly worse db is performing perfectly fine. and from my experience, you should…

> static page cache behind cdn

I wouldn't want my page behind a CDN. CDNs make users much more trackable across sites.

My point isn't that CDNs are bad for everyone. My point is, once more, that most questions are not as simple as they may appear.

Post reply on HN