Thoughts on Uber’s List of Postgres Limitations

Post by AndyNemmity » Wed, Aug 03, 2016, 6:59 PM UTC

brianwawok wrote: post #63

ahachete wrote: post #32

Earlier quoted context omitted.

While I mostly agree with your comment, some NoSQL databases are not suitable for analytics. Indeed, they are terrible at them. Unstructured information is, surprise, unstructured, which means it is harder to query / analyze, since the structure needs to be fully scanned and parsed to perform the computation. I have seen many cases where MongoDB is used for this and suffers from really bad performance.

I think the trap is simple POCS. Do a simple POC where you ram 1 million rows in a MongoDB and then make a webapp to do some basic analytics. Look, it works and get responses within a second. Cool! Then real data comes in at 12 billion rows and your analytics take 3 hours to run. So you try to do the sharding thing, and realize that it works for a while, except now every analytics query needs to hit every shard....

Post by amaks » Wed, Aug 03, 2016, 11:34 PM UTC

mranney wrote: post #20

Illniyar wrote: post #7

I think its worth mentioning again that what Uber ended up using has no resemblence to an RDBMS (single table, manual indexes). So regardless of whether their complaints are justified or not, it should not be taken as an endorsment of mySQL over postgres, but rather of an endorsement of NoSQL over RDBMS . Which is really just what every company at these scales do (except for google and f5 if whitepapers are to be con…

Uber runs a lot of databases in production. We stopped using Postgres a while ago for new applications that talk SQL, and this article explains some of the reasoning. We still have many applications that talk directly to MySQL, and we still have our original API monolith that talks directly to Postgres. All new applications are being built using distributed databases like our in-house Schemaless system which happens…

Post by [deleted] » Wed, Aug 03, 2016, 11:45 PM UTC

bigato wrote: post #39

pilif wrote: post #22

Earlier quoted context omitted.

you can use pg_upgrade with -k - it will complete within seconds. Afterwards, things will be slow until a complete analyze updates the statistics, but the update itself can be done in seconds. I have updated ~2TB of database from 9.0 all the way to 9.5 over the years.

The problem with this is that if anything fails, you can potentially corrupt your data and have no backup plan. To make that option safe, you would have to copy your data directory first, and you need to be offline for that. So you have to add the time it takes to make that copy.

Post by snaky » Thu, Aug 04, 2016, 12:59 AM UTC

landmark3 wrote: post #27

Dowwie wrote: post #8

Does anyone know the back story to Uber - why didn't it try to improve Postgres rather than move on to feed on another host?

because they couldn't find what they were looking for in Posgres forum

Post by debaserab2 » Thu, Aug 04, 2016, 4:35 AM UTC

thinkMOAR wrote: post #60

debaserab2 wrote: post #47

Earlier quoted context omitted.

I don't know, if you're going to critique the very talented engineers at Uber, seeing your blog fall over due to capacity doesn't lend you a lot of credibility.

On what information do you base your opinion that the engineers at uber are very talented? So far what i have seen, the only thing Uber is talented at is violating local laws and then throwing sacks of money at it to pay fines or whatever. (and inflating their own (bubble)value, but probably not many people agree with that) Seeing their blogs mysql-> postgres followed by a postgres -> mysql migration, doesn't give me…