Live data from Hacker News

Thoughts on Uber’s List of Postgres Limitations

blog.2ndquadrant.com

31–40 of 108 posts

Re: Thoughts on Uber’s List of Postgres Limitations

#31
post #8

Does anyone know the back story to Uber - why didn't it try to improve Postgres rather than move on to feed on another host?

> why didn't it try to improve Postgres

Honestly, why would they? If there is a product that does what they need, why spend resources improving another.

Just because it's open source and they could spend money improving it, doesn't mean a company will spend that money improving it.

Re: Thoughts on Uber’s List of Postgres Limitations

#32
post #7

I think its worth mentioning again that what Uber ended up using has no resemblence to an RDBMS (single table, manual indexes). So regardless of whether their complaints are justified or not, it should not be taken as an endorsment of mySQL over postgres, but rather of an endorsement of NoSQL over RDBMS . Which is really just what every company at these scales do (except for google and f5 if whitepapers are to be con…

> Which is really just what every company at these scales do No they aren't. You need to understand that databases are being used in a whole range of different ways than you think. Batch/streaming analytics are typically done with HDFS, Cassandra, HBase, MongoDB etc. Event aggregation often with a time series database like Druid, InfluxDB etc. Web API serving layers are still the domain of lightweight SQL databases l…

While I mostly agree with your comment, some NoSQL databases are not suitable for analytics. Indeed, they are terrible at them.

Unstructured information is, surprise, unstructured, which means it is harder to query / analyze, since the structure needs to be fully scanned and parsed to perform the computation.

I have seen many cases where MongoDB is used for this and suffers from really bad performance.

Re: Thoughts on Uber’s List of Postgres Limitations

#34
post #10

I came here to write a snarky comment, but now I can write two ;) first: if you think any particular db platform is clearly a winner in "db wars", you are naive. there are so many factors involved in configuring the db, the backend, the frontend etc. that you can always find a case where: the supposedly winning db is failing, or the supposedly worse db is performing perfectly fine. and from my experience, you should…

That second point is a really unnecessarily belittling straw man, and I think such comments are counterproductive to the discussion.

Re: Thoughts on Uber’s List of Postgres Limitations

#35
post #10

I came here to write a snarky comment, but now I can write two ;) first: if you think any particular db platform is clearly a winner in "db wars", you are naive. there are so many factors involved in configuring the db, the backend, the frontend etc. that you can always find a case where: the supposedly winning db is failing, or the supposedly worse db is performing perfectly fine. and from my experience, you should…

It's not about being the best, it's just about comparing offerings.

Re: Thoughts on Uber’s List of Postgres Limitations

#36
post #34
post #10

I came here to write a snarky comment, but now I can write two ;) first: if you think any particular db platform is clearly a winner in "db wars", you are naive. there are so many factors involved in configuring the db, the backend, the frontend etc. that you can always find a case where: the supposedly winning db is failing, or the supposedly worse db is performing perfectly fine. and from my experience, you should…

That second point is a really unnecessarily belittling straw man, and I think such comments are counterproductive to the discussion.

And these moralizing comments make HN dull and dreary. Compared to 7-10 years ago sometimes I feel that suddenly we're in some sort of new Victorian era.

Re: Thoughts on Uber’s List of Postgres Limitations

#37
I do think that you need to learn how to use in the best way the technologies you have chosen or that are present in your current setup. No matter if it is a MySQL, Postgres or any other DB, it requires as a part of a job, learn how to use at it's best. The points on the article are good, however it's true that Postgres had problems with scalability not so long ago. That's changing, however I think that other data stores have addressed the problem of availability and scalability earlier and gained maturity during the last years.

Also, there is something that caused some noise to me:

    This point is correct; PostgreSQL indexes currently use a
    direct pointer between the index entry and the heap tuple 
    version. InnoDB secondary indexes are “indirect indexes” 
    in that they do not refer to the heap tuple version 
    directly, they contain the value of the Primary Key (PK) 
    of the tuple.
That's true, but the article doesn't make explicit that the PK on InnoDB is a clustered index and, that there are other optimizations like adaptive hashing to make read queries faster.

Re: Thoughts on Uber’s List of Postgres Limitations

#38
post #10

I came here to write a snarky comment, but now I can write two ;) first: if you think any particular db platform is clearly a winner in "db wars", you are naive. there are so many factors involved in configuring the db, the backend, the frontend etc. that you can always find a case where: the supposedly winning db is failing, or the supposedly worse db is performing perfectly fine. and from my experience, you should…

The fact that you can find exceptions do not change the fact that in the common case, tech x is better than tech y. Unless your use case clearly is one of the exceptions most of the time, you still win by choosing the tech that produces the best results most of the time, instead of the one that produces the best results in the exceptional cases only.

So that tech-x versus tech-y is still very relevant.

Re: Thoughts on Uber’s List of Postgres Limitations

#39
post #22
post #21

Potentially the most useful part of this post to me was this part > 2ndQuadrant is working on highly efficient upgrades from earlier major releases, starting with 9.1 → 9.5/9.6. I hadn't heard of that before. Anybody know more about this? I'm currently babysitting a 9.1 deployment which we desperately want to get upgraded. The amount of downtime this can tolerate a very limited and I was currently tasked with coming…

you can use pg_upgrade with -k - it will complete within seconds. Afterwards, things will be slow until a complete analyze updates the statistics, but the update itself can be done in seconds. I have updated ~2TB of database from 9.0 all the way to 9.5 over the years.

The problem with this is that if anything fails, you can potentially corrupt your data and have no backup plan. To make that option safe, you would have to copy your data directory first, and you need to be offline for that. So you have to add the time it takes to make that copy.

Re: Thoughts on Uber’s List of Postgres Limitations

#40
post #10

I came here to write a snarky comment, but now I can write two ;) first: if you think any particular db platform is clearly a winner in "db wars", you are naive. there are so many factors involved in configuring the db, the backend, the frontend etc. that you can always find a case where: the supposedly winning db is failing, or the supposedly worse db is performing perfectly fine. and from my experience, you should…

re: your second point, this isn't necessarily anything to do with database scalability or tuning.

exactly my point. sometimes the solution has nothing to do with the perceived problem.
Post reply on HN