Live data from Hacker News

PostgreSQL Scalability: Towards Millions TPS

akorotkov.github.io

81–90 of 222 posts

Re: PostgreSQL Scalability: Towards Millions TPS

#81
post #3

I don't know much but to me it seems PostgreSQL is probably one of the most open and supporting communities maybe this is reason alot of new faces are looking at it including me.

It's also a great example of a technology which is mature and well-tested yet actively growing and improving. Most open source projects have a lot to learn from Postgres.

What would you say those learning points are? What key factors would you introduce into another OSS project that PostgreSQL currently employs?

Re: PostgreSQL Scalability: Towards Millions TPS

#82
Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because:

(1 it is a better technical fit for a very specific problem

(2 there is already a legacy db in place

I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown.

The only time I will advocate for anything other than Postgres is when Wordpress is involved. If the data model is simple enough then MySQL is more than up for the task, and it avoids an additional database dependency.

Thankfully all the ORM's that are worth using support MySQL and Postgres, so using both is very doable.

### Useful Postgres (or SQL in general) tools/libraries :

Bookshelf ORM http://bookshelfjs.org/

PostgREST automated REST API https://github.com/begriffs/postgrest

Sqitch git style migration management http://sqitch.org/

Re: PostgreSQL Scalability: Towards Millions TPS

#83
post #43

Earlier quoted context omitted.

I am in no way a fan of Oracle but, c'mon, that's a little over the top, don't you think?

Reminds me of a talk by Bryan Cantrill he talked a bit (I am paraphrasing) that he didn't believe that people could be purely evil and when looking into Larry Ellisson he actually discovered he donated several hundred mil. to a nonprofit (dedicated to prolonging human life). On closer look it turned the sole purpose of that nonprofit was working on prolonging life of one particular human namely Larry Ellisson. (while…

Bryan Cantrill has a very hyperactive delivery when giving a talk. Personally, I love it, though it isn't for everyone. https://www.youtube.com/results?search_query=bryan+cantrill

Re: PostgreSQL Scalability: Towards Millions TPS

#84
post #66

Earlier quoted context omitted.

MySQL is considered more "old school" and not great at being standards compliant when installed with default settings. See https://twitter.com/robconery/status/189086889486192640 Also, MySQL is run by Oracle...and Oracle a horrible company run by horrible people that behave horribly, and should never ever be trusted in any form. Any developer relying or Oracle code should not be trusted ever again, and Oracle consult…

What you are describing is no longer the default configuration. Say what you like about oracle, they have been a great steward of MySQL.

> oracle, they have been a great steward of MySQL

I'm not so sure everyone will agree with reports of closed source regression tests and source histories[1][2] (I know these stories are old)

[1]: http://techcrunch.com/2012/08/18/oracle-makes-more-moves-to-... [2]: https://news.ycombinator.com/item?id=4400797

Re: PostgreSQL Scalability: Towards Millions TPS

#85
post #12

Slightly tangential but I'm genuinely curious, does any have a theory as to why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL or MariaDB? Considering the relative obscurity of the former it seems somewhat inexplicable.

> why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL

You'll see the same phenomenon on Slashdot.

MySQL is popular among a subset of programmers: web developers. In corporations, Microsoft SQL and Oracle are more popular. Further, MySQL is popular among a subset of web developers: those who use PHP. Among web developers who use Python, Postgres seems more popular.

My suspicion is that MySQL's popularity is tied to those web-hosting plans (1 GB of storage! 1 TB of bandwidth! 10 databases! $10/month!). They were almost always MySQL databases. Web hosts that offered Postgres databases were few and far between. This article corroborates my theory: http://rodner.blogspot.com/2008/01/what-makes-mysql-so-popul.... MySQL's company (which at the time was "MySQL" not Oracle) pushed it in at a critical time in the development of the web.

I have written applications that use: Postgres, MySQL, SQLite, Microsoft SQL, and Oracle. My favorite by far is Postgres. The strangest thing to me is not that MySQL is more popular than Postgres, but that anyone uses Microsoft or Oracle at all. Not only do they cost a lot, but from a purely technical standpoint they are worse.

Re: PostgreSQL Scalability: Towards Millions TPS

#86

Andres is a great coworker to have at Citus Data, though I first ran into him on the mailing lists shortly after starting at Citus myself. I was tasked with figuring out "why do certain read-only workloads fail miserably under high concurrency?" I had never touched PostgreSQL before, nor any Linux performance tools, but I noticed that replacing certain buffer eviction locks with atomic implementations could drastical…

I wonder, does Intel's TSX/HLE help with these workloads? If it's read-only then I'd expect that it'd be able to elide a lot of the locking (assuming the Intel-designed heuristics do the job).

I'd bought one of the first haswell notebooks to play around with tsx. Before I'd time to do so Intel found the tsx bug...

I hope to have time to play around with it one I have new hardware (I refuse to do performance development on virtual).

But honestly, most remaining performance/scalability problems in pg are more algorithmically caused. So micro optimization, and that's what I'd call tax/hle, aren't likely to biggest bottleneck.

Re: PostgreSQL Scalability: Towards Millions TPS

#87
post #68

Always nice to hear about throughput improvements in Postgres. How do these changes affect more heterogeneous workflows, of mixed reads and writes? A little better? A lot better? A little worse?

Very dependent on the workload. The optimization isn't specific to reads or writes, but in many cases your bottleneck when writing will be elsewhere.

Re: PostgreSQL Scalability: Towards Millions TPS

#88
post #82

Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because: (1 it is a better technical fit for a very specific problem (2 there is already a legacy db in place I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown. The only time I will advocate for anything other than P…

What do you mean by MySQL avoiding an additional database dependency?

Re: PostgreSQL Scalability: Towards Millions TPS

#89
post #88
post #82

Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because: (1 it is a better technical fit for a very specific problem (2 there is already a legacy db in place I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown. The only time I will advocate for anything other than P…

What do you mean by MySQL avoiding an additional database dependency?

[deleted]

Re: PostgreSQL Scalability: Towards Millions TPS

#90
post #88
post #82

Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because: (1 it is a better technical fit for a very specific problem (2 there is already a legacy db in place I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown. The only time I will advocate for anything other than P…

What do you mean by MySQL avoiding an additional database dependency?

IIRC PHP has built-in support for MySQL out of the box.
Post reply on HN