Live data from Hacker News

PostgreSQL Scalability: Towards Millions TPS

akorotkov.github.io

31–40 of 222 posts

Re: PostgreSQL Scalability: Towards Millions TPS

#31
post #12

Slightly tangential but I'm genuinely curious, does any have a theory as to why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL or MariaDB? Considering the relative obscurity of the former it seems somewhat inexplicable.

There's been a large migration off of mysql to postgres simply because Oracle got the rights to mysql when they purchased Sun. That is what made me consider, ditching mysql.

The final straw came when I stood up a mysql 5.6 instance to use as a data warehouse for about 5TB of data (15 billion rows). To my horror after spending a few weeks on this project I discovered that mysql only supported a small subset of the SQL language. Most of the needed sql features for data warehouse queries were simply not there (no WITH clause, no ROW_NUMBER(), only one sequence per table, no schema support, and many others that SQL report ninjas love to use)

With that I had to go back to management and convince them to put more time into the project so we could try again with postgresql. There was a large learning curve to go to postgres. It was just different in many areas, but at the same time, reminded me alot of IBM DB2 (both were a product of the 1980's if you look at the history). Postgres fit into our plans nicely, it actually had a full blown intelligent query parser and optimizer unlike mysql. From what I learned, mysql's purpose was to stop people from using text files as storage points in the early 2000's. But once you graduate from that, it's onward to postgres from there.

Re: PostgreSQL Scalability: Towards Millions TPS

#33
post #12

Slightly tangential but I'm genuinely curious, does any have a theory as to why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL or MariaDB? Considering the relative obscurity of the former it seems somewhat inexplicable.

It's because Python tends to be the most common programming language for startups due to being easy to hire for, having good library support for a wide range of use cases, being good for web development, etc. And MySQL is focused on catering to enterprise customers who don't use Python. The last time I heard, they had something like one person working part time on Python support, so the drivers weren't nearly as reli…

seems like that would be pretty tough to get reliable stats on something like that.

http://www.tiobe.com/tiobe_index

Re: PostgreSQL Scalability: Towards Millions TPS

#34
post #28

Earlier quoted context omitted.

It's because Python tends to be the most common programming language for startups due to being easy to hire for, having good library support for a wide range of use cases, being good for web development, etc. And MySQL is focused on catering to enterprise customers who don't use Python. The last time I heard, they had something like one person working part time on Python support, so the drivers weren't nearly as reli…

Err, no. Python probably isn't even in the top 5 of languages startups choose for their main stack.

Not the most sound method but I was demonstrating the relative popularity of programming languages to my SO the other day using find in the Who's Hiring thread for May and python was the most popular.

Surprised me as well.

Re: PostgreSQL Scalability: Towards Millions TPS

#35
post #23

How does PostgreSQL compare to VoltDB? I'm trying to get a handle on the different databases, and VoltDB sounds exciting, but everyone's talking about PostgreSQL. Then there's Mnesia which I hear is, as all things Erlang, excellent, though it's kinda tied to Erlang. I know it's hard to say what's best, but what would you say is the best DB for a completely new multilingual project that needs throughput but prioritize…

> How does PostgreSQL compare to VoltDB? If you don't know the difference, you probably want Postgres. VoltDB is a specialty database for things like high frequency trading. It wouldn't make sense to use for, say, a consumer app or web startup.

Yeah, if I start a new project, I default to Postgres: https://journal.dedasys.com/2015/02/21/i-default-to-postgres...

I'd consider something else if I'm really, really sure that it's better suited to whatever niche problem than Postgres, but it'd take a lot of thinking and convincing myself.

Re: PostgreSQL Scalability: Towards Millions TPS

#37
post #12

Slightly tangential but I'm genuinely curious, does any have a theory as to why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL or MariaDB? Considering the relative obscurity of the former it seems somewhat inexplicable.

MySQL is considered more "old school" and not great at being standards compliant when installed with default settings. See https://twitter.com/robconery/status/189086889486192640 Also, MySQL is run by Oracle...and Oracle a horrible company run by horrible people that behave horribly, and should never ever be trusted in any form. Any developer relying or Oracle code should not be trusted ever again, and Oracle consult…

I am in no way a fan of Oracle but, c'mon, that's a little over the top, don't you think?

Re: PostgreSQL Scalability: Towards Millions TPS

#38

Sounds like the padding stuff is a false sharing issue. They might want to look into putting 128 bytes of padding between the data structures as well: http://www.forwardscattering.org/post/29

That's what the discussed padding patch does (except to only padding to 64bytes on most platforms). There's a downside though - on low concurrency the padding reduces the cache hit ratio sufficiently enough to cause a slowdown. Given we're in code freeze anyway, I've not spent a lot of though on that yet; but I suspect that rearchitecting things so the lines are dirtied fewer times, is the better fix; with less poten…

Would it be possible to dynamically choose the padding at server start time? Given that cache line sizes vary that seems like it might be a prudent choice.

Re: PostgreSQL Scalability: Towards Millions TPS

#39
post #12

Slightly tangential but I'm genuinely curious, does any have a theory as to why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL or MariaDB? Considering the relative obscurity of the former it seems somewhat inexplicable.

It's because Python tends to be the most common programming language for startups due to being easy to hire for, having good library support for a wide range of use cases, being good for web development, etc. And MySQL is focused on catering to enterprise customers who don't use Python. The last time I heard, they had something like one person working part time on Python support, so the drivers weren't nearly as reli…

Python has had rock-solid, widely used MySQL libraries for well over a decade. I've never seen evidence that MySQL and Python aren't a strong (and popular) combination.

Re: PostgreSQL Scalability: Towards Millions TPS

#40

Man, how I wish WordPress had originally chosen to use PostgreSQL instead of MySQL back in the day.

AFAIK WordPress was based on a stack that worked well on shared hosts of its day. They didn't really choose a lot of that stuff so much as have it chosen for them by the hosting community.
Post reply on HN