The connection handling section was surprising to me, reading that Postgres uses a process per connection! This is pretty shocking to me, in a bad way.
pgbouncer works pretty well for that. I kinda wish it was part of the default Postgres install.
Why Uber Engineering Switched from Postgres to MySQL
21–30 of 306 posts
Re: Why Uber Engineering Switched from Postgres to MySQL
#22Fyi... a related (not duplicate) discussion of a previous Uber story: https://news.ycombinator.com/item?id=10923848
Re: Why Uber Engineering Switched from Postgres to MySQL
#23I've heard from technical leaders at multiple now well established unicorns how they'd never use postgres or switched from postgres simply because MySQL has a lot more tooling built it and many more people are exposed to its shortcomings at "web scale" so that it's very well known where and when things will break. Disclaimer, I'm a hardcore Postgres user myself, but I also keep tabs on the other tools.
OTOH, it's important to consider that most places are not and will never be "unicorns", so analyzing things at that scale may not make sense. Just because Facebook managed to make something work doesn't mean that you should follow the same path.
Re: Why Uber Engineering Switched from Postgres to MySQL
#24this reads like a laundry list of buzzwords that were designed to justify not throwing any effort into postgresql and just going with a new shiny toy (not mysql. yes. i know it's been around for a while). it happens everywhere.
From the post: > [...] This design difference means that the MySQL replication binary log is significantly more compact than the PostgreSQL WAL stream. Doesn't sound like what you described at all.
so you're pulling one line from the article to tell me that i'm wrong?
on-disk format/write amplification: > For tables with a large number of secondary indexes, these superfluous steps can cause enormous inefficiencies. For instance, if we have a table with a dozen indexes defined on it, an update to a field that is only covered by a single index must be propagated into all 12 indexes to reflect the ctid for the new row.
How wide is their data? Depending on the answer to this, it could be that they've over-indexed, have a poor indexing strategy, or are reacting to the poor queries generated by an ORM (not sure if they use one, or if they hand-code their own SQL).
data corruption: everyone has bugs. https://bugs.mysql.com/search.php?search_for=&status=Active&... add in mysql's tendency to loosely-adhere to the SQL standard and there are many ways that you can actually corrupt your own data.
i'm not here to debate whether postgres is better than mysql. i'm just saying that it seems like a lot of research went into justifying a switch. who knows, maybe that research could have been spent optimizing their current environment.
Re: Why Uber Engineering Switched from Postgres to MySQL
#25this reads like a laundry list of buzzwords that were designed to justify not throwing any effort into postgresql and just going with a new shiny toy (not mysql. yes. i know it's been around for a while). it happens everywhere.
MySQL can hardly be described as shiny new toy. The peak of its hype cycle was around 2001
Re: Why Uber Engineering Switched from Postgres to MySQL
#26Re: Why Uber Engineering Switched from Postgres to MySQL
#27this reads like a laundry list of buzzwords that were designed to justify not throwing any effort into postgresql and just going with a new shiny toy (not mysql. yes. i know it's been around for a while). it happens everywhere.
MySQL can hardly be described as shiny new toy. The peak of its hype cycle was around 2001
"Schemaless" on top of it and a shiny new "microservices" layer. Please, by all means, continue to take one aspect of the stack in an attempt to misrepresent me.
Re: Why Uber Engineering Switched from Postgres to MySQL
#28The connection handling section was surprising to me, reading that Postgres uses a process per connection! This is pretty shocking to me, in a bad way.
One could use connection pooling. Quoting from the tuning guide [0]:
max_connections sets exactly that: the maximum number of client
connections allowed. This is very important to some of the
below parameters (particularly work_mem) because there are some
memory resources that are or can be allocated on a per-client
basis, so the maximum number of clients suggests the maximum
possible memory use. Generally, PostgreSQL on good hardware can
support a few hundred connections. If you want to have
thousands instead, you should consider using connection pooling
software to reduce the connection overhead.
[0] https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serv...
[1] https://wiki.postgresql.org/wiki/Replication,_Clustering,_an...
Re: Why Uber Engineering Switched from Postgres to MySQL
#29Facebook maintains it's own fork [0] of MySQL. A couple of interesting talks are also available: MySQL at Facebook, Current and Future [1] and Massively Distributed Backup at Facebook Scale [2]. [0] https://github.com/facebook/mysql-5.6 [1] https://www.youtube.com/watch?v=jqwegP9xwVE [2] https://www.youtube.com/watch?v=UBHcmP2TSvk
from what I understand FB uses Mysql as permanent storage, not as relational database
Re: Why Uber Engineering Switched from Postgres to MySQL
#30Posts like this are important. We too often rely on a buzz-word heuristic and that's how you end up with dozens of random technologies that are harder to maintain and don't necessarily solve any of your problems. This method is good, because it shows that when you understand the problem the right way, you can find the right solution, even if by popularity it looks like a "step backwards" Massive Kudos.
I strongly disagree. it would have been useful f they'd stuck to problems without well-known solutions.
Sadly, they also mixed in issues which are easily solved, or in a particularly egregious case, where they just complain about a bug. As though MySQL never had a bug. That was silly.
My read of it was: Postgres annoyed us a few times, and we got fed up with its, so now something different will annoy us. Please look forward to our blog post in 4 years about how we're using X instead of MySQL/Schemaless because those were also imperfect.