Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Uber Engineering Switched from Postgres to MySQL

eng.uber.com

Post reply on HN
Print view View original

131–140 of 306 posts

Re: Why Uber Engineering Switched from Postgres to MySQL

#131

Post by pritambaral »

snarfy wrote: post #111

I wonder how much of this could have been solved by using a different file system. There is all of this talk about the physical layer but no mention of the file system used. > Typically, write amplification refers to a problem with writing data to SSD disks: a small logical update (say, writing a few bytes) becomes a much larger, costlier update when translated to the physical layer. This is exactly the type of probl…

How would any filesystem help with that? SSDs typically write entire blocks, even if the OS asks them to only write a few bytes. That's just how SSDs work.

Re: Why Uber Engineering Switched from Postgres to MySQL

#132

Post by jedberg »

yeukhon wrote: post #128
jedberg wrote: post #116

> MySQL supports multiple different replication modes: > Statement-based replication replicates logical SQL statements (e.g., it would literally replicate literal statements such as: UPDATE users SET birth_year=770 WHERE id = 4) Postgres has that too (using a 3rd party tool, but it's an officially supported tool). We were using it on reddit 10 years ago. It caused a lot of problems. I wouldn't call that an advantage…

I was wondering how could that happen? It sounds like someone is trying to do two things in parallel. I would expect def my_view(request): try: receipt = generate_receipt(...) receipt.send_email(...) except SomeKindOfEmailError as e: # okay do something else and this should be synchronous and thus blocking. So to not block, they either wrote co-routines (asynchronous) or execute things in parallel. Have I interpreted…

Open db connection

get data for receipt

generate receipt

send email

write success to database

close connection

To a junior programmer this would probably look reasonable, and to be fair, it takes some experience and getting burned, or good training, to know it is not.

Re: Why Uber Engineering Switched from Postgres to MySQL

#133

Post by chucky_z »

fjordan wrote: post #76
ledjon wrote: post #51

I would argue that most of these Postgres "flaws" are actually advantages over MySQL when you look at them holistically rather than the very specific Uber use-case. Postgres's MVCC is superior (can rollback DDL, can add indexes online, can have open read transactions for a VERY long time without impacting other parts of the system) Postgres supports many types of indexes, not just b-tree. One thing it doesn't have is…

Tools such as the percona toolkit ( https://www.percona.com/software/mysql-tools/percona-toolkit ) provide the ability to perform safe and performant online alters for MySQL. Behind the scenes it actually creates a new table with the new schema and utilizes triggers to apply writes to the new table as the original data is being copied. Once it completes the table is renamed and the triggers are removed. Percona also…

online schema change is absolutely riddled with crippling bugs if you use foreign keys. Just a heads up, we've had to basically take everything it does and make an in-house version. The idea and execution are great, when it actually works.

Re: Why Uber Engineering Switched from Postgres to MySQL

#134

Post by midom »

_Codemonkeyism wrote: post #117
busterarm wrote: post #87

Earlier quoted context omitted.

Can't agree with this post enough. I find their whole writeup to be terribly myopic. When they started their service, Postgres was almost certainly the right choice for what they were building and their MySQL setup was not. Now Postgres is less effective for them. These kind of tech switches are _inevitable_ if you're making the right choices for your organization. This strikes me as very similar to the article where…

"1B+ rows in it with mysql?" Been there, really no fun.

As long as you can shard (across multiple instances, or even within same instance, to avoid B-Tree latching), 1B+ rows within MySQL is piece of cake.

Also, MySQL* is getting LSM-Tree support lately, which makes high performance data ingestion combined with OLTP workload quite feasible.

* https://github.com/facebook/mysql-5.6/tree/webscalesql-5.6.2...

Re: Why Uber Engineering Switched from Postgres to MySQL

#135

Post by [deleted] »

_Codemonkeyism wrote: post #117
busterarm wrote: post #87

Earlier quoted context omitted.

Can't agree with this post enough. I find their whole writeup to be terribly myopic. When they started their service, Postgres was almost certainly the right choice for what they were building and their MySQL setup was not. Now Postgres is less effective for them. These kind of tech switches are _inevitable_ if you're making the right choices for your organization. This strikes me as very similar to the article where…

"1B+ rows in it with mysql?" Been there, really no fun.

[deleted]

Re: Why Uber Engineering Switched from Postgres to MySQL

#136

Post by Illniyar »

appleflaxen wrote: post #2

This was a great overview and write-up. Anyone know why they are using MySQL over MariaDB[1]? 1. https://mariadb.org/

Well there new database is basically a key value store built on MySQL, so I think there isn't much to be gained by using MariaDB and it will add needless additional complexity.

Re: Why Uber Engineering Switched from Postgres to MySQL

#137

Post by serge2k »

tracker1 wrote: post #66
millamox wrote: post #30

Earlier quoted context omitted.

I strongly disagree. it would have been useful f they'd stuck to problems without well-known solutions. Sadly, they also mixed in issues which are easily solved, or in a particularly egregious case, where they just complain about a bug. As though MySQL never had a bug. That was silly. My read of it was: Postgres annoyed us a few times, and we got fed up with its, so now something different will annoy us. Please look…

I got a similar impression... though with Uber's scale, funding and resources, they probably could have worked with and through their issues with Postgres. I'm actually surprised they didn't take a multi-pronged approach to their issues. Since they're using Schemaless, I'm curious why they didn't go for one of the many non-sql databases that may well be a much closer match to their use case. It seems to me that Cassa…

> though with Uber's scale, funding and resources, they probably could have worked with and through their issues with Postgres

Then they wouldn't get to build cool new stuff and write blog posts about how they had to build cool new stuff because OMG UBER SCALE.

Re: Why Uber Engineering Switched from Postgres to MySQL

#138

Post by nine_k »

jedberg wrote: post #116

> MySQL supports multiple different replication modes: > Statement-based replication replicates logical SQL statements (e.g., it would literally replicate literal statements such as: UPDATE users SET birth_year=770 WHERE id = 4) Postgres has that too (using a 3rd party tool, but it's an officially supported tool). We were using it on reddit 10 years ago. It caused a lot of problems. I wouldn't call that an advantage…

Sending an email to a local MTA should be pretty fast.

That is, if you cannot and will not do async stuff in your program, there are ready-made tools that will do that particular thing asynchronously for you, and have been doing so for years (or even decades).

Re: Why Uber Engineering Switched from Postgres to MySQL

#139

Post by midom »

0xmohit wrote: post #5

Facebook maintains it's own fork [0] of MySQL. A couple of interesting talks are also available: MySQL at Facebook, Current and Future [1] and Massively Distributed Backup at Facebook Scale [2]. [0] https://github.com/facebook/mysql-5.6 [1] https://www.youtube.com/watch?v=jqwegP9xwVE [2] https://www.youtube.com/watch?v=UBHcmP2TSvk

technically, it is a patch set, it gets frequently rebased against the upstream

Re: Why Uber Engineering Switched from Postgres to MySQL

#140

Post by chucky_z »

tracker1 wrote: post #66
millamox wrote: post #30

Earlier quoted context omitted.

I strongly disagree. it would have been useful f they'd stuck to problems without well-known solutions. Sadly, they also mixed in issues which are easily solved, or in a particularly egregious case, where they just complain about a bug. As though MySQL never had a bug. That was silly. My read of it was: Postgres annoyed us a few times, and we got fed up with its, so now something different will annoy us. Please look…

I got a similar impression... though with Uber's scale, funding and resources, they probably could have worked with and through their issues with Postgres. I'm actually surprised they didn't take a multi-pronged approach to their issues. Since they're using Schemaless, I'm curious why they didn't go for one of the many non-sql databases that may well be a much closer match to their use case. It seems to me that Cassa…

What replication issues are you referring too? I never once had a problem with pgsql's replication across 8.1->9.5.

It would randomly die, but that was always either my fault or the applications fault, never pgsql itself.

The lack of master-master seems to be the big thing everyone mentions, but PostgresXL is currently in a usable-in-production state.

Post reply on HN