Migrating Facebook to MySQL 8.0 • Hacker News Board

Re: Migrating Facebook to MySQL 8.0

#171

Post by vosper » Thu, Jul 22, 2021, 9:28 PM UTC

evanelias wrote: post #98

andrewmcwatters wrote: post #86

Earlier quoted context omitted.

I didn't downvote you, but you were downvoted for this. So maybe I'll explain further, instead, to stir conversation. As you deal with more and more rows, it becomes imperative that all of your where clauses hit indexes/indices, but even beyond that, with large enough row sizes, it's not enough to provide fast responses. I suspect they deal with this in the form of some sort of caching outside of MySQL, but I haven't…

Facebook is heavily sharded, which keeps the size of each physical table at reasonable levels. Similar story at nearly every large tech company using MySQL, aside from some more recent ones that go for the painful "shove everything in a huge singular AWS Aurora instance" approach :)

> the painful "shove everything in a huge singular AWS Aurora instance" approach :)

I haven't used Aurora - what's painful about this approach (apart from potentially being locked-in to AWS / Aurora)?

Re: Migrating Facebook to MySQL 8.0

#172

Post by dkersten » Thu, Jul 22, 2021, 9:32 PM UTC

gunapologist99 wrote: post #152

twic wrote: post #124

Earlier quoted context omitted.

> I absolutely adore Postgres's tweaks to the SQL language Do you mean additions to the SQL standard, or bits of the standard that MySQL doesn't implement, or something else? The reason i ask is that i also enjoy writing SQL for PostgreSQL, but as far as i know, i am sticking to standard SQL. Perhaps there are things i'm missing, or things i like which i haven't realised are nonstandard!

If you use any of the really amazing JSON or Array stuff, you're stepping outside of the standard.. but it's worth it!

I absolutely love Postgres' JSON and array support. Its saved me from needing a document database in a few cases, which drastically simplified the overall deployment (one DB instead of two, or trying to represent relational stuff in a document DB). I really like JSON for stuff that is user defined, that I don't need to query into, but postgres' support means I still can should I ever need to. Which has happened once or twice for debugging.

Re: Migrating Facebook to MySQL 8.0

#173

Post by beberlei » Thu, Jul 22, 2021, 9:36 PM UTC

takeda wrote: post #147

skrebbel wrote: post #135

Earlier quoted context omitted.

Anyone on the in-crowd who can share a bit about Oracle's motivations here? I recall a Bryan Cantrill talk about how Oracle is this completely amoral machine that just wants to make money, and how that drives their every decision [0]. Is this no longer the case? Or is it, and if so, how does improving MySQL and giving it away for free making Oracle money? Or was it always hyperbole and Oracle is improving MySQL becau…

I guess I'm cynical, but to me looks like Oracle renewed interest in MySQL after PostgreSQL gained momentum. MySQL is not really a competitor to Oracle but PostgreSQL overlaps with both of them.

Oracle is selling Enterprise features and Support, i am sure they make a lot of money with MySQL.

Re: Migrating Facebook to MySQL 8.0

#174

Post by zepearl » Thu, Jul 22, 2021, 9:40 PM UTC

PeterZaitsev wrote: post #82

cmrdporcupine wrote: post #4

My old-man brain still can't compute this statement: "MySQL, an open source database developed by Oracle" . I know this is now factually true, but late-90s me keeps looking over my shoulder and freaking out.

I think MySQL Engineering team does great job with Engineering, Do not want to deal with Oracle Sales ? I would invite you to talk to Percona for Commercial Support and Services Options Full disclosure - I'm CEO at Percona :)

I got sad when I read that TokuDB in Percona & MariaDB was being dropped :(

Replaced TokuDB with MyRocks but it's totally different => now finally managed to have good ~constant performance insert&deletes (had to test & tune a lot), but "updates" remain a problem (they write a lot as I understand that any update always has to rewrite the entire row).

Using InnoDb instead of MyRocks is NOK for me (problems with deletes and related hole punching + bad that tables can never shrink when stuff gets deleted), I don't see other alternatives... .

I'm now experimenting with other DBs (right now with CockroachDB but it doesn't seem to be very stable, might then try as well Postgres but I'm scared by the lack of SQL-hints to cover worst case scenarios).

Re: Migrating Facebook to MySQL 8.0

#175

Post by annexrichmond » Thu, Jul 22, 2021, 9:41 PM UTC

gresrun wrote: post #57

Thaxll wrote: post #16

Earlier quoted context omitted.

Youtube also use MySQL.

YouTube used to use a MySQL-based system called Vitess[0]. It no longer uses it and has migrated to Spanner. [0]: https://vitess.io/

was that migration a technical decision or a political one?

Re: Migrating Facebook to MySQL 8.0

#176

Post by munk-a » Thu, Jul 22, 2021, 9:45 PM UTC

twic wrote: post #124

munk-a wrote: post #96

Earlier quoted context omitted.

I carried out a (quite ill-advised in retrospect) migration from MySQL to Postgres on a mature codebase with absolutely no automated test coverage and it went pretty smoothly. We had some bumpy performance directly after the production migration but we were able to reach parity after about two months of observing and adding targeted indices. I absolutely adore Postgres's tweaks to the SQL language, that dialect is am…

> I absolutely adore Postgres's tweaks to the SQL language Do you mean additions to the SQL standard, or bits of the standard that MySQL doesn't implement, or something else? The reason i ask is that i also enjoy writing SQL for PostgreSQL, but as far as i know, i am sticking to standard SQL. Perhaps there are things i'm missing, or things i like which i haven't realised are nonstandard!

Personally, back before JSON became a standard thing - I made heavy use of HSTORE to clean up some insanely cludgey string packing BS we'd been doing - leveraging recursive CTEs using the UNION definition allowed us to move a fixed tier system into a self-referential one that saved us a lot of headaches as well.

Other folks have mentioned :: instead of CAST AS (which is really nice when you need to write it a bunch of times) but it's mostly the diverse built-in function set and constraint support that keeps me an enthusiastic Postgres supporter.

Re: Migrating Facebook to MySQL 8.0

#177

Post by evanelias » Thu, Jul 22, 2021, 9:48 PM UTC

vosper wrote: post #171

evanelias wrote: post #98

Earlier quoted context omitted.

Facebook is heavily sharded, which keeps the size of each physical table at reasonable levels. Similar story at nearly every large tech company using MySQL, aside from some more recent ones that go for the painful "shove everything in a huge singular AWS Aurora instance" approach :)

> the painful "shove everything in a huge singular AWS Aurora instance" approach :) I haven't used Aurora - what's painful about this approach (apart from potentially being locked-in to AWS / Aurora)?

Aurora lets you grow your db storage up to 128 TB, but things become difficult and slow long before this point. This is true operationally (e.g. schema changes, restoring from a backup, etc) as well as potentially for application / query performance issues.

That said, sharding can be very painful too in other ways. But at least it permits infinite horizontal scaling, and it makes many operational tasks easier since they can be performed more granularly on each smaller shard.

Besides, once you get anywhere near Aurora's storage limit, you'll need to shard anyway. So it really just buys you some time before the inevitable.

Re: Migrating Facebook to MySQL 8.0

#178

Post by MuffinFlavored » Thu, Jul 22, 2021, 9:50 PM UTC

ksec wrote: post #6

MySQL 5.6 was released on 5 February 2013 . MySQL 8.0 was released on 19 April 2018 . >The 8.0 migration has taken a few years so far. We have converted many of our InnoDB replica sets to running entirely on 8.0. At the scale of Facebook I wonder if they are the largest MySQL user on the planet. And I take this opportunity to ask, does anyone know how does the MySQL roadmap works? What sort of features are coming or…

> At the scale of Facebook I wonder if they are the largest MySQL user on the planet

I have to imagine they use other databases as well, right? We're much smaller than Facebook and we have some Mongo and some Postgres and a mix of Go/Rust/Scala/node.js.

I wonder how many boxes they have dedicated to be being MySQL machines. I wonder how big their tables are and how many writes per second they see during the day.

Re: Migrating Facebook to MySQL 8.0

#179

Post by FpUser » Thu, Jul 22, 2021, 10:02 PM UTC

>"MySQL, an open source database developed by Oracle, "

Sorry but there is a difference between "developed" as in originally created and "being developed" which is the case here in relation to Oracle.

Re: Migrating Facebook to MySQL 8.0

#180

Post by dekhn » Thu, Jul 22, 2021, 10:10 PM UTC

Thaxll wrote: post #16

ksec wrote: post #6

MySQL 5.6 was released on 5 February 2013 . MySQL 8.0 was released on 19 April 2018 . >The 8.0 migration has taken a few years so far. We have converted many of our InnoDB replica sets to running entirely on 8.0. At the scale of Facebook I wonder if they are the largest MySQL user on the planet. And I take this opportunity to ask, does anyone know how does the MySQL roadmap works? What sort of features are coming or…

Youtube also use MySQL.

google internally has always had a split mind about mysql. It never really jibed with the underlying google architecture but played an important role in multiple products (at least partly due to its replication features). Replacing it completely has turned out to be a huge challenge. I gained a fair amount of respect for mysql after seeing it applied at scale, but it was also a bit scary.