Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Uber Engineering Switched from Postgres to MySQL

eng.uber.com

Post reply on HN
Print view View original

181–190 of 306 posts

Re: Why Uber Engineering Switched from Postgres to MySQL

#181

Post by pella »

leblancfg wrote: post #166
pella wrote: post #58

Roadmaps ( PostgreSQL ) 2016-2017-... * Postgres Professional roadmap ( Pluggable storages, Multimaster cluster with sharding, Effective partitioning, Adaptive query planning, Page-level data compression, Connection pooling, Native querying for jsonb with indexing support, ....) https://wiki.postgresql.org/wiki/Postgres_Professional_roadm... * EnterpriseDB database server roadmap ( Parallelism, Replication, Vertical…

My big question here is why they decided to move over to MySQL instead of using the Citus (also open source) Postgres extension. They don't mention it, so we don't know whether it was considered, and if so, why it was not selected. Postgres' rapid feature growth in the past years is interesting in itself.

Citus is Open Sourced since "Mar 2016", so it is not so old.

https://github.com/citusdata/citus/blob/release-5.1/CHANGELO...

"citus v5.0.0 (March 24, 2016)

- Public release under AGPLv3

- PostgreSQL extension compatible with PostgreSQL 9.5 and 9.4"

Re: Why Uber Engineering Switched from Postgres to MySQL

#182

Post by maximegarcia »

_Codemonkeyism wrote: post #117
busterarm wrote: post #87

Earlier quoted context omitted.

Can't agree with this post enough. I find their whole writeup to be terribly myopic. When they started their service, Postgres was almost certainly the right choice for what they were building and their MySQL setup was not. Now Postgres is less effective for them. These kind of tech switches are _inevitable_ if you're making the right choices for your organization. This strikes me as very similar to the article where…

"1B+ rows in it with mysql?" Been there, really no fun.

https://github.com/soundcloud/lhm

Saved my life a few time :)

Re: Why Uber Engineering Switched from Postgres to MySQL

#183

Post by dijit »

darkr wrote: post #178
mml wrote: post #157

Earlier quoted context omitted.

Experience with Mysql replication (even simple master/slave) leads me to believe Uber is going to have some rather nasty surprises at some point.

MySQL has had solid and flexible replication options for a long time. Postgres has only just started to catch up in the last couple of years. Don't get me wrong, I would generally choose Postgres over MySQL for an RDBMS with replication requirements these days, but I'm not sure I would have made that same descion a few years ago. There are valid reasons that long established companies such as Google, Twitter, Faceboo…

it's not that "replication options just started to catch on" with postgresql, it was that when the postgresql team take something into core it has to be fully fleshed out and fully supported with no surprises.

And with that in mind, I would take the slow to deliver replication options postgresql supplies in base over the previous third-party options which where comparable to the built-in options in mysql... and I'd take it over mysql's built-in options.

and I've used both postgresql and mysql in highly transactional/reliable production environments for over 5 years.

Re: Why Uber Engineering Switched from Postgres to MySQL

#184

Post by LoSboccacc »

jedberg wrote: post #116

> MySQL supports multiple different replication modes: > Statement-based replication replicates logical SQL statements (e.g., it would literally replicate literal statements such as: UPDATE users SET birth_year=770 WHERE id = 4) Postgres has that too (using a 3rd party tool, but it's an officially supported tool). We were using it on reddit 10 years ago. It caused a lot of problems. I wouldn't call that an advantage…

I've tried to setup a replicated postgres with autofailover and it honestly is a pita.

the only sources of failover are rando scripts over the internet, that you have to download hammer in to your version dialect and hope you don't trigger one of the many uncovered failover modes.

sure log shipping works, but that's far, FAR from a working solution. the gap requires ton of development hour, testing etc.

can't really blame people for using things with mature tooling.

(but I didn't switch to MySQL that'd be madness! there are plenty of good, replicated redundant stores out there, both nosql and sql)

Re: Why Uber Engineering Switched from Postgres to MySQL

#185

Post by LoSboccacc »

mamcx wrote: post #169
spotman wrote: post #120

Earlier quoted context omitted.

Try enforcing this on teams that use ORMs like hibernate with 500 developers. Super, duper, common issue, you will find this at every large shop at some point in its life time, usually around the time of hiring people and expanding extremely fast, and taking on some tech debt. All functions of extreme scale, hyper growth, and yeah, not following the absolute best practices all the time, but tech debt is like any debt…

Something that hit me researching how build a relational language and thinking how hard could be to remove SQL and put instead my own flavor, (super-oversimplification): - SQL "bad" - ORM "good" - NoSql apis "good" So, the thing is that when facing with SQL "everyone" try to "abstract" it more. Or instead use NoSql, because is "easier". Fine. Then if exist a market demand for a better API for the databases, why the d…

it's very hard to abstract away from SQL. to be honest, many nosql have integrated concepts similar to grouping, having, joins in their API as well as they matured, because that's what data elaboration needs.

Re: Why Uber Engineering Switched from Postgres to MySQL

#187

Post by scotty79 »

Funny how the article is just:

- we used X in a fashion that suited us best

- it caused us problems Y because of some technicalities of X

- so we switched to Z and we could avoid Y thanks to how Z handles the technicalities differently than Y

and the top rated HN comments are:

- you used the X wrong

- all the technicalities of X that caused you problems Y are actually superior features of X

Re: Why Uber Engineering Switched from Postgres to MySQL

#188

Post by liuliu »

mamcx wrote: post #169
spotman wrote: post #120

Earlier quoted context omitted.

Try enforcing this on teams that use ORMs like hibernate with 500 developers. Super, duper, common issue, you will find this at every large shop at some point in its life time, usually around the time of hiring people and expanding extremely fast, and taking on some tech debt. All functions of extreme scale, hyper growth, and yeah, not following the absolute best practices all the time, but tech debt is like any debt…

Something that hit me researching how build a relational language and thinking how hard could be to remove SQL and put instead my own flavor, (super-oversimplification): - SQL "bad" - ORM "good" - NoSql apis "good" So, the thing is that when facing with SQL "everyone" try to "abstract" it more. Or instead use NoSql, because is "easier". Fine. Then if exist a market demand for a better API for the databases, why the d…

LINQ: https://msdn.microsoft.com/en-us/library/bb397926.aspx

Most of the time, the problem is to "embed" a foreign language (SQL) into an existing one (Python, Ruby, etc.), but you can do it in a proper way such as LINQ, then you have type-checking etc.

Re: Why Uber Engineering Switched from Postgres to MySQL

#189

Post by mamcx »

LoSboccacc wrote: post #185
mamcx wrote: post #169

Earlier quoted context omitted.

Something that hit me researching how build a relational language and thinking how hard could be to remove SQL and put instead my own flavor, (super-oversimplification): - SQL "bad" - ORM "good" - NoSql apis "good" So, the thing is that when facing with SQL "everyone" try to "abstract" it more. Or instead use NoSql, because is "easier". Fine. Then if exist a market demand for a better API for the databases, why the d…

it's very hard to abstract away from SQL. to be honest, many nosql have integrated concepts similar to grouping, having, joins in their API as well as they matured, because that's what data elaboration needs.

But still everyone do it. That must tell something, right?

And I'm not talking about NoSql borrowing some relational concepts, but the opposite, and more directly, the API.

I know that most folks (including the ones that downvote!) have no clue what I'm talking about, because almost nobody (recently) have experience in talking against a database without SQL. Is like if the only way to talk to a NoSql was using Json + REST. That constrain badly your mind.

The inner relational model is far more rich, and simpler, SQL was not designed to be used by developers and it show (specially when you have contrived syntax as with CTE)

And before you ask what is the problem with CTE, is exactly because the way to have that is create a contrived syntax that obscure what is goin on. SQL is too restricted, yet too broad for interfacing.

Re: Why Uber Engineering Switched from Postgres to MySQL

#190

Post by oneweekwonder »

harel wrote: post #170
bogomipz wrote: post #164

Earlier quoted context omitted.

Could you elaborate?

You can read it in their next blog post: "How we migrated back to PostgreSql from mySql after migrating from PostgreSql to mySql". My experience with mySql replication was OK, but I did nothing fancy - just master/slave standard stuff. However the database itself was allowing you to do stuff by default (at the time at least) that is really bad. Not to mention scaling while I was using it was pretty much "you're on yo…

Personally I prefer Postgres. But at the shop we use MySQL. One of the leads personally believe in Percona Server[0] and mentions some of the old mysql folks work there. I like their docs and the idea of a more tuned mysql config.

[0]: https://www.percona.com/doc/percona-server/5.7/index.html

Post reply on HN