Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Uber Engineering Switched from Postgres to MySQL

eng.uber.com

Post reply on HN
Print view View original

301–306 of 306 posts

Re: Why Uber Engineering Switched from Postgres to MySQL

#301

Post by snuxoll »

LoSboccacc wrote: post #274
fanf2 wrote: post #269

Earlier quoted context omitted.

Does repmgr handle most of this?

most of it. but in the master-slave configuration the clients can only work if connecting to the master, and when the master switch, scripts needs to go to each clients and update their config. not immensely bad if you also use pgbouncer so you can do it on the fly without restarting the whole client, but exceptionally vulnerable to split brains and the like.

I've handled this with a DNS CNAME record that points to the active master, though pgBouncer is a valid solution as well.

Re: Why Uber Engineering Switched from Postgres to MySQL

#302

Post by mace »

I think a fair summary is "We had a few problems with PostgreSQL mostly due to our rapid growth. We rethought the problem and changed the way we use a relational database for large-scale storage and are now using MySQL as a dumb key-value store."

The conclusion reinforces this: "Postgres served us well in the early days of Uber, but we ran into significant problems scaling Postgres with our growth."

I read this a both endorsement of PostgreSQL as well as highlighting some of the problems that any large-scale use of it would run into.

Re: Why Uber Engineering Switched from Postgres to MySQL

#303

Post by riking »

yeukhon wrote: post #143
jedberg wrote: post #132

Earlier quoted context omitted.

Open db connection get data for receipt generate receipt send email write success to database close connection To a junior programmer this would probably look reasonable, and to be fair, it takes some experience and getting burned, or good training, to know it is not.

Thanks for the illustration. But if Uber team was using ORM, then expect ORM to take care of the pool of connections. get data for receipt would be a SELECT and only write success to database would do INSERT or UPDATE. I expect junior programmer to complete the above in at least two SQL calls. I have a feeling they were trying something smart. EDIT : hmm reading the other commenter above, probably they are trying to…

Grapevine says that they were holding a transaction open for the entire duration of the Uber ride.

Re: Why Uber Engineering Switched from Postgres to MySQL

#305

Post by fogetti »

mamcx wrote: post #169
spotman wrote: post #120

Earlier quoted context omitted.

Try enforcing this on teams that use ORMs like hibernate with 500 developers. Super, duper, common issue, you will find this at every large shop at some point in its life time, usually around the time of hiring people and expanding extremely fast, and taking on some tech debt. All functions of extreme scale, hyper growth, and yeah, not following the absolute best practices all the time, but tech debt is like any debt…

Something that hit me researching how build a relational language and thinking how hard could be to remove SQL and put instead my own flavor, (super-oversimplification): - SQL "bad" - ORM "good" - NoSql apis "good" So, the thing is that when facing with SQL "everyone" try to "abstract" it more. Or instead use NoSql, because is "easier". Fine. Then if exist a market demand for a better API for the databases, why the d…

I wish your thinking would be more widespread. I never understood how come that restful HTTP became very popular without having any universal query language around and merely providing an HTTP client with each service built, while in the DB world everybody is obsessed with SQL? Why not just provide the developer with an API/library for the underlying algorithms/datastructures?

For example I hate so much when I have to read mySQL documentation jus to find out what went wrong when I used ORDER BY and the optimization didn't kick in? I mean, if I have to keep in my head the whole optimization mechanism of the DB engine and all the steps of the filesort algorithm going on behind the scenes just to write a single wretched SQL statement, then what's the point of SQL in the first place?

And this was just one example. The truth is that the SQL abstraction is leaking. Most of the commands don't encapsulate anything at all.

Re: Why Uber Engineering Switched from Postgres to MySQL

#306

Post by Groxx »

_Codemonkeyism wrote: post #117
busterarm wrote: post #87

Earlier quoted context omitted.

Can't agree with this post enough. I find their whole writeup to be terribly myopic. When they started their service, Postgres was almost certainly the right choice for what they were building and their MySQL setup was not. Now Postgres is less effective for them. These kind of tech switches are _inevitable_ if you're making the right choices for your organization. This strikes me as very similar to the article where…

"1B+ rows in it with mysql?" Been there, really no fun.

Also been there. Percona Toolkit works fine (though it's not load-aware enough, so we changed the backfill logic).

Plus, in 5.7, there are a fair number of online-DDL changes, and adding indexes is (usually) one of them.

Post reply on HN