Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we lost Uber as a user

postgresql.org

Post reply on HN
Print view View original

41–50 of 310 posts

Re: Why we lost Uber as a user

#41

Post by LeifCarrotson »

rimantas wrote: post #21

but have you used MySQL? Maybe I am too nitpicky, but I always get suspicious when people make claims about technology whose name they cannot spell right.

In this case, Fabian's CV (linked from his GitHub profile, linked from his HN profile) does claim MySQL experience - correctly capitalized.

But yes, I also get suspicious when people can't spell a technology correctly. For example, anyone who has spent time reading some documentation will know that Lua is capitalized in title case, and not written LUA.

Re: Why we lost Uber as a user

#42

Post by viraptor »

nubela wrote: post #36
DiabloD3 wrote: post #33

Earlier quoted context omitted.

For the kind of stuff Uber stores, they may actually be doing it wrong (given what that Postgre mailing list post says) because that is one hell of an ugly use case for any DB. I would have tried solving it by loading a dual E5v4 server full of 3TB and a slew of SSDs for L2ARC+ZIL under ZFS: more SSDs > bigger SSDs because the absolute worst case SSD performance that any and all SSDs suffer from is random reads (not…

You wouldn't wanna be an Uber investor because choice of DBMS? As a programmer, I think this is the problem with developers thinking that technical problems are a bigger deal than it is.

Mysql eats and corrupts data by design. For a company responding to real time events in physical world, that can be a big issue. I know they're trying to improve their defaults lately, but a lot of weird behaviour remains. And you don't have to be an expert DBA to know that choosing a technology known for silent data corruption is risky.

Re: Why we lost Uber as a user

#43

Post by DiabloD3 »

nubela wrote: post #36
DiabloD3 wrote: post #33

Earlier quoted context omitted.

For the kind of stuff Uber stores, they may actually be doing it wrong (given what that Postgre mailing list post says) because that is one hell of an ugly use case for any DB. I would have tried solving it by loading a dual E5v4 server full of 3TB and a slew of SSDs for L2ARC+ZIL under ZFS: more SSDs > bigger SSDs because the absolute worst case SSD performance that any and all SSDs suffer from is random reads (not…

You wouldn't wanna be an Uber investor because choice of DBMS? As a programmer, I think this is the problem with developers thinking that technical problems are a bigger deal than it is.

Uber relies entirely on their database. If the database is slow, or cannot be considered reliable under extreme load, or can lose data in ways that are hard to recover, then Uber can potentially lose a lot of money especially during peak hours.

Yes, I'm aware there are systems for MySQL to handle failure, but I'm also aware of the systems for Postgre, and Postgre's failure handling seem to be far saner and easier to recover from. Defense in depth against failure is easier in Postgre from my experience.

This would be like Elon Musk blogging about how "oh yeah, sometimes the brand new Tesla factory shuts down completely because sometimes the power goes out; but we're using really popular well known power distribution systems, so we're industry compliant, so everything is okay."

If Elon Musk blogged that, HN would go apeshit, and rightfully so.

Re: Why we lost Uber as a user

#44

Post by lmm »

mrfusion wrote: post #19

I don't understand if they mean that foreign key relationships are contributing to the problem. If so is it possible to turn off foreign key constaints during a big load? Would it be worth removing indexes during a big load? (Currently fighting with an etl that can't get above 500 rows / second even after using copy from)

Indices trade reads for writes (there is no free lunch); if you index every column of your database you'll be able to do any read query reasonably fast but all writes will be slow, if you index nothing then writes will be fast but every read will be a full table scan. Presumably they have their indices because they need them for their read queries.

Re: Why we lost Uber as a user

#45

Post by brightball »

In reading that, my first thought was "why in the world would Uber do that?"

In every single performance tuning a scale story that I've read over the past decade, the very first point of order is: remove joins from high traffic queries. It seems like Uber has gone the complete opposite direction.

Re: Why we lost Uber as a user

#46

Post by erikb »

dagss wrote: post #29
mnw21cam wrote: post #18

Earlier quoted context omitted.

That kind of thinking is probably what spawned the whole "do the join in the app, not the database" anti-pattern. The truth is, the database is going to be much faster at performing a join than loading the contents of two tables into your app and iterating. If you need the data that results from doing a join, doing a join is the best way to get it. Unless you already have your entire database in-memory in your app, t…

What you are missing here is "denormalization" -- e.g. many-to-many relationships. You can either use a JOIN with a table on a "normalized" database, or keep managing the result of the join in application code. Loading the entire tables into application code very seldom has anything to do with it... A more realistic example is, do you get Alice's pets by doing a JOIN on tables Person, Pet, PetOwnedByPerson ("SQL") --…

Yes "denormalization" was the word for that. Is that still a thing?

Re: Why we lost Uber as a user

#47

Post by WoodenChair »

czep wrote: post #39
oblio wrote: post #38

Earlier quoted context omitted.

Well, the main difference I'd expect from Uber vs a bank is that the majority of Uber employees are (software) engineers while the majority of bank employees are economists.

The majority of Uber employees... are drivers. Clearly proportion of software engineers cannot be a good metric for what makes a "tech" company.

Drivers are independent contractors, not employees.

Re: Why we lost Uber as a user

#49

Post by erikb »

mnw21cam wrote: post #18
erikb wrote: post #12

I agree it's a nice thing. But some kind of answer may also be good. Like restructuring your data to become faster. I also wonder what happened the last few (10) years. When I was in university I'm pretty sure I learned that JOIN was Satan's mother and if you have a big DB you need to avoid JOINs as much as possible. That's not a big deal today anymore, it seems.

That kind of thinking is probably what spawned the whole "do the join in the app, not the database" anti-pattern. The truth is, the database is going to be much faster at performing a join than loading the contents of two tables into your app and iterating. If you need the data that results from doing a join, doing a join is the best way to get it. Unless you already have your entire database in-memory in your app, t…

I think the "do the join in the app" anti-pattern was developed by this group of people who think that in app the programming environment they know and that it's working in their dev environment and that is all that needs to be considered to make a decision.

I would generalize what you say even further: The database is faster at most of the data crunching you need.

Post reply on HN