but have you used MySQL?
Maybe I am too nitpicky, but I always get suspicious when people make claims about technology whose name they cannot spell right.
Why we lost Uber as a user
21–30 of 310 posts
Re: Why we lost Uber as a user
#22Earlier quoted context omitted.
Postgres uses MVCC, which is basically copy-on-write: every UPDATE to a row actually creates a new row. Once all transactions that could see the old row have finished, the old row can be pruned.
Ah, thanks for the clarification - the bloat issue makes more sense now.
[deleted]
Re: Why we lost Uber as a user
#23I don't understand if they mean that foreign key relationships are contributing to the problem. If so is it possible to turn off foreign key constaints during a big load? Would it be worth removing indexes during a big load? (Currently fighting with an etl that can't get above 500 rows / second even after using copy from)
The reason the indexes are their is because the table is involved in multiple JOINs, which without the indexes are very slow. I suspect it's a straight index, not a foreign key constraint.
Re: Why we lost Uber as a user
#24The attitude with which the article is discussed in the mailing list is admirable.
Couldn't help contrasting it to some popular programming language mailing lists, especially a CSP inspired language by a major search company.
Re: Why we lost Uber as a user
#25Its weird seeing a post mortem for losing a user (I really want to say customer) from a piece of FOSS. Its also weird (still!) to consider Uber a tech company, rather than a company that happens to use tech.
They don't "happen to use tech", their whole business is based on tech, and not in the way a bank would use tech say, but especially on tech they create (the client, the reservations system, extra services, etc).
It's like saying Google is just an ad company that "happens to use tech".
Re: Why we lost Uber as a user
#26I don't understand if they mean that foreign key relationships are contributing to the problem. If so is it possible to turn off foreign key constaints during a big load? Would it be worth removing indexes during a big load? (Currently fighting with an etl that can't get above 500 rows / second even after using copy from)
All indexes on a pg table need updating if any value in the row is modified, that's the write amplification. The indexes are needed for joins, they're not so important for foreign keys.
Certainly checking foreign key constraints works better with indexes, but those are usually the other way around - verifying row exists with primary key, ie only one index. Updates to primary key would benefit from index on foreign key columns, but that's much rarer.
Re: Why we lost Uber as a user
#27but have you used MySQL? Maybe I am too nitpicky, but I always get suspicious when people make claims about technology whose name they cannot spell right.
He also spelled Postgres "postgres" ;)
Re: Why we lost Uber as a user
#28The PoststgreSQL project never fails to impress me. I know that there are some use cases which are not currently covered by it versus alternatives, but I have consistently got the feeling that everybody involved in the project is supremely professional and interested in building an excellent database – and the focus is on how to work to fix these use cases, instead of pointless mudslinging. Class act.
> The PoststgreSQL project never fails to impress me.
Except the name is a bit clunky and hard to write ;)
Re: Why we lost Uber as a user
#29I agree it's a nice thing. But some kind of answer may also be good. Like restructuring your data to become faster. I also wonder what happened the last few (10) years. When I was in university I'm pretty sure I learned that JOIN was Satan's mother and if you have a big DB you need to avoid JOINs as much as possible. That's not a big deal today anymore, it seems.
That kind of thinking is probably what spawned the whole "do the join in the app, not the database" anti-pattern. The truth is, the database is going to be much faster at performing a join than loading the contents of two tables into your app and iterating. If you need the data that results from doing a join, doing a join is the best way to get it. Unless you already have your entire database in-memory in your app, t…
What you are missing here is "denormalization" -- e.g. many-to-many relationships. You can either use a JOIN with a table on a "normalized" database, or keep managing the result of the join in application code. Loading the entire tables into application code very seldom has anything to do with it...
A more realistic example is, do you get Alice's pets by doing a JOIN on tables Person, Pet, PetOwnedByPerson ("SQL") -- or by having an array column "pets" in Person? ("NoSQL")
Re: Why we lost Uber as a user
#30The PoststgreSQL project never fails to impress me. I know that there are some use cases which are not currently covered by it versus alternatives, but I have consistently got the feeling that everybody involved in the project is supremely professional and interested in building an excellent database – and the focus is on how to work to fix these use cases, instead of pointless mudslinging. Class act.
True, they're admitting there's a problem (which exists for this very specific user case) and not just deflecting the criticism.
I'm impressed, I'd say a lot of projects couldn't have handled this so nicely