This might explain a lot of the performance problems we've seen trying to use Postgresql as an event store, dumping some 10s of millions of rows/day into a table that has a few indexes (no foreign keys, but trying to speed up queries). Sounds like it's time to investigate alternatives.
Why we lost Uber as a user
151–160 of 310 posts
Re: Why we lost Uber as a user
#152Earlier quoted context omitted.
For the kind of stuff Uber stores, they may actually be doing it wrong (given what that Postgre mailing list post says) because that is one hell of an ugly use case for any DB. I would have tried solving it by loading a dual E5v4 server full of 3TB and a slew of SSDs for L2ARC+ZIL under ZFS: more SSDs > bigger SSDs because the absolute worst case SSD performance that any and all SSDs suffer from is random reads (not…
> Given that, there are only four dbs worth using: Postgre, Oracle, DB2, and MS SQL This is total bullshit. Even a modest size database (3-5 terabyte) for a small company like mine, chokes a relational database bigtime on time series. Especially the modern use cases where you are ingesting fast. You need Cassandra or Hbase which are very serious pieces of technology and are definitely "worth using", and will crush an…
Suggested edit: s/This is total bullshit/Perhaps you overlooked something:/.
Otherwise, this would have been a good comment, I think.
Re: Why we lost Uber as a user
#153Earlier quoted context omitted.
or you can use a materialized view https://www.postgresql.org/docs/current/static/rules-materia...
I don't think materialized views are an answer to this problem, precisely because postgres materialized views do not automatically update when the underlying data has changed.
Sure, you have to write the code to do updates; but "materialized views" are a technique long used even in SQL databases that had no automated support for them; denormalized derived tables that are used for regular, recurring queries while the DB retains normalized base tables are a common thing (heck, in Enterprise environments, I've more than once run into a setup where custom maintenance of jury-rigged materialized views are used in a database system with strong materialized view support simply because the system has been around and maintained longer than the server software has had that support.)
Re: Why we lost Uber as a user
#154Earlier quoted context omitted.
Yes, I have a lot more respect for this than say MongoDb which claims to be great at everything.
For the kind of stuff Uber stores, they may actually be doing it wrong (given what that Postgre mailing list post says) because that is one hell of an ugly use case for any DB. I would have tried solving it by loading a dual E5v4 server full of 3TB and a slew of SSDs for L2ARC+ZIL under ZFS: more SSDs > bigger SSDs because the absolute worst case SSD performance that any and all SSDs suffer from is random reads (not…
This was my first thought, and generally what I always think when I hear about update heavy use cases. Typically they represent some issue in the object model, and often driven by an ORM layer.
I always ask is there a need for absolute correctness, and would this be better handled by a batch processing method.
I'm not sure any hardware would have solved their issue though. It may not have actually been disk speed, but issues in the mvcc design itself.
Re: Why we lost Uber as a user
#155The attitude with which the article is discussed in the mailing list is admirable.
Couldn't help contrasting it to some popular programming language mailing lists, especially a CSP inspired language by a major search company.
Couldn't help contrasting it to the comments on this very site.
Re: Why we lost Uber as a user
#156Earlier quoted context omitted.
That might be more than slightly confusing here on Hacker News.
psql? pgsql?
pg sequel or pg ess-queue-el
are both very good, short, and difficult to confuse with another project or product
i think psql refer specifically to the command line tool of pgsql
Re: Why we lost Uber as a user
#157Earlier quoted context omitted.
psql? pgsql?
Neither works: psql is the bundled postgres shell and (pl/)pgsql is posgres's procedural SQL extension (inspired by Oracle/s pl/sql) so both are pretty ambiguous.
pgsql is exclusive to postgresql, and from the context, it should very easy to know if you talking about the programming language or postgresql
Re: Why we lost Uber as a user
#158The PoststgreSQL project never fails to impress me. I know that there are some use cases which are not currently covered by it versus alternatives, but I have consistently got the feeling that everybody involved in the project is supremely professional and interested in building an excellent database – and the focus is on how to work to fix these use cases, instead of pointless mudslinging. Class act.
> The PoststgreSQL project never fails to impress me. Except the name is a bit clunky and hard to write ;)
> The PostststgreSQL project never fails to impress me.
Let's see how many iterations we'll need for people to actually notice what was pointed out here ;)
Re: Why we lost Uber as a user
#159Earlier quoted context omitted.
> The PoststgreSQL project never fails to impress me. Except the name is a bit clunky and hard to write ;)
Maybe, but it's a much better name than mysql. I feel like a little kid every time I say "my" anything.
For the record, My is Lars Wirzenius's daughter.
Re: Why we lost Uber as a user
#160Does anyone else think the scenario in the explanation is an unreasonable request to make of a relational database? I think that if you've created a design that requires you to update a 50K row table 500 times a second that itself is heavily indexed and used heavily in joins, you have a software design problem more than a database problem. I wouldn't expect any database to handle that and am surprised that mysql does…
You're not wrong. But mostly we haven't collectively agreed that relational databases aren't great for highly-indexed rapid-update join tables.
I think we will at some point. That's the primary original use case for a lot of NoSQL, and the reason Twitter had so much trouble with relational databases.
But these are cultural understandings, and those move slowly. Also, we're poorly (collectively) equipped to handle subtlety in these discussions, so mostly we're trying to move from "relational databases are perfect for all use cases" to "NoSQL databases are perfect for all use cases" -- which is even less true, not more true.
Culturally, this is a hard thing to keep in our collective brain.