Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we lost Uber as a user

postgresql.org

Post reply on HN
Print view View original

211–220 of 310 posts

Re: Why we lost Uber as a user

#211

Post by dubbel »

pizza234 wrote: post #116
dkersten wrote: post #98

Earlier quoted context omitted.

I was recently advised by a DB consultant whose area of expertise is MySQL that 5.7 is still too new and risky and that he would advise against upgrading for at least another 6 months or more. He feels that the releases come out much, much too unstable and unpolished and that it typically takes at least a year since release before he's comfortable running it in production. I don't know enough about MySQL to know if t…

I can confirm that the query optimizer introduced a rather serious bug (significantly suboptimal plan for queries involving low cardinality indices), which caused serious issues in our system. This, in addition to the fact that index merging has been broken in MySQL 5.6 for more than an year now (in some cases it will cause empty resultsets to be returned), and that it is still broken on MySQL 5.7

Do you happen to have a link to the bug report for the first issue you described? I'm wondering whether I saw a similar thing in a benchmark I tested.

Re: Why we lost Uber as a user

#212

Post by web007 »

methehack wrote: post #145

Does anyone else think the scenario in the explanation is an unreasonable request to make of a relational database? I think that if you've created a design that requires you to update a 50K row table 500 times a second that itself is heavily indexed and used heavily in joins, you have a software design problem more than a database problem. I wouldn't expect any database to handle that and am surprised that mysql does…

Absolutely - going from Postgres to MySQL is only trading one set of problems for another. It's a longer runway, but not infinite.

Their exact use-case is what things like Cassandra were built for - insanely high writes / updates. They're also built to split your load across N systems, as long as you're still using a monolithic database (even with read replicas) you physically can't get the same performance that you could with one of the NOSQL distributed systems. YMMV for specific performance, I've seen a huge MySQL burn through queries like butter and a Cassandra cluster crawl on a tiny data set.

The second part of your question is the real crux of the problem - "heavily indexed and used heavily in joins". Neither of those (RDBMS or NOSQL) work well in either scenario. This article indicates why PG isn't great, the Uber article indicates some of the (minor) downsides of MySQL. They're already using Schemaless to bend their RDBMS into a fancy key-value store, so they're halfway to using a real one with their home-built indexes already. For NOSQL you generally don't get joins, unless you write them yourself. You also end up with manual "write amplification" since you denormalize, write the data 10 times to index it 10 different ways. You can be smart about it so it's not exactly 10x, but you'll end up with more than your original problematic throughput, albeit spread across more systems.

Re: Why we lost Uber as a user

#213

Post by caoilte »

HappyTypist wrote: post #128
vmateixeira wrote: post #80

Earlier quoted context omitted.

Acording to UK's Companies House search, Uber's nature of business (SIC) is 74990 - Non-trading company . This is quite vague but it doesn't mention technology. Interestingly, they were also called UBER TECHNOLOGIES LTD and UBERTECHNOLOGY LIMITED at some point in time, but these companies are now dissolved.

That's because Uber UK is a pure support office, at least that's what their lawywers and accountants argue. The real company in Luxembourg.

If that were true the nature of their company would be 'tax dodge'

Re: Why we lost Uber as a user

#214

Post by brazzledazzle »

_RPM wrote: post #144
Finnucane wrote: post #136

Earlier quoted context omitted.

Adult supervision required?

Reddit styles puns aren't allowed here, that's why you're getting downvoted.

This is getting off topic (and maybe this whole thread should be forked and isolated) but is it explicitly (or implicitly) against any rules/moderation guidelines or just discouraged by the community? Don't get me wrong though, I don't care for them and I believe they do more harm than good. Just curious.

Re: Why we lost Uber as a user

#215

Post by jamespo »

Roboprog wrote: post #152
vegabook wrote: post #102

Earlier quoted context omitted.

> Given that, there are only four dbs worth using: Postgre, Oracle, DB2, and MS SQL This is total bullshit. Even a modest size database (3-5 terabyte) for a small company like mine, chokes a relational database bigtime on time series. Especially the modern use cases where you are ingesting fast. You need Cassandra or Hbase which are very serious pieces of technology and are definitely "worth using", and will crush an…

Suggested edit: s/This is total bullshit/Perhaps you overlooked something:/. Otherwise, this would have been a good comment, I think.

I think there's something to be said for the Linus style

Re: Why we lost Uber as a user

#216

Post by radiowave »

bluejekyll wrote: post #104
oneloop wrote: post #20

Earlier quoted context omitted.

Also, wasn't there a ridiculous issue that they had where the db can't be bigger than 4gb on a 32 bit file system because that's the largest size a file can have...?

While I don't care about mongoDB, who's running 32bit anymore? And for a production DB?

For example, the 32 bit mongoDB instance that's managing a grand total of 2 wifi access points here.

Re: Why we lost Uber as a user

#217

Post by tomcam »

angelbob wrote: post #160
methehack wrote: post #145

Does anyone else think the scenario in the explanation is an unreasonable request to make of a relational database? I think that if you've created a design that requires you to update a 50K row table 500 times a second that itself is heavily indexed and used heavily in joins, you have a software design problem more than a database problem. I wouldn't expect any database to handle that and am surprised that mysql does…

You're not wrong. But mostly we haven't collectively agreed that relational databases aren't great for highly-indexed rapid-update join tables. I think we will at some point. That's the primary original use case for a lot of NoSQL, and the reason Twitter had so much trouble with relational databases. But these are cultural understandings, and those move slowly. Also, we're poorly (collectively) equipped to handle sub…

That was an elegantly nuanced answer with just the right amount of context. Thank you.

Re: Why we lost Uber as a user

#218

Post by brazzledazzle »

spacecop wrote: post #185
_RPM wrote: post #144

Earlier quoted context omitted.

Reddit styles puns aren't allowed here, that's why you're getting downvoted.

what an enlightened community of robot people

I don't think it's that at all. Pun threads generally make for very boring reading/discussion and tend to rise to the top because they're quick to digest, uncontroversial and mildly amusing. If you're looking for more substantial comments (arguably most HN users are) these can be a real chore to slog through.

Re: Why we lost Uber as a user

#219

Post by rplst8 »

menzoic wrote: post #17
ZanyProgrammer wrote: post #13

Its weird seeing a post mortem for losing a user (I really want to say customer) from a piece of FOSS. Its also weird (still!) to consider Uber a tech company, rather than a company that happens to use tech.

Just curious, what's your definition of "Tech Company"? All services provided by Uber are purely technical. Drivers and Riders are customers of Uber's technology. The full name of the company is "Uber Technologies Inc."

I wasn't sure what to think of that comment either when I first read it, but I sort of see where he is coming from. One side you have companies like Oracle, Microsoft, and IBM types that actually develop new forms of technology and sell the technology to people. Then there are companies that leverage technology in other industries to "disrupt" like OpenTable, Uber, and AirBnB. Then there are companies like Google and Facebook that straddle that line. They've developed and contributed back huge advances in technology, they mostly make their money from another industry, but also sell some tech in certain areas (mostly Google).

Re: Why we lost Uber as a user

#220

Post by GrayShade »

Jweb_Guru wrote: post #174
geophile wrote: post #118

Question about Postgres architecture: Why were secondary indexes designed to refer to ctids instead of primary keys?

Among other things, there's a substantial performance penalty for secondary index lookups with clustered indices (since they need to traverse two index structures).

I'm not sure about the other databases, but in MS SQL you can include columns in the index leaves. This mitigates the need for the second lookup and can be even faster than the Postgres approach.
Post reply on HN