Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we lost Uber as a user

postgresql.org

Post reply on HN
Print view View original

231–240 of 310 posts

Re: Why we lost Uber as a user

#231

Post by zd4akaq85a »

web007 wrote: post #212
methehack wrote: post #145

Does anyone else think the scenario in the explanation is an unreasonable request to make of a relational database? I think that if you've created a design that requires you to update a 50K row table 500 times a second that itself is heavily indexed and used heavily in joins, you have a software design problem more than a database problem. I wouldn't expect any database to handle that and am surprised that mysql does…

Absolutely - going from Postgres to MySQL is only trading one set of problems for another. It's a longer runway, but not infinite. Their exact use-case is what things like Cassandra were built for - insanely high writes / updates. They're also built to split your load across N systems, as long as you're still using a monolithic database (even with read replicas) you physically can't get the same performance that you…

An Uber engineer at a conference said that none of the open-source NoSQL systems could handle their load, and they they had to heavily hack one of them (which I think was Cassandra but the memory is vague) to get the last bit of performance out of it while they were building Schemaless.

Re: Why we lost Uber as a user

#232

Post by rdslw »

joshmanders wrote: post #66
ZanyProgrammer wrote: post #55

Earlier quoted context omitted.

That's the company line, sure.

The IRS sees it that way too.. Just because you consider them employees doesn't make them employees.

Only in USA. And this only after paying $ 100M to settle lawsuit (http://www.latimes.com/business/technology/la-fi-tn-uber-law...) - they literally paid for drvrs not being considered employees in LA. Until next case.

In EU, they're considered what they are: nice gimmick to shift company cost to workers, and taxate workers salary with 20% tax, while having (on company side) fixed cost: https://www.jacobinmag.com/2016/07/uber-drivers-app-rideshar...

Re: Why we lost Uber as a user

#233

Post by specialist »

methehack wrote: post #145

Does anyone else think the scenario in the explanation is an unreasonable request to make of a relational database? I think that if you've created a design that requires you to update a 50K row table 500 times a second that itself is heavily indexed and used heavily in joins, you have a software design problem more than a database problem. I wouldn't expect any database to handle that and am surprised that mysql does…

Agreed, I got stuck on that point too. Leaning on UPDATE like that is a (schema, app) design problem, not a database problem.

Re: Why we lost Uber as a user

#234

Post by qwertyuiop924 »

vegabook wrote: post #102
DiabloD3 wrote: post #33

Earlier quoted context omitted.

For the kind of stuff Uber stores, they may actually be doing it wrong (given what that Postgre mailing list post says) because that is one hell of an ugly use case for any DB. I would have tried solving it by loading a dual E5v4 server full of 3TB and a slew of SSDs for L2ARC+ZIL under ZFS: more SSDs > bigger SSDs because the absolute worst case SSD performance that any and all SSDs suffer from is random reads (not…

> Given that, there are only four dbs worth using: Postgre, Oracle, DB2, and MS SQL This is total bullshit. Even a modest size database (3-5 terabyte) for a small company like mine, chokes a relational database bigtime on time series. Especially the modern use cases where you are ingesting fast. You need Cassandra or Hbase which are very serious pieces of technology and are definitely "worth using", and will crush an…

I consider your argument misinformed.

Technology moves fast. Very fast. If a piece of tech gets old enough to be called a dinosaur, and is still run at brand-new companies, it's doing something right, even if it's not for your usecase.

And the fact is, not all of us have 3-5TB time seriesdatabases. And time series data can cause a lot of DBs to choke.

The point is, I'm glad Cassandra and Hbase, or a dedicated TSDB, or whatever, work for you. Your usecase is not the same as the rest of us, and assuming that it is is causing you to come to some incorrect conclusions.

Re: Why we lost Uber as a user

#235

Post by jacques_chester »

PeCaN wrote: post #180
vegabook wrote: post #102

Earlier quoted context omitted.

> Given that, there are only four dbs worth using: Postgre, Oracle, DB2, and MS SQL This is total bullshit. Even a modest size database (3-5 terabyte) for a small company like mine, chokes a relational database bigtime on time series. Especially the modern use cases where you are ingesting fast. You need Cassandra or Hbase which are very serious pieces of technology and are definitely "worth using", and will crush an…

There are column-oriented relational databases for that. DB2 and SQL Server both implement columnar systems for OLAP-esque workloads pretty well, and I think Oracle has something like that as well. In the Free software space, MonetDB is pretty good and I believe there's some Postgres extension to store data column-wise (which is probably not competitive with ground-up column stores like MonetDB, but beats row-wise st…

For columnar and massively-parallel workloads, Greenplum was opensourced this year[0]. It was originally forked from PostgreSQL 8.2.

There's also Apache HAWQ (incubating), which takes Greenplum's SQL parser and distributed query planner to use as front-ends for Hadoop[1].

Disclaimer: I work for Pivotal, which opensourced these systems.

[0] http://greenplum.org/

[1] http://hawq.incubator.apache.org/

Re: Why we lost Uber as a user

#236

Post by qwertyuiop924 »

oneloop wrote: post #20
spriggan3 wrote: post #10

Earlier quoted context omitted.

> MongoDb which claims to be great at everything lol right, with MongoDb, Map Reduce is a joke, GridFS is slow and barely usable, the storage is extremely inefficient, the "query engine" slow, and don't get me started on their "full text search" engine. MongoDb is a successful marketing stunt in the "Nodejs era".

Also, wasn't there a ridiculous issue that they had where the db can't be bigger than 4gb on a 32 bit file system because that's the largest size a file can have...?

Umm... Wow, mongo. Boy, am I glad I didn't decide to pick you.

Let's get some statistics. MySQL and Postgres both have and undefined max DB size, and max table sizes upwards of 16 and 32 TB, respectively.

SQLite has no max table size I could find, but had a max DB size of 140 TB. 140TB. And this is from a database that reccomends not using it if your data grows too large.

Re: Why we lost Uber as a user

#237

Post by PeCaN »

jamespo wrote: post #215
Roboprog wrote: post #152

Earlier quoted context omitted.

Suggested edit: s/This is total bullshit/Perhaps you overlooked something:/. Otherwise, this would have been a good comment, I think.

I think there's something to be said for the Linus style

Yeah, but generally only when you're actually correct. If you're not, you look like an idiot. Apparently the parent hasn't heard of column-store relational databases (which are kinda great for time series).

(I'm most familiar with SQL Server; DB2 and Oracle apparently have similar functionality.)

Apparently their 3-5TB time series table chokes ‘dinosaur’ relational databases. For a time series table you should almost certain be using a column-store index with CREATE CLUSTERED COLUMNSTORE INDEX. Depending on how much they're querying vs inserting, that alone could prevent SQL Server from choking. If it's more insertion-heavy, but they still need to run a lot of ad-hoc queries, SQL Server 2016 supports using a nonclustered column-store index alongside a row-store. You can insert with the row-store and query with the column-store. If it's still choking with that, it might be time to check out the database structure and see what's up. One of the more common killers is putting a lot of data in the time series table. Generally it's more efficient to have a clustered column-store index table that contains the timestamp and ids to metadata, and keep the metadata itself in row-store tables with appropriate indexes.

Re: Why we lost Uber as a user

#238

Post by PeCaN »

jacques_chester wrote: post #235
PeCaN wrote: post #180

Earlier quoted context omitted.

There are column-oriented relational databases for that. DB2 and SQL Server both implement columnar systems for OLAP-esque workloads pretty well, and I think Oracle has something like that as well. In the Free software space, MonetDB is pretty good and I believe there's some Postgres extension to store data column-wise (which is probably not competitive with ground-up column stores like MonetDB, but beats row-wise st…

For columnar and massively-parallel workloads, Greenplum was opensourced this year[0]. It was originally forked from PostgreSQL 8.2. There's also Apache HAWQ (incubating), which takes Greenplum's SQL parser and distributed query planner to use as front-ends for Hadoop[1]. Disclaimer: I work for Pivotal, which opensourced these systems. [0] http://greenplum.org/ [1] http://hawq.incubator.apache.org/

Whoa, that looks quite slick. Thanks for your work!

Re: Why we lost Uber as a user

#239

Post by jacques_chester »

PeCaN wrote: post #238
jacques_chester wrote: post #235

Earlier quoted context omitted.

For columnar and massively-parallel workloads, Greenplum was opensourced this year[0]. It was originally forked from PostgreSQL 8.2. There's also Apache HAWQ (incubating), which takes Greenplum's SQL parser and distributed query planner to use as front-ends for Hadoop[1]. Disclaimer: I work for Pivotal, which opensourced these systems. [0] http://greenplum.org/ [1] http://hawq.incubator.apache.org/

Whoa, that looks quite slick. Thanks for your work!

All credit belongs to my peers in the Big Data division, and the engineering directors who successfully argued to opensource our entire portfolio of data tech (Greenplum, HAWQ, Gemfire and MADLib).

Re: Why we lost Uber as a user

#240

Post by vertex-four »

nilkn wrote: post #221
vertex-four wrote: post #190

Earlier quoted context omitted.

Well, not really. Their product is really a two-sided market with prices controlled by themselves - matching drivers, unregulated and not employed by Uber, to passengers. They could do that with nothing more than a call center and a spreadsheet of roughly where drivers are if the tech didn't exist, and it'd still be cheaper and avoid the monopoly of the taxi companies. Whether it'd largely avoid the ire of law enforc…

Wouldn't that more or less just make them into a taxi company? That approach just wasn't working well. The main difference between Uber and taxi companies is that Uber fully utilized modern technology from the ground up to build the platform, and as a result they were able to realize major improvements in terms of speed, reliability, and cost-effectiveness.

I think Uber is, in practice, an unregulated taxi company, allowing it to cut costs and pay its drivers less, and it would be just as popular if it had no tech at all.
Post reply on HN