Live data from Hacker News

The Uber Engineering Tech Stack, Part I: The Foundation

eng.uber.com

81–90 of 194 posts

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#81

Earlier quoted context omitted.

It's easy to imagine the simplest stack that can serve the core features of any service, and that is well served by a single box. What's missing from the picture is the infrastructure to replicate this 500 times by separate teams, monitoring all of it, backup, auditing, aggregating customer and business metrics, back-office systems, and more. Plus the fact that these things always grow organically and embed a host of…

This. Armchair software architecture is so easy when you can gloss over the details that make a product great. Also routing, ETAs, geocoding/search, etc. WebGL visualizations and such are probably overkill, but if it makes the company more fun to work for then it probably breaks even, at worst.

Couldn't agree more. It's all the invisible details that cause the load.

It's a much more trivial example, but highlights the point well I think - we have pages in the app I work on that would respond in ~100ms, but might have a single sentence on them that takes another 100ms to generate because of the complex data relationships involved in figuring out what that sentence needs to say. The 'request handler' might be 20 lines of code, with a 50 line util function to generate that line of text. No armchair architect will ever take into account things like that, but the end result is a page that is just a bit more personalised to the user and therefore improves their experience.

In an app of any real size, I imagine there are anywhere from hundreds to many thousands of tiny little details like this that all together drastically increase the amount of power needed to run a service.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#82
post #72

Earlier quoted context omitted.

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

Like bureaucracy, the complexity of your tech stack grows to accommodate the number of people available to work on it. Go crazy on the hiring, and what do you expect all those people to do all day?

That's a very uncharitable views, and it could be argued the other way around, that the hiring occurs to support the need for more people in engineering. I'm not certain which way around it goes with Uber, but I've seen both.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#83

Earlier quoted context omitted.

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

True, I think their CTO must be confused and easily mislead by techies who just want to get the latest buzz words onto their cvs. Silly comments such as - we gain true insight from pretty graphics rather than tedious sql queries - says it all for me. I suppose they have to burn the insane amount of capital they raised $50 Billion somehow?

I recommend reading "The Visual Display of Quantitive Information" by Tufte. I would have partially agreed with you before, but I really do think that correct visualisation of data can make it vastly more useful, and as a few other commenters have noted, Uber has a big challenge to differentiate themselves from Lyft and others, and effective use of data could well be one of their differentiators.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#84

Earlier quoted context omitted.

Why do you think NoSQL means not important data? MongoDB eats data but I am not sure every NoSQL database does. NoSQL does add certain kinds of complexity, but also simplifies certain problems. Depends where your hard problems are..

MongoDB doesn't eat data any more than any other database. It's used by eBay, Foursquare, Adobe, Facebook etc. And NoSQL databases underpin most of the popular websites around today. It's nonsense to assume all of that data isn't valuable.

Foursquare? For real?

Like what's gonna happen if they have a couple of corrupt data? A minor incovenience at worst?

Is anyone gonna lose millions? Nah. Anyone gonna die? Nah. Anyone gonna get sued? Naaaaaah

Also Facebook uses MySQL for their primary data. Pretty sure it's the same for ebay. Don't know about Adobe, I bet it's the same deal there too.

People get so excited when they hear some big company using X, but they have no clue in what capacity it's used. I can guarantee you that all the data that matters, that need to be consistent and whole are in some kind RDBMS.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#85

Earlier quoted context omitted.

Because in the 2 decades I'm in the industry I see RDBMS make the world spin and NoSQL DBs destroying companies and families. MongoDB and CouchDB eat data for breakfast, I know that from 1st hand experience. And all the others DBs that claim that do not keep cropping up in Aphyr's blog. I ain't saying that all NoSQL dbs are useless. I'm just saying that proposing and choosing an RDBMS solution is going to be the righ…

In the last 2 decades in the industry as well I've never lost data with MongoDB, Riak or Cassandra but have with Oracle, DB2 and PostgreSQL. After all databases are just software and there will always be bugs. Some people just get tripped up by different ones. And you are woefully ignorant to think the RDBMS is the right choice for 99% of projects. Especially since you think that the 1% of remaining users are purely…

>In the last 2 decades in the industry as well I've never lost data with MongoDB, Riak or Cassandra but have with Oracle, DB2 and PostgreSQL

Yet every test proves otherwise. Also, use Google to see how people have lost data with MongoDB. Mongo is not considered a serious piece of technology by any scientist or engineer I know. Postgres though is universally considered an engineering marvel.

>Hint: think about the schema problems associated with storing auto generated features from deep learning models.

Hint: The problem you mentioned? Even less than 1%

Calling me ignorant doesn't change reality you know.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#86

Earlier quoted context omitted.

All it takes is for one server in one datacenter to be slightly different, or perhaps you had a bugfix that needed to go out for users in one area, but you couldn't take the risk of a flaky deploy for the areas that didn't need it, now you've got a deploy that will be a lot more complicated or error-prone than a loop around deployment to one location.

All it takes is for one server in one datacenter to be slightly different But doesn't a global system still run on multiple DCs, at least for redundancy? perhaps you had a bugfix that needed to go out for users in one area, but you couldn't take the risk of a flaky deploy for the areas that didn't need it But if you have a single global system, you can't even make that decision. To be clear, I'm not arguing against i…

> But doesn't a global system still run on multiple DCs, at least for redundancy?

If you've got 2 levels of separation - servers and 'groups' (whether they are datacenters, or whatever) - you've got 2 levels at which that special casing needs to happen. If you only have 1 level - servers - i.e. one deployment, even if that's across multiple datacenters, you only have 1 place to special case. I'd say that's easier.

> But if you have a single global system, you can't even make that decision.

Good point, but my point was that it will be simpler and less error prone in general. You might not be able to push the bugfix, or you might have to risk the deploy globally, but I think either would be better in the long run for a simpler deployment. It is a trade-off though.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#87
post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

You know that saying : "I didn't have time to write a short letter so I wrote a long one". With this kind of technology stack you end up when you try to move fast. I'm sure that if more time and thought would have been put into it, it would have been more elegant and simple. But has time these days ?

I had similar thoughts. "Wow. All those moving parts. Each one of which could fail." Each new piece of unique technology added means that it's probability of failure gets multiplied against what you already have.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#88

Earlier quoted context omitted.

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

True, I think their CTO must be confused and easily mislead by techies who just want to get the latest buzz words onto their cvs. Silly comments such as - we gain true insight from pretty graphics rather than tedious sql queries - says it all for me. I suppose they have to burn the insane amount of capital they raised $50 Billion somehow?

That's quite condescending; their CTO is no rookie.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#89

Earlier quoted context omitted.

This. Armchair software architecture is so easy when you can gloss over the details that make a product great. Also routing, ETAs, geocoding/search, etc. WebGL visualizations and such are probably overkill, but if it makes the company more fun to work for then it probably breaks even, at worst.

Couldn't agree more. It's all the invisible details that cause the load. It's a much more trivial example, but highlights the point well I think - we have pages in the app I work on that would respond in ~100ms, but might have a single sentence on them that takes another 100ms to generate because of the complex data relationships involved in figuring out what that sentence needs to say. The 'request handler' might be…

> No armchair architect will ever take into account things like that

An armchair architect would say it's not needed. They would question whether spending 50% of your response time generating a single sentence is in any way worth it, and wonder what kind of architectural mistakes led to that.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#90
post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

> Yet that's what has to work well to beat the competition, which is taxi dispatchers with paper maps, phones, and radios.

Large taxi companies started to switch to GPS-enabled electronic dispatch systems at least 12 years ago.

Post reply on HN