Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Uber Engineering Tech Stack, Part I: The Foundation

eng.uber.com

Post reply on HN
Print view View original

41–50 of 194 posts

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#41

Post by daniel-levin »

Animats wrote: post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

Well, to some extent, they do. Their geofencing [0] service takes in coordinates and returns the geofences these coordinates fall into. They make this faster by pruning irrelevant geofences:

>> Instead of indexing the geofences using R-tree or the complicated S2, we chose a simpler route based on the observation that Uber’s business model is city-centric; the business rules and the geofences used to define them are typically associated with a city. This allows us to organize the geofences into a two-level hierarchy where the first level is the city geofences (geofences defining city boundaries), and the second level is the geofences within each city.

[0] https://eng.uber.com/go-geofence/

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#43

Post by rimantas »

dkarapetyan wrote: post #23
e1g wrote: post #21

I'd love to know how many people are responsible for devops/operations/app at various stages of any company's journey. Wikipedia says Uber employs 6,500 people so if even 15% of that is on the tech side of the business that's still 1,000+ people allocated to tech. I think this metric would be a useful reality check for a "modern" SaaS project with 3-10 people that's trying to emulate a backend structure similar to th…

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

Exactly. I saw the presentation of one of thei tech guys and was very surprised by it. First, by a number of IT people they have, second, by the work they do. It looked inventibg problems and solving problems for the sake if problems and solutions, i. e. no business value in it.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#44

Post by afarrell »

Animats wrote: post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

Why would Uber/Lyft use be localized? I tend to use it far more when in an area where I'm unfamiliar with the local transit network.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#45

Post by garyclarke27 »

dkarapetyan wrote: post #23
e1g wrote: post #21

I'd love to know how many people are responsible for devops/operations/app at various stages of any company's journey. Wikipedia says Uber employs 6,500 people so if even 15% of that is on the tech side of the business that's still 1,000+ people allocated to tech. I think this metric would be a useful reality check for a "modern" SaaS project with 3-10 people that's trying to emulate a backend structure similar to th…

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

True, I think their CTO must be confused and easily mislead by techies who just want to get the latest buzz words onto their cvs. Silly comments such as - we gain true insight from pretty graphics rather than tedious sql queries - says it all for me. I suppose they have to burn the insane amount of capital they raised $50 Billion somehow?

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#46

Post by imaginenore »

afarrell wrote: post #44
Animats wrote: post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

Why would Uber/Lyft use be localized? I tend to use it far more when in an area where I'm unfamiliar with the local transit network.

But you are local to that unfamiliar area when you search for a ride. That's what they mean.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#47

Post by genmon »

Animats wrote: post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

Uber is pretty formidable in building and growing a two-sided market. I suspect it's tuned continuously, at high resolution (in space and time), with levers I mostly don't know. And that's got to be a big contribution to the complexity of the stack.

This of it this way. A standard e-commerce site, or SaaS with low-touch marketing... there are a crazy number of KPIs to monitor, loads of levers (e.g. what's the right discount to fix basket abandonment). The instrumentation to track all these conversation funnels (and to do A/B tests to see what works) is half the job.

But at least we have a common understanding of the metrics and the levers -- for SaaS, say, there are tons of similar services, and the knowledge is shared in the community.

Uber? How do you grow while maintaining market liquidity every evening of every week? If you artificially hike demand from passengers in a particular neighbourhood (say with coupons), does WOM amongst potential drivers work to increase the driver pool before the passengers get frustrated and move to Lyft?

All of this is new. So how do you create the tech to track not just all the data you need, but all the data you might need, plus the capabilities to do tests to figure out what levers to pull? Hard.

I have no particular insight into this. But my guess is that Uber isn't flying blind - their growth has been no accident - and the complexity of their tech is due to instrumentation not operations.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#48

Post by matthewrudy »

asdfaoeu wrote: post #28
Animats wrote: post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

> We assign cities to the geographically closest data center, but every city is backed up on a different data center in another location.

I tried to debug this from Hong Kong, and the API calls definitely go to the US.

Maybe in Europe they resolve to a European data center, but in Asia they still seem to send everything via the US.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#49

Post by matthewrudy »

Animats wrote: post #25

It's interesting that they don't break the problem apart geographically. It's inherent in Uber that you're local. But their infrastructure isn't organized that way. Facebook originally tried to do that, then discovered that, as they grew, friends weren't local. Uber doesn't need to have one giant worldwide system. Most of their load is presumably positional updates. Uber wants both customers and drivers to keep their…

I work on an uber-like system, but with ~3 backend devs rather than 100s.

We made the opposite decision, cloning our full stack for each new market.

That's great for scalability, but is a nightmare for devops.

If anything we want to find a way to move to one global system, And then slice down the bits that can be local:

Eg. Create a local order matching service, but keep orders, payments, and user accounts global

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#50

Post by UK-AL »

dkarapetyan wrote: post #23
e1g wrote: post #21

I'd love to know how many people are responsible for devops/operations/app at various stages of any company's journey. Wikipedia says Uber employs 6,500 people so if even 15% of that is on the tech side of the business that's still 1,000+ people allocated to tech. I think this metric would be a useful reality check for a "modern" SaaS project with 3-10 people that's trying to emulate a backend structure similar to th…

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

Docker and Cassandra don't solve basic routing. They solve the fact you have have millions of people hitting you site, and many servers to manage.

You didn't have that problem in the 1970s

Post reply on HN