What I Wish I Had Known Before Scaling Uber [video] • Hacker News Board

Re: What I Wish I Had Known Before Scaling Uber [video]

#21

Post by bryanlarsen » Wed, Sep 28, 2016, 1:58 PM UTC

jon-wood wrote: post #16

asimuvPR wrote: post #8

Earlier quoted context omitted.

I've come to question releasing often as a result.

I can see the argument that if releasing causes things to break then don't release so frequently, but in practice the end result of that is lots of things breaking at once and having to unpick everything. Debugging is much easier if you're debugging a single change fresh in your mind.

Yeah, debugging 2 bugs that are shadowing each other often takes several orders of magnitude longer to debug than just a single issue.

Re: What I Wish I Had Known Before Scaling Uber [video]

#22

Post by fitzwatermellow » Wed, Sep 28, 2016, 2:00 PM UTC

Link to Slides here:

https://gotocon.com/dl/goto-chicago-2016/slides/MattRanney_W...

Any speculation as to why Uber doesn't just want to use something like Netflix Eureka / Hystrix instead?

Re: What I Wish I Had Known Before Scaling Uber [video]

#23

Post by buzzdenver » Wed, Sep 28, 2016, 2:06 PM UTC

Did I miss what WIWIK stands for ?

Re: What I Wish I Had Known Before Scaling Uber [video]

#24

Post by ex_amazon_sde » Wed, Sep 28, 2016, 2:07 PM UTC

kenrose wrote: post #6

tiborsaas wrote: post #2

"Uber is most reliable over the weekends when engineers don't change it" :)

We see this pattern at PagerDuty over the majority of our customers. There is a definite lull in alert volume over the weekends that picks up first thing Monday morning. It's led to my personal conclusion that most production issues are caused by people, not errant hardware or systems.

> most production issues are caused by people

It's a well known fact, both for systems and networks.

Re: What I Wish I Had Known Before Scaling Uber [video]

#25

Post by jon-wood » Wed, Sep 28, 2016, 2:08 PM UTC

kowdermeister wrote: post #19

amelius wrote: post #17

Earlier quoted context omitted.

Why is that a problem? You could count several smaller cities as a single large city, as far as sharding is concerned.

That's like a post office way of thinking about the internet, we already past that point. Plus you haven't solved the problem of managing 800 microservices and 2000 programmers, but now you have a deployment problem also.

With 800 services and 2000 programmers you already have a deployment problem.

Re: What I Wish I Had Known Before Scaling Uber [video]

#26

Post by nkassis » Wed, Sep 28, 2016, 2:08 PM UTC

amelius wrote: post #4

I didn't see the video. But given that a cab-service has a natural sharding point (i.e., per city), I don't get why scaling is such an issue.

Not really what the video addresses for the most part. Mostly talking about their development process and impact of breaking up their application into 1000s of microservices. This is more to handle their development team scaling, rate of change etc...

Re: What I Wish I Had Known Before Scaling Uber [video]

#27

Post by jjbiotech » Wed, Sep 28, 2016, 2:09 PM UTC

buzzdenver wrote: post #23

Did I miss what WIWIK stands for ?

What I wish I knew

Re: What I Wish I Had Known Before Scaling Uber [video]

#28

Post by jon-wood » Wed, Sep 28, 2016, 2:09 PM UTC

esrauch wrote: post #11

tiborsaas wrote: post #7

Earlier quoted context omitted.

Cab service on a planetary scale :) They have 2000 engineers, 800 microservices and 8000 GIT repositories. Does it still seem trivial to you? EDIT: to downvoters: why shoot the messenger? :)

I'm curious how a company ends up with 10 git repositories per microservice, and 4 repositories per engineer?

My guess (and it is just a guess) is that each of these services is made up of multiple even smaller libraries which are each being version controlled as a distinct entity. I'm not sure I could fully get behind working at that level of granularity, but it would explain how you end up with quite so many git repositories.

Re: What I Wish I Had Known Before Scaling Uber [video]

#29

Post by Thrillington » Wed, Sep 28, 2016, 2:10 PM UTC

jon-wood wrote: post #16

asimuvPR wrote: post #8

Earlier quoted context omitted.

I've come to question releasing often as a result.

I can see the argument that if releasing causes things to break then don't release so frequently, but in practice the end result of that is lots of things breaking at once and having to unpick everything. Debugging is much easier if you're debugging a single change fresh in your mind.

While having tooling for tight debugging loops allows short release cycles, it leaves that decision up to the business

Re: What I Wish I Had Known Before Scaling Uber [video]

#30

Post by jjbiotech » Wed, Sep 28, 2016, 2:12 PM UTC

esrauch wrote: post #11

tiborsaas wrote: post #7

Earlier quoted context omitted.

Cab service on a planetary scale :) They have 2000 engineers, 800 microservices and 8000 GIT repositories. Does it still seem trivial to you? EDIT: to downvoters: why shoot the messenger? :)

I'm curious how a company ends up with 10 git repositories per microservice, and 4 repositories per engineer?

From what I understand, they split their microservices into logical packages and libraries, each of which have their own repo. Then some microservices have a separate repo to track configuration changes.