Earlier quoted context omitted.
Maybe my ignorance, but do they have services like left-pad-as-a microservice? I can't understand which 1000 microservices you can derive for a cab-renting application.
left-pad-as-a microservice You can also have another service to monitor the left-pad service.
What I Wish I Had Known Before Scaling Uber [video]
141–150 of 284 posts
Re: What I Wish I Had Known Before Scaling Uber [video]
#142Earlier quoted context omitted.
Facebook noticed the same thing. http://www.darkcoding.net/software/facebooks-code-quality-pr... - "Exhibit C: Our site works when the engineers go on holiday"
Hospital mortality rates in each department are also lowest when there's a conference for that specialty nearby. The doctors all go there, few/no routine surgeries are done that day, far fewer people die.
Re: What I Wish I Had Known Before Scaling Uber [video]
#143Earlier quoted context omitted.
"Move fast and break things" is the worst thing to happen to software development that I can remember.
I think it's OK to move fast and break things when you're in the initial product development stage, where you need to develop features quickly and try lots of new stuff to see what sticks. Once you have paying customers though, it may be time to slow down and start caring about stability, at least for your main branch.
Re: What I Wish I Had Known Before Scaling Uber [video]
#144I think the world of service architecture is roughly divided in two camps: (1) people who still naively think that Rest/JSON is cool and schemas and databases should be flexible and "NoSQL" is nice and (2) people who (having gone through pains of (1)) realized that strong schemas, things like Thrift, Protobufs, Avro are a good thing, as is SQL and relational databases, because rigid is good. (Camp 1 is more likely to…
Eg rolling out a public API in Thrift/protobuf will severely difficult its adoption, whereas Rest/JSON is pretty much the standard - but then building microservices that communicate to each other in Rest/JSON quickly leads to a costly, hard to maintain, inconsistent mess.
We're obsessed with "one fits all" absolutes in tech. We should have more "it depends" imho
Re: What I Wish I Had Known Before Scaling Uber [video]
#145Earlier quoted context omitted.
I'm curious how a company ends up with 10 git repositories per microservice, and 4 repositories per engineer?
I have over 3 dozen private Git repos for personal amusement or utility projects for stuff that professional consumer software doesn't work as well as what I want. Then I have almost at least 10 repos on GutHub (not this userid). At my last work, 9 month contract, I had 4 I think, aside from our main team repos just to manage personal info documents, my bin folder, etc. It's easy for one developer to have several. On…
Re: What I Wish I Had Known Before Scaling Uber [video]
#146Earlier quoted context omitted.
I've come to question releasing often as a result.
It seems like so many "best practices" are really thinly veiled attempts at exploding complexity with only tenuous potential business advantages. We create ourselves so many of the problems we are paid to solve.
Sounds lucrative ;)
Re: What I Wish I Had Known Before Scaling Uber [video]
#147"Uber is most reliable over the weekends when engineers don't change it" :)
I suspect that the issue isn't the engineers. It's the changes. A system running just fine will usually continue to run just fine. Until it's changed.
Some events (or "changes") I've seen in last year that caused a system to stop running fine:
- analyst ran query that locked reporting DB, which had ripple effects
- integration partner "tweaked" schema of data uploaded to us
- NIC started dropping packets
- growing data made backup software exceed expected window, locking files
- critical security patch released (didn't cause a problem exactly, but did change the system from having no known security holes to having a big one)
On and on. So, yeah, I'm not disputing the idea that changes are often the source of problems. I'm just saying that any moderately complex system is constantly encountering new events whether or not the developers are making changes.
This isn't a rebuttal of your statement so much, as it is rebuttal of a common attitude I see in business folks. A lot of non-technical executives seem to have the mentality that software is "done" after it's built, which is naive IMO.
Active systems require active maintenance. You can avoid a lot of problems with intelligent architecture and robust instrumentation/monitoring, but at the end of the day systems rot and will eventually stop running fine.
And even if your perfectly planned and architected system runs in total isolation on a private server where security isn't a major concern, you'll still build up technical debt if you aren't routinely upgrading major libraries, etc. You'll get to a point where you want to implement some feature, and while there are 3 excellent OSS libraries for your platform to accomplish it, none of them is compatible with your 5 year old version.
You probably know all of this and might even agree with most of it. I just had a visceral reaction to the idea that "a system running just fine will usually continue to run just fine," and felt compelled to respond. :)
Note: most of my experience is with public or enterprise web applications. I imagine other types of systems have other problems.
Re: What I Wish I Had Known Before Scaling Uber [video]
#148Earlier quoted context omitted.
left-pad-as-a microservice You can also have another service to monitor the left-pad service.
And one to aggregate its logs.
And yet another one to ensure that the log-aggregation-monitoring-service is working.
Re: What I Wish I Had Known Before Scaling Uber [video]
#149Earlier quoted context omitted.
Maybe my ignorance, but do they have services like left-pad-as-a microservice? I can't understand which 1000 microservices you can derive for a cab-renting application.
I'm not sure if it's ignorance that leads people to dismiss things they don't understand so much as it is a coping mechanism similar to that which gave birth to various religions. Ignorance just lets it thrive after the birth.
It'll be interesting to know the number of people oncall at any given time and the number of prod alerts per hour/day/week.
Re: What I Wish I Had Known Before Scaling Uber [video]
#150Earlier quoted context omitted.
Facebook noticed the same thing. http://www.darkcoding.net/software/facebooks-code-quality-pr... - "Exhibit C: Our site works when the engineers go on holiday"
"Move fast and break things" https://xkcd.com/1428/