Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What I Wish I Had Known Before Scaling Uber [video]

youtube.com

Post reply on HN
Print view View original

91–100 of 284 posts

Re: What I Wish I Had Known Before Scaling Uber [video]

#91

Post by fail2fail2ban »

KayEss wrote: post #59
gtrubetskoy wrote: post #57

I think the world of service architecture is roughly divided in two camps: (1) people who still naively think that Rest/JSON is cool and schemas and databases should be flexible and "NoSQL" is nice and (2) people who (having gone through pains of (1)) realized that strong schemas, things like Thrift, Protobufs, Avro are a good thing, as is SQL and relational databases, because rigid is good. (Camp 1 is more likely to…

I still kind of think you can have great REST and JSON and make full use of a great RDBMS. REST/JSON with strong schemas (and loose where that makes sense), using Postgres and C++.

REST/JSON makes sense between a human backed browser and a middle tier; between machines, SOAP is the best contract first technology, or at least a custom XSD backed XML schema.

Re: What I Wish I Had Known Before Scaling Uber [video]

#92

Post by jrochkind1 »

kazinator wrote: post #87
jrochkind1 wrote: post #86

Earlier quoted context omitted.

with computers.

With zeros, there are generally no issues, until you add ones.

Nah, you're making a mistake with the zeroes, you should totally stick to all ones instead, everyone knows ones are better than zeroes (they're one more man! are you blind?!)

Re: What I Wish I Had Known Before Scaling Uber [video]

#93

Post by wolfpwner »

oftenwrong wrote: post #74
gtrubetskoy wrote: post #57

I think the world of service architecture is roughly divided in two camps: (1) people who still naively think that Rest/JSON is cool and schemas and databases should be flexible and "NoSQL" is nice and (2) people who (having gone through pains of (1)) realized that strong schemas, things like Thrift, Protobufs, Avro are a good thing, as is SQL and relational databases, because rigid is good. (Camp 1 is more likely to…

At one time I implemented services that honoured the robustness principle - "be liberal in what you accept, and conservative in what you send". However, I have found that if you are liberal in what you accept, other services can end up dependent on quirks of your liberal acceptance. More recently I have become a believer in the "be conservative in both" strategy. Rigid is good.

But you are oftenwrong.

Re: What I Wish I Had Known Before Scaling Uber [video]

#94

Post by taf2 »

gtrubetskoy wrote: post #57

I think the world of service architecture is roughly divided in two camps: (1) people who still naively think that Rest/JSON is cool and schemas and databases should be flexible and "NoSQL" is nice and (2) people who (having gone through pains of (1)) realized that strong schemas, things like Thrift, Protobufs, Avro are a good thing, as is SQL and relational databases, because rigid is good. (Camp 1 is more likely to…

the short summary of what you describe is...

1. new business evolving quickly to meet and discover the product that fits the market they are chasing.

2. established and optimizing for a possibly still growing market but very well established set of features and use cases. They can take longer to deliver new features and can save lots of money by optimizing.

Re: What I Wish I Had Known Before Scaling Uber [video]

#95

Post by Osiris »

gtrubetskoy wrote: post #57

I think the world of service architecture is roughly divided in two camps: (1) people who still naively think that Rest/JSON is cool and schemas and databases should be flexible and "NoSQL" is nice and (2) people who (having gone through pains of (1)) realized that strong schemas, things like Thrift, Protobufs, Avro are a good thing, as is SQL and relational databases, because rigid is good. (Camp 1 is more likely to…

I've worked on both types of projects. My previous project used Scala on the backend and Node for the public API. Thrift was used to maintain persistent, reusable, connections between the public API and the backend, and to enforce strict typing on message values.

What I really liked about Thrift is that all I needed to know how to use the service was the thrift definition file. It was self-documenting.

My current project uses HTTP and JSON to communicate from the public API to the backend services. There is significantly more overhead (latency and bandwidth) and no enforced document structure (moving toward Swagger to help with that).

HTTP+JSON is great for the front-end where you need a more universally parsable response, but when you control the communication between two systems, something like Thrift/Protobuf solves a lot of problems that a common with REST-ish services.

Re: What I Wish I Had Known Before Scaling Uber [video]

#96

Post by cagataygurturk »

paukiatwee wrote: post #37

Just to confirm, 1000 microservices in this case is 1000 different apps (e.g.different docker images) running simultaneously? 1000 microservices in this case not 1000 microservice instances (e.g. docker instances)? If it is 1000 microservices as in different apps, then they must have at least 2000 running apps (at least 2 instances per app for HA). Maybe uber only have 200 "active" microservice app running at the sam…

Maybe my ignorance, but do they have services like left-pad-as-a microservice? I can't understand which 1000 microservices you can derive for a cab-renting application.

Re: What I Wish I Had Known Before Scaling Uber [video]

#97

Post by mrinterweb »

novaleaf wrote: post #82
gtrubetskoy wrote: post #57

I think the world of service architecture is roughly divided in two camps: (1) people who still naively think that Rest/JSON is cool and schemas and databases should be flexible and "NoSQL" is nice and (2) people who (having gone through pains of (1)) realized that strong schemas, things like Thrift, Protobufs, Avro are a good thing, as is SQL and relational databases, because rigid is good. (Camp 1 is more likely to…

Camps 1 and 2 are not mutually exclusive (well, except your inflammatory "naively" comment). Rest/JSON is a well understood, broadly adopted, low friction RPC format. NoSQL is not always MongoDB (for example, Google Datastore is ACID compliant), and schema enforcement via an ORM layer I would argue is actually a good thing, as it provides schema validation at compile time.

Rest/JSON is not a RPC. Technologies like Thrift can be used for RPC. It depends on how you define low friction for rest + JSON. JSON is schemaless, and that can be great for prototyping, but as soon as deployed services get out of sync, in terms of how they communicate, it becomes more of a burden than an advantage. Thrift, protobuf, avro, can enforce schemas and can raise exceptions for communication mismatches so less defensive programming is needed checking json responses. For internal service communication, I really think using a schema enforcing communication protocol is a good thing.

Re: What I Wish I Had Known Before Scaling Uber [video]

#98

Post by fail2fail2ban »

adekok wrote: post #43
tiborsaas wrote: post #2

"Uber is most reliable over the weekends when engineers don't change it" :)

I suspect that the issue isn't the engineers. It's the changes. A system running just fine will usually continue to run just fine. Until it's changed.

Or until the power is cycled. That's why it's best to do the crazy monkey or whatever it's called, walk into the production offline datacenter, online for testing (you do have geographically redundant datacenters right?) and pull out a half dozen random NIC cables. You can simulate with software tools but nothing beats the real test.

Re: What I Wish I Had Known Before Scaling Uber [video]

#99

Post by jsprogrammer »

Yhippa wrote: post #44
kenrose wrote: post #6

Earlier quoted context omitted.

We see this pattern at PagerDuty over the majority of our customers. There is a definite lull in alert volume over the weekends that picks up first thing Monday morning. It's led to my personal conclusion that most production issues are caused by people, not errant hardware or systems.

After working at various enterprises over the years (where deployments are slower in some cases) I've noticed you'd do a Thursday/Friday/weekend deployment, everything "looks good" and you'll still have a bunch of issues Monday morning due to users finally using the system en masse.

The user experience is pleasant as well. Nothing better than your dependency breaking at 4PM on Friday with no support until Monday.

Re: What I Wish I Had Known Before Scaling Uber [video]

#100

Post by softawre »

fail2fail2ban wrote: post #98
adekok wrote: post #43

Earlier quoted context omitted.

I suspect that the issue isn't the engineers. It's the changes. A system running just fine will usually continue to run just fine. Until it's changed.

Or until the power is cycled. That's why it's best to do the crazy monkey or whatever it's called, walk into the production offline datacenter, online for testing (you do have geographically redundant datacenters right?) and pull out a half dozen random NIC cables. You can simulate with software tools but nothing beats the real test.

> crazy monkey

Literally lol'd. It's "chaos monkey".

Post reply on HN