Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we built Uber Engineering's highest query-per-second service using Go (2016)

eng.uber.com

Post reply on HN
Print view View original

21–30 of 124 posts

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#21

Post by Kye »

drchickensalad wrote: post #20
loriverkutya wrote: post #18

Earlier quoted context omitted.

Too much time and not enough things to do.

How does this happen?! They're so expensive

Give me twenty billion dollars to make a ride hailing app and I'll show you.

More seriously: they raised tens of billions of dollars to make a ride hailing app and research self-driving cars. Throwing more money at a problem doesn't solve it faster, but it does pay for hiring people, and headcount is seen as a proxy for doing stuff.

When a measure becomes a target, it ceases to be a good measure.

https://en.wikipedia.org/wiki/Goodhart%27s_law

Some big start-ups[0] solve this conundrum by investing in adjacent companies to find the solutions they're paid to find. Outsourcing! This still has limits because there's so much money flowing around right now. Tossing a few million at a company with hundreds still won't solve the problem faster.

[0] Start-up definition for this post: a company that has taken funding but hasn't yet found a sustainable business model. That's how Uber is still a start-up with more money than most companies make in decades.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#22

Post by twic »

scarejunba wrote: post #8

There's a famous 'rebuttal' post to this here https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

So Uber's algorithm takes 58756 ns/op, and the fastest non-brutal algorithm took 471 ns/op. That's about 100 times faster.

They used 40 machines to serve New Year's Eve traffic. Perhaps with a better algorithm, they could have got away with one.

(Possibly not, because there's still a lot of HTTP and JSON munging work to be done, and the network card becomes a bottleneck at some point)

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#23

Post by matt2000 »

foobarian wrote: post #12
hobofan wrote: post #2

> High performance in throughput and latency. In our main data center serving non-China traffic alone, this service handled a peak load of 170k QPS with 40 machines running at 35% CPU usage on NYE 2015. The response time was > Geofence lookups are required on every request from Uber’s mobile apps and must quickly (99th percentile For a 100ms total budget, a ~50ms 99th percentile for a single microservice doesn't soun…

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

My sense is that most people moving to Go are coming from Node and that explains their excitement about the performance. Not many upsides when you’re already on Java.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#24

Post by asdfman123 »

foobarian wrote: post #12
hobofan wrote: post #2

> High performance in throughput and latency. In our main data center serving non-China traffic alone, this service handled a peak load of 170k QPS with 40 machines running at 35% CPU usage on NYE 2015. The response time was > Geofence lookups are required on every request from Uber’s mobile apps and must quickly (99th percentile For a 100ms total budget, a ~50ms 99th percentile for a single microservice doesn't soun…

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

Yeah, boring programming languages are criminally underrated. Fro instance:

> High developer productivity. Go typically takes just a few days for a C++, Java or Node.js developer to learn, and the code is easy to maintain. (Thanks to static typing, no more guessing and unpleasant surprises).

This is why the rest of the world doesn't use JS for everything.

> There is a lot of momentum behind Go at Uber, so if you’re passionate about Go as an expert or a beginner, we are hiring Go developers. Oh, the places you’ll Go!

If they did it in Java, they wouldn't have to recruit for programmers quite so hard -- they could pull them out of a hat, and fairly easily get a few wizards with two decades' worth of experience in it.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#25

Post by sgt »

gmurad2 wrote: post #19
foobarian wrote: post #12

Earlier quoted context omitted.

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

Don't sell the JVM short. Using modern concurrency models (e.g.: Vertx) it will outperform go in throughput and latency.

I've changed my opinion a bit about Vertx combined with Java.

Personally I tried to push it for several years, but most Java developers (and I mean 95% in my particular case) seem to start resenting it over time.

It's hard to write good services in Vertx, mostly due to its asynchronous model combined with Java's verbosity and boiler plate.

Many teams have junior developers and IMHO it's simply not safe expecting them to write production grade services (albeit simple) mostly on their own. It can be done, but it drains the rest of the team. A more productive Java team would have used something like Spring Boot.

Also, with any technology, there are some gotchas .. and with Vertx they are much harder to figure out. In the end we changed to another Java framework.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#26

Post by YawningAngel »

sgt wrote: post #25
gmurad2 wrote: post #19

Earlier quoted context omitted.

Don't sell the JVM short. Using modern concurrency models (e.g.: Vertx) it will outperform go in throughput and latency.

I've changed my opinion a bit about Vertx combined with Java. Personally I tried to push it for several years, but most Java developers (and I mean 95% in my particular case) seem to start resenting it over time. It's hard to write good services in Vertx, mostly due to its asynchronous model combined with Java's verbosity and boiler plate. Many teams have junior developers and IMHO it's simply not safe expecting them…

I suspect that Loom Fibers are going to offer parity with golang in this area soon enough.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#27

Post by sidlls »

asdfman123 wrote: post #24
foobarian wrote: post #12

Earlier quoted context omitted.

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

Yeah, boring programming languages are criminally underrated. Fro instance: > High developer productivity. Go typically takes just a few days for a C++, Java or Node.js developer to learn, and the code is easy to maintain. (Thanks to static typing, no more guessing and unpleasant surprises). This is why the rest of the world doesn't use JS for everything. > There is a lot of momentum behind Go at Uber, so if you’re p…

They also would have to sift through some really terrible programmers who have fully bought into the worst practices of enterprise-y OO development.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#28

Post by asdfman123 »

sidlls wrote: post #27
asdfman123 wrote: post #24

Earlier quoted context omitted.

Yeah, boring programming languages are criminally underrated. Fro instance: > High developer productivity. Go typically takes just a few days for a C++, Java or Node.js developer to learn, and the code is easy to maintain. (Thanks to static typing, no more guessing and unpleasant surprises). This is why the rest of the world doesn't use JS for everything. > There is a lot of momentum behind Go at Uber, so if you’re p…

They also would have to sift through some really terrible programmers who have fully bought into the worst practices of enterprise-y OO development.

Is that really true? Honest question. Or is that just bias against things that are not new and fancy?

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#29

Post by asdfman123 »

Slartie wrote: post #14
merb wrote: post #10

Earlier quoted context omitted.

well basically ubers blog posts are really low quality and from the outside their engineering descisions are kinda vague. like the switch from postgres to mysql. I'm still clueless how you can have so much money and one of the biggest engineering team, but still can't correctly engineer your stuff. I mean, everybody makes wrong decisions or errors in production code.

From what I gathered off of several HN comments of Uber insiders, the size of their engineering team is actually more likely to be one of the causes for bad engineering decisions. They seem to have too many engineers running around for too little actual work to be done, which results in those engineers coming up with stuff to do to keep them busy (and of course to ensure they appear to be busy and worth their money t…

Also, it's resume-driven development. Five years later they want to be able to be the guy who can say, "Yeah, I was the original designer behind Kafka," but substituting in a brand-new Uber technology.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#30

Post by dashwav »

matt2000 wrote: post #23
foobarian wrote: post #12

Earlier quoted context omitted.

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

My sense is that most people moving to Go are coming from Node and that explains their excitement about the performance. Not many upsides when you’re already on Java.

I mean there is a definite performance gain from Java to Go, although I would agree most people probably wouldn't hit it - but if its just as easy to write, might as well use the better language. On top of that Go makes concurrency so much easier to factor in and program with, which in the backend server space is a very important asset over Java.

As an interesting addendum, I found this to be a really interesting resource when comparing languages at a very base level, as it is very well written and you can actually look at the source code/ thesis papers for all of the implementations: https://github.com/ixy-languages/ixy-languages

Post reply on HN