Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we built Uber Engineering's highest query-per-second service using Go (2016)

eng.uber.com

Post reply on HN
Print view View original

31–40 of 124 posts

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#31

Post by sidlls »

asdfman123 wrote: post #28
sidlls wrote: post #27

Earlier quoted context omitted.

They also would have to sift through some really terrible programmers who have fully bought into the worst practices of enterprise-y OO development.

Is that really true? Honest question. Or is that just bias against things that are not new and fancy?

It's really true, in my experience. I managed a team in a company that used Spring Boot for services and batch jobs and Python for data and analytics (including ML). My team had its feet in both worlds so I had to hire for that. My experience trying to find programmers who could code and engineer first and not be slaves to the Spring framework was quite terrible in this context.

Another thing of note: the python side of the team could regularly move across and help with bugs or issues in our services and jobs, but the reverse was rarely true. In the 2+ years I had that team there was one person hired for the services side who could do it, and he preferred Go to either Java or Python.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#33

Post by zzbzq »

asdfman123 wrote: post #28
sidlls wrote: post #27

Earlier quoted context omitted.

They also would have to sift through some really terrible programmers who have fully bought into the worst practices of enterprise-y OO development.

Is that really true? Honest question. Or is that just bias against things that are not new and fancy?

Totally true, the whole reason half the popular programming languages exist is that their communities are overreacting to the bad/crazy things that happen in the other half of popular programming languages. Java and C++ communities have a lot of bloated ideas, and the languages themselves lend to that.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#34

Post by geodel »

asdfman123 wrote: post #24
foobarian wrote: post #12

Earlier quoted context omitted.

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

Yeah, boring programming languages are criminally underrated. Fro instance: > High developer productivity. Go typically takes just a few days for a C++, Java or Node.js developer to learn, and the code is easy to maintain. (Thanks to static typing, no more guessing and unpleasant surprises). This is why the rest of the world doesn't use JS for everything. > There is a lot of momentum behind Go at Uber, so if you’re p…

I agree about performance. Java would be totally able to do it.

I feel this article is about as exciting those Java article describing 'Hello World' http server running under 256MB memory as earth shattering.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#35

Post by sethammons »

Rapzid wrote: post #4
hobofan wrote: post #2

> High performance in throughput and latency. In our main data center serving non-China traffic alone, this service handled a peak load of 170k QPS with 40 machines running at 35% CPU usage on NYE 2015. The response time was > Geofence lookups are required on every request from Uber’s mobile apps and must quickly (99th percentile For a 100ms total budget, a ~50ms 99th percentile for a single microservice doesn't soun…

With that kinda volume the 99th is not interesting at all. Should be looking at the 99.99th.

I would argue they should be considering max. Percentiles are good for understanding macro trends. But if every request is sacred, max needs to be monitored. Even at four nines, there are a couple dozen requests per second that are greater than what you are observing. How much greater? Who knows, if you are not observing max. Could be minutes in latency.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#36

Post by woeirua »

This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#37

Post by lclarkmichalek »

sethammons wrote: post #35
Rapzid wrote: post #4

Earlier quoted context omitted.

With that kinda volume the 99th is not interesting at all. Should be looking at the 99.99th.

I would argue they should be considering max. Percentiles are good for understanding macro trends. But if every request is sacred, max needs to be monitored. Even at four nines, there are a couple dozen requests per second that are greater than what you are observing. How much greater? Who knows, if you are not observing max. Could be minutes in latency.

Max is pretty horrible, as any comparison requires identical cohort sizes. P99.99 is much better if you want to go wild.

Given that the service probably did not have a 100% success rate, and almost certainly had timeouts, the "max" would also likely be at the timeout.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#38

Post by MrBuddyCasino »

dashwav wrote: post #30
matt2000 wrote: post #23

Earlier quoted context omitted.

My sense is that most people moving to Go are coming from Node and that explains their excitement about the performance. Not many upsides when you’re already on Java.

I mean there is a definite performance gain from Java to Go, although I would agree most people probably wouldn't hit it - but if its just as easy to write, might as well use the better language. On top of that Go makes concurrency so much easier to factor in and program with, which in the backend server space is a very important asset over Java. As an interesting addendum, I found this to be a really interesting res…

> I mean there is a definite performance gain from Java to Go

I don't think thats true.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#39

Post by sethammons »

lclarkmichalek wrote: post #37
sethammons wrote: post #35

Earlier quoted context omitted.

I would argue they should be considering max. Percentiles are good for understanding macro trends. But if every request is sacred, max needs to be monitored. Even at four nines, there are a couple dozen requests per second that are greater than what you are observing. How much greater? Who knows, if you are not observing max. Could be minutes in latency.

Max is pretty horrible, as any comparison requires identical cohort sizes. P99.99 is much better if you want to go wild. Given that the service probably did not have a 100% success rate, and almost certainly had timeouts, the "max" would also likely be at the timeout.

And that would be a good signal and worth investigating. Why timeouts? I run a few services doing billions of daily queries. I watch max in addition to percentiles.
Post reply on HN