Live data from Hacker News

How we built Uber Engineering's highest query-per-second service using Go (2016)

eng.uber.com

61–70 of 124 posts

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#61
post #36

This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

Thanks for the shout out. If anyone has questions or comments feel free to reach out.

Also I'm looking for work so if anyone is interested snag my email from my profile.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#62

Earlier quoted context omitted.

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

My sense is that most people moving to Go are coming from Node and that explains their excitement about the performance. Not many upsides when you’re already on Java.

Why is it that every Java application I've encountered in my 20 years of corporate IT, has been a performance pig? Is it just a case of bad developers, making shit code? Is it difficult to make Java perform well, but when it does, it shines? I can't have accidentally interacted with ONLY the crappiest Java apps in my tenure.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#63
post #36

This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

I'm currently implementing an rtree in elm. Does anyone have good references about rtrees that they like?

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#64
post #10

There's a famous 'rebuttal' post to this here https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

well basically ubers blog posts are really low quality and from the outside their engineering descisions are kinda vague. like the switch from postgres to mysql. I'm still clueless how you can have so much money and one of the biggest engineering team, but still can't correctly engineer your stuff. I mean, everybody makes wrong decisions or errors in production code.

[deleted]

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#65
post #36

This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

I've used RTrees for this too (In Java) and they're great.

Hundreds of thousands of queries a second per cpu core on millions of data points.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#66
post #55
post #44

I think this write up is very fair for a solid engineering team. Is it groundbreaking and eye opening? Absolutely not, I’d say the most “hmm I didn’t know that” part of the entire thing was the part about R-Trees and S2. Is that bad? Absolutely not. These guys did the work, logged their performance and are sharing their story. However, and I believe this is where the animosity in the comments is coming from - given t…

> I guess Leetcoding and Production Engineering really are different things. This is key point. They delivered value to the business. That's the only thing that matters.

> They delivered value to the business. That's the only thing that matters.

Except this is an engineering blog post, so the engineering part actually matters.

And, as demonstrated in the article [1] linked around this discussion, there were better engineering approaches that would have been even "better for business", as being more efficient means lower costs per transaction and/or higher throughput.

[1] https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#67
post #36

This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

clearly the most efficient solution is to pontificate about other people's implementations while never having done anything of the sort at that scale yourself.

While he was a bit rude I wouldnt assume he hasn't built something at this sort of scale.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#68
post #43

Earlier quoted context omitted.

If that's 40 bare metal machines... assuming 60ish HT cores, and 35% CPU... that's not that earth shattering. We do that in Java :)

As some other posts have pointed out, their algorithm was holding them back not the language.

Well, switching from nodejs to Go for this use case was probably already a big win even if the algorithm didn't change.

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#69
post #36

This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

clearly the most efficient solution is to pontificate about other people's implementations while never having done anything of the sort at that scale yourself.

For future internet-searchers that stumble upon this, it might be helpful to point out that better solutions exist..

Re: How we built Uber Engineering's highest query-per-second service using Go (2016)

#70
post #4

Earlier quoted context omitted.

With that kinda volume the 99th is not interesting at all. Should be looking at the 99.99th.

The latency to human user is what matters. If I get an answer in under 100ms as a user 99% of the time, I don’t care whether 100 or 100K times per day someone else hits that 1% worst case.

It has nothing to do with your experience being 99% good enough and more to do with the size of the revenue opportunity that could be lost messing up 1% of requests.
Post reply on HN