Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Uber Engineering Tech Stack, Part I: The Foundation

eng.uber.com

Post reply on HN
Print view View original

171–180 of 194 posts

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#171

Post by dkarapetyan »

kod wrote: post #169
dkarapetyan wrote: post #159

Earlier quoted context omitted.

Same reason any other software is not good software. Chances are you don't need it and are reaching for a shiny tool. Kafka requires zookeeper and in my experience zookeeper is an operational nightmare. If you need an event bus then there are many out there that are much simpler and easier to maintain operationally with much simpler failure modes. Don't just reach for something because it has been the most common thi…

Many out there that scale like Kafka, so surely you can name some? Yes, zookeeper is a turd, but it's a battle tested turd. Distributed systems aren't easy to get right.

Rabbitmq, perfectly fine message bus in pretty much all use cases. Easier to operate and maintain without any extra dependencies and much simpler failure modes. A few more: zeromq, sqs, hornetq, nats, nsq, etc. Any one of those will most certainly fulfill whatever use case you have.

The point being kafka has a very heavy operational overhead and you better understand what you are getting into and what bargain you're making for the scalability you mention.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#172

Post by kod »

dkarapetyan wrote: post #171
kod wrote: post #169

Earlier quoted context omitted.

Many out there that scale like Kafka, so surely you can name some? Yes, zookeeper is a turd, but it's a battle tested turd. Distributed systems aren't easy to get right.

Rabbitmq, perfectly fine message bus in pretty much all use cases. Easier to operate and maintain without any extra dependencies and much simpler failure modes. A few more: zeromq, sqs, hornetq, nats, nsq, etc. Any one of those will most certainly fulfill whatever use case you have. The point being kafka has a very heavy operational overhead and you better understand what you are getting into and what bargain you're…

I've used Rabbitmq, it most certainly does not fulfill the volume requirements I have.

The fact that you are comparing zeromq to Kafka is pretty good evidence that you have no idea what you are talking about, and are just tossing out names from google. I'm a little disappointed, honestly, I hoped you were aware of something I hadn't heard of.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#173

Post by Animats »

tlrobinson wrote: post #56
ricardobeat wrote: post #52

Earlier quoted context omitted.

It's easy to imagine the simplest stack that can serve the core features of any service, and that is well served by a single box. What's missing from the picture is the infrastructure to replicate this 500 times by separate teams, monitoring all of it, backup, auditing, aggregating customer and business metrics, back-office systems, and more. Plus the fact that these things always grow organically and embed a host of…

This. Armchair software architecture is so easy when you can gloss over the details that make a product great. Also routing, ETAs, geocoding/search, etc. WebGL visualizations and such are probably overkill, but if it makes the company more fun to work for then it probably breaks even, at worst.

The Uber attract screen that seems to show lots of cars near your location has no relationship to reality.

WebGL is mostly client side. Ship the list of data points and let the client do it.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#174

Post by dkarapetyan »

kod wrote: post #172
dkarapetyan wrote: post #171

Earlier quoted context omitted.

Rabbitmq, perfectly fine message bus in pretty much all use cases. Easier to operate and maintain without any extra dependencies and much simpler failure modes. A few more: zeromq, sqs, hornetq, nats, nsq, etc. Any one of those will most certainly fulfill whatever use case you have. The point being kafka has a very heavy operational overhead and you better understand what you are getting into and what bargain you're…

I've used Rabbitmq, it most certainly does not fulfill the volume requirements I have. The fact that you are comparing zeromq to Kafka is pretty good evidence that you have no idea what you are talking about, and are just tossing out names from google. I'm a little disappointed, honestly, I hoped you were aware of something I hadn't heard of.

There are two ways to solve problems in engineering. You either bring the problem closer to your existing solutions by redefining the problem or you keep the problem the same and bring your solutions closer to the problem.

Sounds like you are unwilling to redefine your problem so that it is amenable to solutions that are not kafka.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#175

Post by dkarapetyan »

pfarnsworth wrote: post #167
dkarapetyan wrote: post #23

Earlier quoted context omitted.

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

I love how any armchair quarterback on HN can sit back and dismiss the work of thousands of engineers as bloat, with no actual qualifications of their own. Build a multi-billion dollar company that has satisfied customers around the world, and then let's hear what you have to say. Let me guess, you also came up with the idea for Google Adsense and the iPhone in high school, right?

The business idea is a separate concern. A solid business model can withstand all sorts of abuse and incompetence at all levels. Many eBay CEOs besides their best efforts to destroy the company have been unable to do so. Similarly for PayPal and a few other companies that have excellent product/market fit.

Each of those companies survives despite the best efforts of 1000s of engineers and managers to over-engineer and justify their salaries. So the logic of "1000s of people have worked on this so it must be valuable" is incorrect reasoning. The more pertinent question is how do these companies survive despite all the over-engineering that is happening? Once you ask that question you are almost surely led to the conclusion that the technology is not as relevant as people would like to think and inefficiencies at the technology level have very little effect on actual business outcomes when the product itself provides value people are willing to pay for.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#176

Post by kod »

dkarapetyan wrote: post #174
kod wrote: post #172

Earlier quoted context omitted.

I've used Rabbitmq, it most certainly does not fulfill the volume requirements I have. The fact that you are comparing zeromq to Kafka is pretty good evidence that you have no idea what you are talking about, and are just tossing out names from google. I'm a little disappointed, honestly, I hoped you were aware of something I hadn't heard of.

There are two ways to solve problems in engineering. You either bring the problem closer to your existing solutions by redefining the problem or you keep the problem the same and bring your solutions closer to the problem. Sounds like you are unwilling to redefine your problem so that it is amenable to solutions that are not kafka.

Yeah, you recommended a sockets library as an alternative to a distributed durable circular buffer. Not obviously clueful. Might as well recommend Nginx as an alternative to JavaScript.

I need to durably handle billions of events per day. No amount of redefining changes the underlying business problem.

Kafka, on the other hand, has been helping me solve that problem for years.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#177

Post by brianwawok »

rantanplan wrote: post #111
brianwawok wrote: post #108

Earlier quoted context omitted.

Does row count matter that much compared to data size? I.e. if I have a billion rows but they are 2 32-bit ints, that isn't a lot of data (2 GB + index). I guess the index starts to get pretty big.. but I always just think of raw data size vs # of rows.

Remember, it's just a rule of thumb. Now... tables with 2 32-bit ints as columns are not exactly typical RDBMS data. Also, data in RDBMS are... well relational :) Meaning, the rows of just one table are not that important. The data are going to be queried and combined with data from other tables. And I know that typical relational data that consist of hundreds of millions of entries in each table is something that mo…

Fair enough!

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#178

Post by dkarapetyan »

kod wrote: post #176
dkarapetyan wrote: post #174

Earlier quoted context omitted.

There are two ways to solve problems in engineering. You either bring the problem closer to your existing solutions by redefining the problem or you keep the problem the same and bring your solutions closer to the problem. Sounds like you are unwilling to redefine your problem so that it is amenable to solutions that are not kafka.

Yeah, you recommended a sockets library as an alternative to a distributed durable circular buffer. Not obviously clueful. Might as well recommend Nginx as an alternative to JavaScript. I need to durably handle billions of events per day. No amount of redefining changes the underlying business problem. Kafka, on the other hand, has been helping me solve that problem for years.

Let's see. I can handle a few million on a single instance and I have yet to hit any memory or CPU limits indicating I can handle 10x of what I'm currently handling. Oh and it's about 100k or more transactions per hour at peak load. Just from basic operational observation and logs. Also, have yet to see any durability issues and I've managed to do it without kafka. So pretty basic math says the entire thing can be scaled to a "few billion" transactions in a pretty straightforward way. Then again I'm more willing to redefine my problems to come up with simpler solutions.

But this discussion has devolved into personal insults at this point. We have nothing to teach each other it seems.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#179

Post by kod »

dkarapetyan wrote: post #178
kod wrote: post #176

Earlier quoted context omitted.

Yeah, you recommended a sockets library as an alternative to a distributed durable circular buffer. Not obviously clueful. Might as well recommend Nginx as an alternative to JavaScript. I need to durably handle billions of events per day. No amount of redefining changes the underlying business problem. Kafka, on the other hand, has been helping me solve that problem for years.

Let's see. I can handle a few million on a single instance and I have yet to hit any memory or CPU limits indicating I can handle 10x of what I'm currently handling. Oh and it's about 100k or more transactions per hour at peak load. Just from basic operational observation and logs. Also, have yet to see any durability issues and I've managed to do it without kafka. So pretty basic math says the entire thing can be sc…

If nothing else, I could teach you that zeromq has nothing to do with queing or durability.

It's certainly possible that Rabbit has improved in the years since I used it, if it works for your use cases, great. But don't assume that everyone using a popular technology is doing so because of a fad or without understanding the tradeoffs.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#180

Post by orf »

robbles wrote: post #170
orf wrote: post #168

Earlier quoted context omitted.

> the software actually exists to serve the business. Sure it does, but the business also wouldn't exist without the tech in Ubers case (and a lot of other cases). And it's going to be your head on the line when you keep adding these 100ms sentences because the business wants it for no good reason and your page takes 3 seconds to load, and nobody buys anything from the site.

You're making the assumption that the additional features slowing down the service aren't adding value. More common is a "death by 1000 cuts" scenario where the various causes of slowness are apparent to the developers, but quite difficult to remove because they've become necessary to the continued success of the business.

> You're making the assumption that the additional features slowing down the service aren't adding value.

No, I'm questioning whether the value added is greater than the value lost, and in this hypothetical example clearly not. So it's your job to point that out to whoever and not silently obey.

Post reply on HN