Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Sourcing Peloton, Uber’s Unified Resource Scheduler

eng.uber.com

Post reply on HN
Print view View original

61–70 of 77 posts

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#61

Post by losteric »

notyourday wrote: post #57
theon144 wrote: post #51

Earlier quoted context omitted.

>I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google. Thank you so much, I thought I was going crazy. I understand the demands of running a service on the level Uber has, but well, for instance I can't imagine what kind of computati…

Makes it possible for me to get a ride, process payments and refunds even when the data centers are having issues or when there are temporary internet problems. What does the facebook engineering team do?

Sure but that doesn't require thousands of backend engineers unless they are reinventing everything... and I'd be left wondering what they are rebuilding since, over the past year, every one of my trips with Uber has been a bar-lowering experience...

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#62

Post by rtpg »

losteric wrote: post #61
notyourday wrote: post #57

Earlier quoted context omitted.

Makes it possible for me to get a ride, process payments and refunds even when the data centers are having issues or when there are temporary internet problems. What does the facebook engineering team do?

Sure but that doesn't require thousands of backend engineers unless they are reinventing everything... and I'd be left wondering what they are rebuilding since, over the past year, every one of my trips with Uber has been a bar-lowering experience...

To be fair, operations becomes a much bigger deal as you get bigger. It’s not just the app, it’s having your infrastructure not fall over (because 0.1% failure rate means losing a lot of money)

Think of all the random bugs you’ve seen in your job and told yourself “eh, this would take someone 2 weeks to fix and is almost never hit by customers”

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#63

Post by matz1 »

ProAm wrote: post #37

Would people use open source stuff from a morally questionable company? Especially when its just a re-write of existing technology posted to a different github repo? EDIT People get so up in arms about Google and Microsoft working with China and the military, but Uber has done some horrendous stuff on their own. Just curious where people think the line is OK to be.

The thing with mortality is it's highly subjective. What horrible for one person is perfectly acceptable for other.

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#64

Post by drb91 »

platinumrad wrote: post #55
fenwick67 wrote: post #32

Earlier quoted context omitted.

The term "uber" obviously predates the company of the same name...

And nobody is wringing their hands when something is given that name.

Uber is a pretty terrible name for a taxi app.

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#65

Post by notyourday »

losteric wrote: post #61
notyourday wrote: post #57

Earlier quoted context omitted.

Makes it possible for me to get a ride, process payments and refunds even when the data centers are having issues or when there are temporary internet problems. What does the facebook engineering team do?

Sure but that doesn't require thousands of backend engineers unless they are reinventing everything... and I'd be left wondering what they are rebuilding since, over the past year, every one of my trips with Uber has been a bar-lowering experience...

I periodically see the same content on my FB feed because well, it is NBD if I see the same update from my friend several times.

Let me assure you, it is a BFD if I get billed twice for the same trip. So I am pretty sure Uber needs quite a few engineers to make sure that their stuff works correctly every time in every market for every customer.

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#66

Post by warp_factor »

losteric wrote: post #61
notyourday wrote: post #57

Earlier quoted context omitted.

Makes it possible for me to get a ride, process payments and refunds even when the data centers are having issues or when there are temporary internet problems. What does the facebook engineering team do?

Sure but that doesn't require thousands of backend engineers unless they are reinventing everything... and I'd be left wondering what they are rebuilding since, over the past year, every one of my trips with Uber has been a bar-lowering experience...

I think one of the bigger challenge is that when you become bigger you launch more projects to handle the scale and each of those projects introduce new bugs for which you need new team of engineers.

Basically, once you hit scaling I think you end up with a super small team that managed to keep simple (instagram for example) or you end up with a huge team that explodes in complexity and needs to grow exponentially to handle all the extra complexity. Uber is very obviously in the later bucket.

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#67

Post by sciurus »

throwawaymath wrote: post #22
spyspy wrote: post #18

Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.

I wouldn't say this is a rewrite of existing technology. They borrowed concepts from other well-known open source projects, but this is substantially a wrapper around Mesos, not a competing project. The technical overview of Peloton[1] is more clear about this than the open source announcement, which is what's featured here. _________________ 1. https://eng.uber.com/peloton/

Thanks for sharing! That is definitely a better link than the one posted.

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#68

Post by eggie5 »

jjoonathan wrote: post #53
spyspy wrote: post #18

Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.

Does anyone know an open source Uber Michelangelo?

there's pipeline.ai, airbnb said they'd open source theirs this year and also, TFX suite is getting there. Platforms are becoming popular

Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler

#70

Post by nemothekid »

warp_factor wrote: post #60
chillacy wrote: post #54

Earlier quoted context omitted.

Maybe not at the same scale but imo their core system has a lot of tech challenges: * Lots of realtime * Resource scheduling problems * Route optimization problems (especially with shared uber or shared lyft rides)

ok, but: * Everything they do is low data (no video, image or anything high bandwith). * Their whole model can be subdivided into smaller local problems (all users//drivers in the bay area have nothing to do with the users//drivers currently in NYC). yes there is a couple of algorithms to develop for Uber Pool, and for the real time matching but everything else looks like a fairly simple app backend to me.

> Everything they do is low data (no video, image or anything high bandwith).*

So is everyone else? Storage and CDN isn't nearly as complex as ad serving on Facebook (Which is like Uber's matching - it's a realtime marketplace). Ad serving takes up relatively little bits.

Post reply on HN