Immediate reaction: Well, now there's Peloton, the fitness tech company, Peloton, the self-driving truck caravan company, and Peloton, the cluster scheduler...
...and peloton, CMU's NVMe-first db
Open Sourcing Peloton, Uber’s Unified Resource Scheduler
21–30 of 77 posts
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#22Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.
I wouldn't say this is a rewrite of existing technology. They borrowed concepts from other well-known open source projects, but this is substantially a wrapper around Mesos, not a competing project. The technical overview of Peloton[1] is more clear about this than the open source announcement, which is what's featured here.
_________________
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#23This sounds a bit like Two Sigma’s Cook scheduler for Mesos. https://github.com/twosigma/Cook
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#24Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.
Anyone who had to take an Uber after they switched away from Google Maps and onto their in-house half-baked mapping/navigation solution knows this is a huge problem.
I'm assuming like all mapping solutions it'll get better but for now, it's just full of bad routes, over-optimizing turns, out-of-date detours (for MONTHS!) and non-sensical U-turns
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#25Immediate reaction: Well, now there's Peloton, the fitness tech company, Peloton, the self-driving truck caravan company, and Peloton, the cluster scheduler...
...and peloton, CMU's NVMe-first db https://pelotondb.io/
...and https://www.peloton.com and their oil & gas software.
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#26I’ve worked with Mesos pretty extensively before and when Uber first announced Peloton last year I was intrigued. Peloton seems to be a wrapper around Mesos that allows for running smaller, unique jobs without having to write a Mesos framework for each. Writing a Mesos framework for every small job you have can get annoying when you just want to define how your job should run and don’t really care for the resources or task allocation of the job, and it seems like Peloton solves this on Mesos. It’s similar to YARN but not limited to Hadoop. It would have been useful for the project that I worked on because it was more geared for our use case and shifting from Mesos to k8s would’ve been a huge engineering project.
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#27Just call it (Uber) customized mesos. I find this article somehow deceiving and boring. I am pretty sure I can run this peloton thingy with most Mesos API calls.
It pretty much is, but the appeal (at least to me) is abstracting away all the framework writing. It seems like it’s easier to run small, unique jobs on it similar to something like YARN.
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#28Earlier quoted context omitted.
Where do all these names come from? Is it Latin or Greek or something?
Peloton is a somewhat common French word for "ball" that became common in English as a sports term for a grouping in a bicycle race, by way of the Tour de France. https://www.merriam-webster.com/dictionary/peloton
Small detail, that's not what it means in French, it's basically used the same was as in English (a group of cyclist, group of race car drivers, etc.)
It is however pelota, ball, in Spanish.
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#29Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.
Anyone who had to take an Uber after they switched away from Google Maps and onto their in-house half-baked mapping/navigation solution knows this is a huge problem. I'm assuming like all mapping solutions it'll get better but for now, it's just full of bad routes, over-optimizing turns, out-of-date detours (for MONTHS!) and non-sensical U-turns
Not to defend Uber/NIH syndrome, but if Google wanted to specifically charge Uber more for maps because of usage, could they legally do that?
Maybe the rewrites are risk management?
Re: Open Sourcing Peloton, Uber’s Unified Resource Scheduler
#30Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not-invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.
I think you are right.
I think that what is going there is also a bit political. They started to grow their Engineering department so fast that they need to justify the headcounts now. So each team is trying to invent new projects all the time. Anecdotally, this was partially confirmed to me by a friend working there.
I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google.