Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Machine Learning at Uber with Michelangelo

eng.uber.com

Post reply on HN
Print view View original

21–30 of 61 posts

Re: Scaling Machine Learning at Uber with Michelangelo

#21

Post by platz »

jquery wrote: post #20
marmaduke wrote: post #2

I love what Uber does with machines (ML), hate what it (currently) does to people. We recently potted some models from Stan to Pyro (SVI on PyTorch), and it’s been reallly exciting (except for the dark corner of poutines), it really has the performance of something being used in production, except the occasional nan explosion. edit we are lazy and use our GitLab CI/CD to drive model development iteration. It’s not as…

I like what Uber (currently) does to people. Gets passengers from point A to point B efficiently while saving them significant money in the process over alternatives. Metaphorically puts dinner on the table of hundreds of thousands of drivers. Literally puts dinner on the table of millions (UberEats). Has a business model that doesn't rely exposing more eyeballs to more ads, corrupting the press, media, and privacy i…

"Silicon Valley innovation now is directly aimed at oppressing the underclass, and everybody knows it and can see it. They hate Uber. People hate Uber. It means the death of the era of good feelings that came with this constant Moore's Law style innovation.

And that was an unforced error, by Silicon Valley. It was in their DNA. They didn't have to give Travis Kalanick, a guy they despised and never trusted, for good reason—They didn't have to give him all that venture capital.

But they saw him as an expendable probe, so they cynically gave him money, to see how much law-breaking he could get away with in the name of their disruption activities.

That was hubris—and nemesis is well on the way."

- NEXT17 | Bruce Sterling | Live from 2027

Re: Scaling Machine Learning at Uber with Michelangelo

#22

Post by [deleted] »

cocobongo wrote: post #19
marmaduke wrote: post #2

I love what Uber does with machines (ML), hate what it (currently) does to people. We recently potted some models from Stan to Pyro (SVI on PyTorch), and it’s been reallly exciting (except for the dark corner of poutines), it really has the performance of something being used in production, except the occasional nan explosion. edit we are lazy and use our GitLab CI/CD to drive model development iteration. It’s not as…

What does Uber currently do to people that you hate? Uber currently provides people with more than 2 Million jobs [1]. Uber drivers/couriers made almost $13 Billion in the US alone last year [2]. [1] https://medium.com/@gc/ubers-path-forward-b59ec9bd4ef6 [2] https://www.sfchronicle.com/business/article/Uber-drivers-in...

[deleted]

Re: Scaling Machine Learning at Uber with Michelangelo

#23

Post by jquery »

platz wrote: post #21
jquery wrote: post #20

Earlier quoted context omitted.

I like what Uber (currently) does to people. Gets passengers from point A to point B efficiently while saving them significant money in the process over alternatives. Metaphorically puts dinner on the table of hundreds of thousands of drivers. Literally puts dinner on the table of millions (UberEats). Has a business model that doesn't rely exposing more eyeballs to more ads, corrupting the press, media, and privacy i…

"Silicon Valley innovation now is directly aimed at oppressing the underclass, and everybody knows it and can see it. They hate Uber. People hate Uber. It means the death of the era of good feelings that came with this constant Moore's Law style innovation. And that was an unforced error, by Silicon Valley. It was in their DNA. They didn't have to give Travis Kalanick, a guy they despised and never trusted, for good…

In the same talk, Bruce Sterling also said, "Do what China says. It’s the ascendant model. It’s destroys the California ideology. The Silicon Valley companies can’t get a toe-hold there."

As far as I can tell, the guy doesn't like America or even representative democracy very much. Take that for what you will.

Re: Scaling Machine Learning at Uber with Michelangelo

#24

Post by woolvalley »

platz wrote: post #21
jquery wrote: post #20

Earlier quoted context omitted.

I like what Uber (currently) does to people. Gets passengers from point A to point B efficiently while saving them significant money in the process over alternatives. Metaphorically puts dinner on the table of hundreds of thousands of drivers. Literally puts dinner on the table of millions (UberEats). Has a business model that doesn't rely exposing more eyeballs to more ads, corrupting the press, media, and privacy i…

"Silicon Valley innovation now is directly aimed at oppressing the underclass, and everybody knows it and can see it. They hate Uber. People hate Uber. It means the death of the era of good feelings that came with this constant Moore's Law style innovation. And that was an unforced error, by Silicon Valley. It was in their DNA. They didn't have to give Travis Kalanick, a guy they despised and never trusted, for good…

Is it Uber / gig-economy apps you don't like, or the general idea of low income relatively unskilled labor jobs?

Re: Scaling Machine Learning at Uber with Michelangelo

#25

Post by mi_lk »

googlemike wrote: post #15
paulie_a wrote: post #3

It's kinda funny they tout their usage of GPS. I use Uber on a near daily basis and drivers by an large use Google maps. They have out right said "Uber sucks for directions" And if you use express pools it will always say to go the wrong side of an intersection. I like uber because of the drivers, but their fancy technology is flawed.

Please do not conflate GPS with navigation. There is a massive set of problems you can solve with high fidelity GPS Data (Uber knows it is a driver in a car, verifies it with another GPS entity (rider app reports GPS also), etc). There is not that much overlap between great GPS data and great maps - no amount of great GPS data will give you a good basemap. Please let me know if I am not making sense, I am more than h…

Can you expand on the difference between GPS and navigation?

Re: Scaling Machine Learning at Uber with Michelangelo

#26

Post by platz »

woolvalley wrote: post #24
platz wrote: post #21

Earlier quoted context omitted.

"Silicon Valley innovation now is directly aimed at oppressing the underclass, and everybody knows it and can see it. They hate Uber. People hate Uber. It means the death of the era of good feelings that came with this constant Moore's Law style innovation. And that was an unforced error, by Silicon Valley. It was in their DNA. They didn't have to give Travis Kalanick, a guy they despised and never trusted, for good…

Is it Uber / gig-economy apps you don't like, or the general idea of low income relatively unskilled labor jobs?

It's the idea of companies externalizing costs onto their labor force, because they refuse to recognize their labor force as "workers".

They avoid responsibility to communities they generate profits in, by exporting negative externalizes at a much higher level than traditional businesses.

Also, I don't think Uber drivers are 'unskilled'. The lowest rung is filtered out by not being able to bring their own $20000 vehicle to participate.

Re: Scaling Machine Learning at Uber with Michelangelo

#27

Post by marmaduke »

sandGorgon wrote: post #17
marmaduke wrote: post #10

Earlier quoted context omitted.

We uh treat models as code, but also have NFS shares setup for the storage and GitLab runner talking to a Slurm cluster to run the models. Results and cross validation upload to GitLab. Main thing we haven’t built out yet are performance dashboards for showing improvement across commits, but with the GitLab APIs that’s a script away (currently we do it by hand)

Thanks for your reply. Actually the question was more around "how do you create your models and what do you mean treating them as code", "why slurm and not something like airflow" , "what is the test/performance setup - backtesting, smoke test" etc etc The Gitlab stuff is easier to understand.

Ah right,

> how do you create your models and what do you mean treating them as code

we start with local Jupyter notebooks, and refactor bits of code into modules that get tested, which for our models mainly means recovering parameters from simulations, and then test them on real data, where we assess performance with LOO approximations for Bayesian models (notably PSIS) and some labeling from experts (which is not taken too seriously tbh)

> why slurm and not something like airflow

because the HPC resources we have access to are built with Slurm, which is super fast, supports DAGs of jobs, schedules our jobs reliably and quickly. I don't really want the other stuff on the Airflow feature list to be honest.

Re: Scaling Machine Learning at Uber with Michelangelo

#28

Post by marmaduke »

ptd wrote: post #18
marmaduke wrote: post #10

Earlier quoted context omitted.

We uh treat models as code, but also have NFS shares setup for the storage and GitLab runner talking to a Slurm cluster to run the models. Results and cross validation upload to GitLab. Main thing we haven’t built out yet are performance dashboards for showing improvement across commits, but with the GitLab APIs that’s a script away (currently we do it by hand)

What do you think are the best dashboard options for showing improvements?

I'd probably set up Grafana talking to Elasticsearch or PostGres, but I haven't thought too hard about it.

Re: Scaling Machine Learning at Uber with Michelangelo

#29

Post by marmaduke »

jquery wrote: post #20
marmaduke wrote: post #2

I love what Uber does with machines (ML), hate what it (currently) does to people. We recently potted some models from Stan to Pyro (SVI on PyTorch), and it’s been reallly exciting (except for the dark corner of poutines), it really has the performance of something being used in production, except the occasional nan explosion. edit we are lazy and use our GitLab CI/CD to drive model development iteration. It’s not as…

I like what Uber (currently) does to people. Gets passengers from point A to point B efficiently while saving them significant money in the process over alternatives. Metaphorically puts dinner on the table of hundreds of thousands of drivers. Literally puts dinner on the table of millions (UberEats). Has a business model that doesn't rely exposing more eyeballs to more ads, corrupting the press, media, and privacy i…

Sorry, I don't believe any of that. It reads like "let them eat cake".

Re: Scaling Machine Learning at Uber with Michelangelo

#30

Post by marmaduke »

cocobongo wrote: post #19
marmaduke wrote: post #2

I love what Uber does with machines (ML), hate what it (currently) does to people. We recently potted some models from Stan to Pyro (SVI on PyTorch), and it’s been reallly exciting (except for the dark corner of poutines), it really has the performance of something being used in production, except the occasional nan explosion. edit we are lazy and use our GitLab CI/CD to drive model development iteration. It’s not as…

What does Uber currently do to people that you hate? Uber currently provides people with more than 2 Million jobs [1]. Uber drivers/couriers made almost $13 Billion in the US alone last year [2]. [1] https://medium.com/@gc/ubers-path-forward-b59ec9bd4ef6 [2] https://www.sfchronicle.com/business/article/Uber-drivers-in...

Treat people like freelancers while paying them like low-wage waiters? $13bn/2mn jobs is $6k/job/yr, not so impressive compared to welfare.
Post reply on HN