Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Uber Engineering Tech Stack, Part I: The Foundation

eng.uber.com

Post reply on HN
Print view View original

131–140 of 194 posts

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#131

Post by dzhiurgis »

mkagenius wrote: post #34
Animats wrote: post #29

Earlier quoted context omitted.

I have to admit, I could see this running for a city the size of SF on a desktop machine under the table at the taxi depot. Uber has 11,000 drivers in SF, but probably only a few thousand are on at any one time. A ride takes a few minutes, so if you figure 3,000 active drivers and 4 rides per hour, that's only about 3 ride transactions per second. You have a transaction at ordering, one at ride start, and one at ride…

> tracking of where all the active drivers are, pinging maybe once a minute. Once every 5 seconds will be more accurate.

Driver app pings central server every 15 seconds. If any ride requests are available, it displays it to driver.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#132

Post by minimaxir »

ryanlm wrote: post #107

I just got rejected from them. I applied for a SE position, but they didn't like me I guess. They send you this really condescending rejection letter. I showed them my programming language that I built in C from scratch, and also my data structure library where I implement all the common data structures found in high level languages that I built from scratch in C, among the many projects I have. It must have been my…

You are likely getting downvoted because it is off-topic, at best.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#133

Post by dominotw »

NotQuantum wrote: post #33

Uber is really strapped for engineering talent. Especially when it comes for SRE. Myself and many friends working SRE at various Bay Area companies get consistently hit up for free lunches and interviews. It's really weird considering that their stack doesn't NEED to be this complex....

what are some of the skills/experience needed to be an SRE?

I've been having a really hard time finding a job due to being a 'jack of all trades' and having no specialty. Just an assumption. I have over a decade of experience building webapps.

I've spent over 500 hrs on interviews over the past 3 months doing countless coding tests/exercises, whiteboard interviews. I just seem to never get past on-site interviews.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#134

Post by dominotw »

rantanplan wrote: post #85
threeseed wrote: post #80

Earlier quoted context omitted.

In the last 2 decades in the industry as well I've never lost data with MongoDB, Riak or Cassandra but have with Oracle, DB2 and PostgreSQL. After all databases are just software and there will always be bugs. Some people just get tripped up by different ones. And you are woefully ignorant to think the RDBMS is the right choice for 99% of projects. Especially since you think that the 1% of remaining users are purely…

>In the last 2 decades in the industry as well I've never lost data with MongoDB, Riak or Cassandra but have with Oracle, DB2 and PostgreSQL Yet every test proves otherwise. Also, use Google to see how people have lost data with MongoDB. Mongo is not considered a serious piece of technology by any scientist or engineer I know. Postgres though is universally considered an engineering marvel. >Hint: think about the sch…

Is data-loss something inherent to nosql tech or just poor implementations?

If its the latter why haven't there been any reliable nosql implementations.

Perhaps its well suited to non transactional, low fi data?

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#135

Post by [deleted] »

minimaxir wrote: post #132
ryanlm wrote: post #107

I just got rejected from them. I applied for a SE position, but they didn't like me I guess. They send you this really condescending rejection letter. I showed them my programming language that I built in C from scratch, and also my data structure library where I implement all the common data structures found in high level languages that I built from scratch in C, among the many projects I have. It must have been my…

You are likely getting downvoted because it is off-topic, at best.

[deleted]

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#136

Post by praneshp »

matthewrudy wrote: post #124
BenPlanetary wrote: post #102

Earlier quoted context omitted.

Your user accounts aren't global? So does that mean if I travel to a new city I have to create a whole new account?

Surprisingly although we have user accounts implemented, most of our customers don't use them. But yes, as it stands, if you create a user account it is distinct per country.

Why would having federated logins be difficult to implement?

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#137

Post by danpalmer »

brianwawok wrote: post #103
danpalmer wrote: post #77

Earlier quoted context omitted.

All it takes is for one server in one datacenter to be slightly different, or perhaps you had a bugfix that needed to go out for users in one area, but you couldn't take the risk of a flaky deploy for the areas that didn't need it, now you've got a deploy that will be a lot more complicated or error-prone than a loop around deployment to one location.

> All it takes is for one server in one datacenter to be slightly different Don't do that. No one is allowed to ssh to boxes. If you need to enforce it by blowing up and rebuilding all servers once per week, do that. > perhaps you had a bugfix that needed to go out for users in one area, but you couldn't take the risk of a flaky deploy for the areas that didn't need it Feature flags. Default off, but flip on a new pa…

> Don't do that. No one is allowed to ssh to boxes. If you need to enforce it by blowing up and rebuilding all servers once per week, do that.

I'd love to do that. I'd love to have no access to production servers, but ultimately that requires far more work to get right than Ansible configuring the same machines again and again. It also means you can't use dedicated hardware as easily, which restricts performance. It's a great situation to be in, but difficult to get to and requires a non-trivial amount of overhead.

At my place of work we deploy somewhere between 5 and 40 times on any given work day (on a team of 5 engineers). That's because we've managed to engineer a reliable and fast deployment process, but that took a long time to get right. It's powerful, but the overhead, particularly on a small team who are under pressure in a startup environment, can be quite large.

I'm not saying you're wrong, in terms of best practice I completely agree, but when the tradeoff is between sales/acquisition/product market fit/etc, and having a 'smooth' devops process, in many cases, the latter must come second.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#138

Post by dkarapetyan »

kccqzy wrote: post #36
dkarapetyan wrote: post #23

Earlier quoted context omitted.

That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual…

You certainly have an interesting argument but it doesn't sound convincing. Can you explain more about how 1970s technology can solve the problems Uber is facing in 2016?

Here you go https://en.wikipedia.org/wiki/Operations_research. Go to town. The history section and the problems addressed section is more than sufficient. You can expand further if necessary and decide for yourself whether what Uber does is in any way novel and whether it requires all the bloat.

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#139

Post by rantanplan »

dominotw wrote: post #134
rantanplan wrote: post #85

Earlier quoted context omitted.

>In the last 2 decades in the industry as well I've never lost data with MongoDB, Riak or Cassandra but have with Oracle, DB2 and PostgreSQL Yet every test proves otherwise. Also, use Google to see how people have lost data with MongoDB. Mongo is not considered a serious piece of technology by any scientist or engineer I know. Postgres though is universally considered an engineering marvel. >Hint: think about the sch…

Is data-loss something inherent to nosql tech or just poor implementations? If its the latter why haven't there been any reliable nosql implementations. Perhaps its well suited to non transactional, low fi data?

NoSQL DBs usually target distributed environments.

So... enter CAP theorem. There's no free lunch. People think we can simply throw away half a century's worth of science because JSON and schemaless are teh awesome derp derp.

Implementation is surely an issue, if you take into account that the mongodb guys had to acquire another company [1] in order to overcome their abysmal write performance. And yet there were people, and benchmarks that were trying to tell us that mongo was faster than RDBMS alternatives. All this circa 2009-2012.

You know what's faster than everything? Writing to /dev/null ;)

Anyways, depending on your use case there might be a NoSQL out there that might fill your needs and it might actually deliver what it claims it can deliver. But it's hard to sift through all this ad-driven, buzzword-ridden informacials that gets thrown around by start-up companies in the DB domain.

Also, DBs are like filesystems; even if the match/science is correct, it needs at least a decade of proven track record before you can say that it works as advertised.

[1] http://www.informationweek.com/software/information-manageme...

Re: The Uber Engineering Tech Stack, Part I: The Foundation

#140

Post by dominotw »

rantanplan wrote: post #139
dominotw wrote: post #134

Earlier quoted context omitted.

Is data-loss something inherent to nosql tech or just poor implementations? If its the latter why haven't there been any reliable nosql implementations. Perhaps its well suited to non transactional, low fi data?

NoSQL DBs usually target distributed environments. So... enter CAP theorem. There's no free lunch. People think we can simply throw away half a century's worth of science because JSON and schemaless are teh awesome derp derp. Implementation is surely an issue, if you take into account that the mongodb guys had to acquire another company [1] in order to overcome their abysmal write performance. And yet there were peop…

> NoSQL DBs usually target distributed environments. So... enter CAP theorem.

Surely FB is not running MYSQL on a single machine. Perhaps i am misunderstanding what you are saying but saying SQL db's dont face the issues of distribution seems a little strange.

Distribution comes into picture from shape and size of the data not data saving/retrieval techniques. yea?

Post reply on HN