Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating Uber's ledger data from DynamoDB to LedgerStore

uber.com

Post reply on HN
Print view View original

331–340 of 345 posts

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#331

Post by shermantanktop »

sokoloff wrote: post #320
shawabawa3 wrote: post #319

Earlier quoted context omitted.

I suspect they already had a team maintaining/debugging/operating dynamodb Hosted services doesn't reduce maintenance costs to zero

They have teams maintaining/debugging/operating their use/consumption of dynamodb; they don't have teams maintaining/debugging/operating/securing dynamodb itself. With LedgerDB, they now have to do both.

Optimizing to address high percentile issues (latency/error/correctness) on someone else’s code follows a bimodal pattern: it’s easy until it is impossible. Doing it on your own code is progressively more difficult and the complexity may cause you to give up, but you normally don’t hit the same wall.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#332

Post by surfingdino »

MarkMarine wrote: post #330
surfingdino wrote: post #312

Earlier quoted context omitted.

> I heard from some X-Uber people that you could call Uber a database company Someone once called Airbus planes "Sun servers with wings". All businesses today have an IT department and those build around the idea of real-time scheduling done by computers are compute/storage businesses first. That's why Uber was able to launch Uber Eats or e-bikes. I am not surprised they have a bunch of busy devs building cool stuff.…

nih-syndrome is a real thing. I'm not saying they don't make cool stuff but the engineering culture that prioritizes creating a new database over using something that already exists, then promoting the people who develop the (70th) new DB over the people who were focused on actually delivering value for the customer, it's not my choice in working environment. The architecture where all the magic lives in the database…

> you don't have a large community of other users that you can fall back on for expertise.

That is one of the most important points of consideration when I am being asked to asses new tech. If the project is not actively supported by the community, I don't recommend it. I just can't expose the client to risk just because there's a new shiny thing.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#333

Post by noncoml »

nostrebored wrote: post #300
noncoml wrote: post #254

Earlier quoted context omitted.

I don’t know you are downvoted. Aligning the personal interest vector with the companies interest vector is a huge problem that is usually underrepresented in NH comments. Usually we only complain about the short sighting of the CEOs that prioritize short term stock gains over long term prosperity, but that also is just a specialized case of the success vector misalignment

This is a poor way to view salaried work. If OP is meeting deliverables and did this on the side, this is a great example of a story that would make me inclined to hire.

I think maybe my point was misunderstood.

I was trying to say that the employee tried to do what’s good for the company. However that backfired as his personal, and his manager’s, success vector are not aligned with the company’s. Hence although he did something that benefited the company it hurt his and his manager’s performance (exactly because they are not aligned with that of the company’s).

Hope it clears things up.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#334

Post by bobby_the_whal »

sa46 wrote: post #299
geodel wrote: post #154

Earlier quoted context omitted.

Would be interesting. Considering TigerBeetle is written in Zig. And Uber is probably only rare big company which has support contract with Zig foundation.

One interesting caveat is that Uber only uses the Zig toolchain, not the Zig language. https://www.uber.com/en-US/blog/bootstrapping-ubers-infrastr...

Only Uber can present cross-compilation as if that hasn't been done for decades. I think the newer generation is just more stupid. Must be the plastic in the water supply.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#335

Post by bobby_the_whal »

shermantanktop wrote: post #175
cdchn wrote: post #157

Earlier quoted context omitted.

Uber is a technology company that tracks 'rides' between drivers that are contractors and customers, and accounts for taking money from one and giving it to another. I wouldn't just call it a core part, I'd go so far as it say it is the intrinsic essence of their business. They're not a bank, but they're not running a branch with tellers taking cash and running ATMs, either.

They are in the transportation market serving transportation needs for a transportation-seeking customer base. How they accomplish that is obviously interesting, but their attempts to move laterally haven’t been amazing from what I can tell (though I don’t follow them closely). They are structured and run like a tech company but imo they don’t produce a tech product.

> They are structured and run like a tech company but imo they don’t produce a tech product.

This kind of comment could kill their stock :P

Uber said they were going to produce self-driving cars, so technically they are a failed tech company, which is now just a company. I am surprised they didn't get crushed by a competitor that didn't waste their capital on worthless stuff. So much for capitalism working. Uber should have gone bankrupt.

There is just no way that their database is actually technologically novel. Uber just doesn't have the expertise.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#336

Post by bobby_the_whal »

citizenpaul wrote: post #308
remus wrote: post #279

Earlier quoted context omitted.

Not that I disagree with your overall point, but I don't think this > I was doing some contract work for a small place that had a GCP Bigtable that was costing $11k+ per month for some reports that were based on data from a 375MB !!! mysql db into big-table for the reports to run. Is a good example. It's just a badly architected system, and you'd have exactly the same problem if you were running the same thing on a m…

I think its a perfect example that cloud providers are profiting on a unsustainable scheme. They rely on their services papering over (literally with money) lack of skill/knowlege of their customers. This tactic either kills the customer or creates someone that is desperate to no longer be your customer.

People are getting dumber by the hour. It's sustainable.

When I design computing systems, I am like "what's the point?", if there is nobody with a brain around to maintain it. The cloud allows companies to have services they could never ever maintain themselves long after I am dead.

We are going full Idiocracy from what I can tell.

You sound way too optimistic, but I am guessing you are still young.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#337

Post by admax88qqq »

TheNewsIsHere wrote: post #209
admax88qqq wrote: post #197

Earlier quoted context omitted.

If that $1 of investment doesn't yield any returns that's not an investment it's just an expense. So yes $1 of savings is worth more than $1 of spending.

You could potentially take it as an investment loss for tax purposes. Whether that’s proper depends greatly on the circumstances surrounding how the money was accounted for and spent.

That's not how taxes work. Investing $1 doesn't count as a loss for tax purposes.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#338

Post by overstay8930 »

influx wrote: post #106

I would gladly pay 6 million/year to not be on call, and have to worry about things like bios and ssd firmware ever again.

Yup, every time we bring up migrations of this sort it pretty much stops when we start talking about maintenance and on-call schedules.

Turns out it costs more to DIY when people start quitting because they have to work during a holiday.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#339

Post by omeid2 »

Retric wrote: post #323
omeid2 wrote: post #316

Earlier quoted context omitted.

Outsourcing doesn't mean off the shelf. Apple still works with third parties to manufacture parts as they can leverage the scale of their overall operations to do it cheaper than it would cost apple to do.

They aren’t just working with from a design or quality control standpoint that’s normal enough, they are also purchasing equipment for 3rd party companies to use which isn’t. At this point they need to be world class experts in everything from batteries, sensors, software, and processors to glass. Tesla does some stuff in house because they can, but they just didn’t have any options when it came to scaling battery pr…

Why are these companies going so far as purchasing equipment for 3rd parties instead of doing it in house?

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#340

Post by Retric »

omeid2 wrote: post #339
Retric wrote: post #323

Earlier quoted context omitted.

They aren’t just working with from a design or quality control standpoint that’s normal enough, they are also purchasing equipment for 3rd party companies to use which isn’t. At this point they need to be world class experts in everything from batteries, sensors, software, and processors to glass. Tesla does some stuff in house because they can, but they just didn’t have any options when it came to scaling battery pr…

Why are these companies going so far as purchasing equipment for 3rd parties instead of doing it in house?

Several reasons, both from Apples perspective and the supplier’s.

Suppliers don’t want to purchase a bunch of equipment and scale their manufacturing capacity for a contract that could go away at any time. So when Apple shows up they are going to price that risk in to their bid or sometimes just outright say no. However, by Apple buying that equipment those risks suddenly get reduced.

Apple on the other hand has huge cash reserves and isn’t that price sensitive. Plus they want to avoid all the issues bring yet another component in house.

Post reply on HN