Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating Uber's ledger data from DynamoDB to LedgerStore

uber.com

Post reply on HN
Print view View original

111–120 of 345 posts

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#111

Post by Rastonbury »

inoop wrote: post #97
sha_r_roh wrote: post #6

Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?

I'd be curious as well to see a more complete cost-benefit analysis, and I'd be especially interested in labor cost. We don't know how much time and head count Uber committed to this project, but I would be impressed if they were able to pull this off with fewer than 6-8 people. We can use that to get a very rough lower-bound cost estimate. For example, AWS internally uses a rule of thumb where each developer should…

Not an engineer, but something like this takes 6-8 people working on only this for a full year?

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#112

Post by dboreham »

gronky_ wrote: post #47
intunderflow wrote: post #41

Earlier quoted context omitted.

Seems to happen with all our blog posts that appear on here (I work at Uber) - I don't get why the originals don't get upvoted but these rehashes do - are our titles just not as good?

Yes, that’s definitely the main reason. It’s called “burying the lede”. Saving $6M is key information that makes this story interesting. It’s buried all the way at the bottom of the first blog and is completely missing from the second blog which focuses specifically on the migration

TaaS : title as a service

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#114

Post by pclmulqdq »

_xnmw wrote: post #18

I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...

You wouldn't want to do 1T records on one server even if you could. At that scale, you would prefer to be somewhat distributed for availability and scalability. Also, SQLite has issues at large scale.

A reasonable number for one server is about 32-128 TB, and 1.7 petabytes with some redundancy fits nicely in ~30 servers with a decent distributed database.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#115

Post by silverquiet »

inoop wrote: post #101
cdchn wrote: post #65

Earlier quoted context omitted.

Developing and maintaining a totally bespoke DB system with that kind of volume even for $5m/yr, spitball you could get yourself 25 top-notch engineers without AI PhDs and have another mil left over for metal. Sounds plenty feasible to have a nice tailored suit for a core part of your business.

> you could get yourself 25 top-notch engineers without AI PhD Not in the US though. According to levels.fyi, an SDE2 makes ~275k/year at Uber. Hire 25 of those and you're already at $6.875MM. In reality you're going to have a mix of SDE1, SDE2, SDE3, and staff so total salaries will be higher. Then you gotta add taxes, office space, dental, medical, etc. You may as well double that number. And that's just the cost o…

Work from home doesn't mean that home has to be in the US.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#116

Post by rapht »

scop wrote: post #91
washywashy wrote: post #67

I pretty much never see engineering salaries factored into these types of savings projects. I assume because engineers are already viewed as a sunk cost or maybe it’s just because it’s way less tangible. Have seen many designs describe how X saves Y dollars but ignores the engineering effort to maintain and build it. Half the time I suspect it’s just so people have something to work on, rather than it being some crit…

That was where my mind went to when I saw the headline. Granted that while I’m not on the Finance side of things and am in fact a developer, “six million” didn’t seem like much at all considering engineer salaries. It’s certainly an achievement, but at what short and long term salaried cost?

A primer on valuation: in many financial contexts, $1 of operating savings may be worth much more than $1 of investment.

That is because an investment is a one-off, so it's actually worth $1, but the savings are recurring, so they are worth the same number of years that a company's profits are valued. Depending on sector and investors' beliefs in the future of companies, this factor is typically in the 5-20x range. That means that $1 of savings is well worth at least $5 of investments.

Factor in anything you want!

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#117

Post by [deleted] »

intunderflow wrote: post #41
drpotato wrote: post #15

The original[1][2] articles are a better read IMO. The link is just a summary of the two with added spelling and grammatical errors that materially impact the meaning. 1. https://www.uber.com/blog/how-ledgerstore-supports-trillions... 2. https://www.uber.com/blog/migrating-from-dynamodb-to-ledgers...

Seems to happen with all our blog posts that appear on here (I work at Uber) - I don't get why the originals don't get upvoted but these rehashes do - are our titles just not as good?

[deleted]

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#118

Post by klysm »

_xnmw wrote: post #18

I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...

Sure but then you get a whole new set of costs and folks you have to hire to maintain that hardware.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#119

Post by geodel »

More power to them. At this point even technically decent teams/companies have given up on developing large, complex systems in favor of SaaS. After carefully evaluating our strategic course of action answer always is AWS.

Its only team who propose alternative they have to justify rigorously how come they differ in conclusion.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#120

Post by shrubble »

inoop wrote: post #97
sha_r_roh wrote: post #6

Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?

I'd be curious as well to see a more complete cost-benefit analysis, and I'd be especially interested in labor cost. We don't know how much time and head count Uber committed to this project, but I would be impressed if they were able to pull this off with fewer than 6-8 people. We can use that to get a very rough lower-bound cost estimate. For example, AWS internally uses a rule of thumb where each developer should…

If the savings are 6 million per year, then in later years it should pay off since the development is a one time cost.
Post reply on HN