The original[1][2] articles are a better read IMO. The link is just a summary of the two with added spelling and grammatical errors that materially impact the meaning. 1. https://www.uber.com/blog/how-ledgerstore-supports-trillions... 2. https://www.uber.com/blog/migrating-from-dynamodb-to-ledgers...
Migrating Uber's ledger data from DynamoDB to LedgerStore
41–50 of 345 posts
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#42I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#43I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
How would you replicate that SQLite DB onto other hosts to achieve redundancy?
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#44Does no one ever delete data? It's hard to believe there's much business value in keeping every individual payment record dating back to 2017.
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#45Earlier quoted context omitted.
If a project fails, do you pay for the loss since you want a share of the profits as well?
Someone probably gets fired so I guess someone does pay the ultimate price.
"The metaverse division has now lost more than $45 billion since the end of 2020"
Your compensation for your work is your salery. So I would say that it's fair that the actual risk taker is benefiting from the potential rewards?
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#46I’m working on a specialized data store[1] that would be perfect for this kind of use case (large “cold” storage with indexing). But I’m having trouble finding potential customers. I’ve tried Google search ads but got 99% spam and 1% potential investors, but 0% potential customers. If anybody has any ideas I’m all ears. 1. https://www.haystackdb.dev/
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#47The original[1][2] articles are a better read IMO. The link is just a summary of the two with added spelling and grammatical errors that materially impact the meaning. 1. https://www.uber.com/blog/how-ledgerstore-supports-trillions... 2. https://www.uber.com/blog/migrating-from-dynamodb-to-ledgers...
Seems to happen with all our blog posts that appear on here (I work at Uber) - I don't get why the originals don't get upvoted but these rehashes do - are our titles just not as good?
Saving $6M is key information that makes this story interesting. It’s buried all the way at the bottom of the first blog and is completely missing from the second blog which focuses specifically on the migration
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#48Earlier quoted context omitted.
1T "records". Any given transaction can have N records. I'm assuming this includes Uber Eats as well.
Still, they have 10B rides in 2023 including Eats, say 75-100B since inception. What would be a record such that each transaction needs 10-15 on average?
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#49I read the article so I roughly know what LedgerStore is - but I have no idea where it is hosted.
> LSG promised shorter indexing lag (i.e., time between when a record is written and its secondary index is created). Additionally, it would give us faster network latency because it was running on-premises within Uber’s data centers.
https://www.uber.com/en-AU/blog/migrating-from-dynamodb-to-l...
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#50Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?
> I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance I'm guessing they know a lot about their costs, and you know very little. There's little value in insulting the team members like this.