Migrating Uber's ledger data from DynamoDB to LedgerStore
1–10 of 345 posts
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#2> Uber migrated all its payment transaction data from DynamoDB and blob storage into a new long-term solution
No way they have 1 trillion transactions right?
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#3> Uber migrated all its payment transaction data from DynamoDB and blob storage into a new long-term solution No way they have 1 trillion transactions right?
1T "records". Any given transaction can have N records. I'm assuming this includes Uber Eats as well.
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#4> Uber migrated all its payment transaction data from DynamoDB and blob storage into a new long-term solution No way they have 1 trillion transactions right?
The blog post says billions of transactions and trillions of indexes (or rather index entries I presume), if I remember correctly.
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#5[deleted]
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#6Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#7I’m working on a specialized data store[1] that would be perfect for this kind of use case (large “cold” storage with indexing). But I’m having trouble finding potential customers. I’ve tried Google search ads but got 99% spam and 1% potential investors, but 0% potential customers. If anybody has any ideas I’m all ears.
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#8Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?
> Payments would not likely be a long-term bet as well
How so? It’s a pretty ubiquitous problem…
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#9So they saved $0.000006 per record, it's really about the little things...
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#10Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?
The estimate sounds suspiciously similar to just the data storage component of DynamoDB. 1.7PB of data and indexes is about $5.1m/year in DynamoDB storage at list.