Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating Uber's ledger data from DynamoDB to LedgerStore

uber.com

Post reply on HN
Print view View original

271–280 of 345 posts

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#271

Post by vineyardmike »

MarkMarine wrote: post #252
xyst wrote: post #185

Uber must have picked up some Google rejects. This type of homegrown project was seen at Google all the time. Usually to aim for a significant promotion. “Designed and built homegrown system to save $Xm! Give me promo, bro?” Just so happened to ignore that it took X+Y additional to build. Also it will probably be going to the G graveyard in a few years.

I heard from some X-Uber people that you could call Uber a database company as much as you could call it a transportation company. Something like 80+ databases invented there in one form or another. Promotion-driven development. I suppose better than blog post driven development, but marginally so.

> I suppose better than blog post driven development, but marginally so.

I find DB development quite interesting, and I find Uber's core product quite not interesting (from an engineering perspective).

So as an outsider without any financial stake in the company, please keep writing about databases!

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#272

Post by vineyardmike »

valiant55 wrote: post #249
shawabawa3 wrote: post #243

Earlier quoted context omitted.

> we spent $25m of engineering resources to save $6m annually This is a huge ROI. Borrowing $25m costs about $1.25m/yr so you're winning even with no upfront costs

I mean, ideally you are still employing those people into the future. Plus was there other opportunities to drive value that would've been better spent?

Ideally for the people, but not a requirement. I doubt they won't be conducting more layoffs either.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#274

Post by riku_iki »

ramesh31 wrote: post #127

Another victim of the "Great Normalization", i.e. that entire generation of garbage tech debt generated during the 2010s that was built on NoSQL stores that never should have been, is now coming due. You could probably make an entire consulting business out of migrating these things to MySQL.

> . You could probably make an entire consulting business out of migrating these things to MySQL.

I think it will be very complex task to run MySQL for 1PB 1T transactions..

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#275

Post by [deleted] »

riku_iki wrote: post #274
ramesh31 wrote: post #127

Another victim of the "Great Normalization", i.e. that entire generation of garbage tech debt generated during the 2010s that was built on NoSQL stores that never should have been, is now coming due. You could probably make an entire consulting business out of migrating these things to MySQL.

> . You could probably make an entire consulting business out of migrating these things to MySQL. I think it will be very complex task to run MySQL for 1PB 1T transactions..

[deleted]

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#276

Post by jakjak123 »

ndriscoll wrote: post #179
kondro wrote: post #29

Earlier quoted context omitted.

Given 30.7TB SSD’s are about $5500 each and you’d need 56 to to get to 1.7PB (with no redundancy). Not to mention that SQLite’s maximum DB size is 140TB. I don’t think you’d be able to fit this much storage into a single machine, especially not for a few thousand a month and SQLite wouldn’t be appropriate for this use-case.

There are 61.44 TB NVMe drives (best price I've seen right now is ~6200. They were ~4800 earlier this year). You can have a 1U server with 32 E1.L slots so you should be able to fit ~1.9PB raw storage into 1U for a little over $200k. Don't know how business financing works, but at 8% interest with a 5 year amortization, that's a bit over $4k/month.

Our ops team actually wanted to do this, but we on the project have nightmares from putting 1PB of database on a single host ><

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#277

Post by yazaddaruvala »

Reading the article it’s clear pretty quickly that Uber was using DynamoDB poorly.

It seems they need strong consistency for certain CUJs and then a lot of data warehousing for historical transactions.

It’s strange to me that they didn’t first convert their 2 table DynamoDB architecture into DynamoDB and Redshift architecture or similar. This is a pretty common pattern.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#278

Post by vertis »

PeterZaitsev wrote: post #239
SkyMarshal wrote: post #236

It seems LedgerStore is not open source [1], and finding any info on it requires following a trail of backlinked Uber blog posts. Here's one with the most info on LedgerStore that I can find, from 2021: https://www.uber.com/en-US/blog/dynamodb-to-docstore-migrati... [1]: https://github.com/uber

Yeah. This looks like some internal solution. In general Uber seems to be high on "not invented here" scale - they like to conclude no existing Open Source solutions are good enough for them and they need to build their own... this is different from Facebook approach for example which chose to made MySQL better by adding MyRocks/RocksDB to it and keep them Open Source.

It's a weird world where Facebook/Meta is becoming a small bastion of hope. Llama 2/3 being an example of bucking the trend of going closed source for LLM models.

Granted it's not quite in the same calibre as OpenAI/Claude, and the real test is when it is and they still release it.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#279

Post by remus »

citizenpaul wrote: post #217

I think there is some reckoning of cloud service providers coming(assuming logical actors...). I was doing some contract work for a small place that had a GCP Bigtable that was costing $11k+ per month for some reports that were based on data from a 375MB !!! mysql db into big-table for the reports to run. They hired some out of school data scientist to do reports and they were doing crazy ineffective things with the…

Not that I disagree with your overall point, but I don't think this

> I was doing some contract work for a small place that had a GCP Bigtable that was costing $11k+ per month for some reports that were based on data from a 375MB !!! mysql db into big-table for the reports to run.

Is a good example. It's just a badly architected system, and you'd have exactly the same problem if you were running the same thing on a massively over provisioned on premise db.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#280

Post by tedd4u »

influx wrote: post #106

I would gladly pay 6 million/year to not be on call, and have to worry about things like bios and ssd firmware ever again.

My "favorite" non-cloud issue was dying/dead RAID card batteries in DB hosts (to preserve unflushed cache in RAM on the card in case of power failure).
Post reply on HN