So did the engineers who proposed this get some kind of bonus considering how much money they saved the company?
If a project fails, do you pay for the loss since you want a share of the profits as well?
Migrating Uber's ledger data from DynamoDB to LedgerStore
31–40 of 345 posts
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#32I’m working on a specialized data store[1] that would be perfect for this kind of use case (large “cold” storage with indexing). But I’m having trouble finding potential customers. I’ve tried Google search ads but got 99% spam and 1% potential investors, but 0% potential customers. If anybody has any ideas I’m all ears. 1. https://www.haystackdb.dev/
I think you should invest some time into improving your landing page and maybe you may see some traction. A good resource for this which I've bookmarked is here(1). Hope that helps.
(1) https://www.indiehackers.com/post/my-step-by-step-guide-to-l...
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#33I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
[0] https://www.sqlite.org/releaselog/3_33_0.html
[1] https://www.sqlite.org/limits.html (#12)
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#34I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#35I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
No it won't. sqlite "only" works with up to 281TB [0] [1] [0] https://www.sqlite.org/releaselog/3_33_0.html [1] https://www.sqlite.org/limits.html (#12)
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#36I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#37I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#38Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?
There is a lot of room in between, where cost estimates are more realistic.
Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#39Re: Migrating Uber's ledger data from DynamoDB to LedgerStore
#40I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
Given 30.7TB SSD’s are about $5500 each and you’d need 56 to to get to 1.7PB (with no redundancy). Not to mention that SQLite’s maximum DB size is 140TB. I don’t think you’d be able to fit this much storage into a single machine, especially not for a few thousand a month and SQLite wouldn’t be appropriate for this use-case.