Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating Uber's ledger data from DynamoDB to LedgerStore

uber.com

Post reply on HN
Print view View original

151–160 of 345 posts

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#151

Post by tinyhouse »

washywashy wrote: post #67

I pretty much never see engineering salaries factored into these types of savings projects. I assume because engineers are already viewed as a sunk cost or maybe it’s just because it’s way less tangible. Have seen many designs describe how X saves Y dollars but ignores the engineering effort to maintain and build it. Half the time I suspect it’s just so people have something to work on, rather than it being some crit…

Spot on. $6M/annually is not much of a saving for a company like Uber ($130B market cap). It only makes sense if it's also more efficient and actually improves the app.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#152

Post by Closi »

sgt wrote: post #109
thangngoc89 wrote: post #81

Earlier quoted context omitted.

I'm replying to the question of how one would replicate SQLite 3 in production for redundancy. I myself consider 10GB would be the limit for using SQLite 3 in read/write in production and switch to PostgreSQL.

That's a huge discrepancy. One half of HN wants to put petabytes on SQLite, while your limit is only 10GB.

Why not use SQLite's own guidance on where SQLite probably isn't appropriate:

- Client/Server applications (Check)

- High-volumes (Check)

- Large datasets (Check)

- High concurrency, particularly for writes (Check)

https://www.sqlite.org/whentouse.html

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#153

Post by Sevii »

bjornsing wrote: post #7

I’m working on a specialized data store[1] that would be perfect for this kind of use case (large “cold” storage with indexing). But I’m having trouble finding potential customers. I’ve tried Google search ads but got 99% spam and 1% potential investors, but 0% potential customers. If anybody has any ideas I’m all ears. 1. https://www.haystackdb.dev/

Most places I've worked we couldn't even consider using a product that wasn't supported by a major cloud provider like this. What problem does your product solve that customers absolutely 100% need it?

Structurally, you are a small entity trying to compete on cost with hyper scalar cloud providers and open source software. Most ISVs like you charge a ton of money for big problems very few enterprise customers have.

I think you need to find a specific use case where your product is a clear winner. Like 'HaystackDB is the best option for healthcare exchanges to use when receiving claims'.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#155

Post by smrtinsert »

washywashy wrote: post #67

I pretty much never see engineering salaries factored into these types of savings projects. I assume because engineers are already viewed as a sunk cost or maybe it’s just because it’s way less tangible. Have seen many designs describe how X saves Y dollars but ignores the engineering effort to maintain and build it. Half the time I suspect it’s just so people have something to work on, rather than it being some crit…

A quick look at their careers shows they hiring eng seemingly anywhere but the US so maybe they've already saved the dollars there.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#156

Post by tinyhouse »

dboreham wrote: post #143
washywashy wrote: post #67

I pretty much never see engineering salaries factored into these types of savings projects. I assume because engineers are already viewed as a sunk cost or maybe it’s just because it’s way less tangible. Have seen many designs describe how X saves Y dollars but ignores the engineering effort to maintain and build it. Half the time I suspect it’s just so people have something to work on, rather than it being some crit…

$6M/y is something like 20 heads (depending on where they are, could be more). So probably it's a win. Hard to see that this could take more than about 5. Add cost of hay and water of course.

It's less that 20 heads. The gross spend for each engineer is probably closer to $0.5 million annually. You can layoff 5% without any impact on the company and save so much more. A company like Uber ($130B market cap) isn't going to bother with building something internally to save $6M/year. The only reason to do it is improve efficiency that actually improves the user experience, which then we're talking about a big deal. Sometimes those things happen only because engineers don't have anything else to do and someone needs a promotion...

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#157

Post by cdchn »

davedx wrote: post #148
cdchn wrote: post #65

Earlier quoted context omitted.

Developing and maintaining a totally bespoke DB system with that kind of volume even for $5m/yr, spitball you could get yourself 25 top-notch engineers without AI PhDs and have another mil left over for metal. Sounds plenty feasible to have a nice tailored suit for a core part of your business.

Is accounting really a core part of Uber's business? They're a transportation company not a bank. I kind of question the premise really

Uber is a technology company that tracks 'rides' between drivers that are contractors and customers, and accounts for taking money from one and giving it to another. I wouldn't just call it a core part, I'd go so far as it say it is the intrinsic essence of their business. They're not a bank, but they're not running a branch with tellers taking cash and running ATMs, either.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#159

Post by cdchn »

inoop wrote: post #101
cdchn wrote: post #65

Earlier quoted context omitted.

Developing and maintaining a totally bespoke DB system with that kind of volume even for $5m/yr, spitball you could get yourself 25 top-notch engineers without AI PhDs and have another mil left over for metal. Sounds plenty feasible to have a nice tailored suit for a core part of your business.

> you could get yourself 25 top-notch engineers without AI PhD Not in the US though. According to levels.fyi, an SDE2 makes ~275k/year at Uber. Hire 25 of those and you're already at $6.875MM. In reality you're going to have a mix of SDE1, SDE2, SDE3, and staff so total salaries will be higher. Then you gotta add taxes, office space, dental, medical, etc. You may as well double that number. And that's just the cost o…

"and have another mil left over for metal" was the part accounting for hardware, infrastructure, etc.

And you can fudge the employee salary a mil or two either way, but the point is that spending that much on a team to build something isn't infeasible or even unreasonable.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#160

Post by tayo42 »

tinyhouse wrote: post #151
washywashy wrote: post #67

I pretty much never see engineering salaries factored into these types of savings projects. I assume because engineers are already viewed as a sunk cost or maybe it’s just because it’s way less tangible. Have seen many designs describe how X saves Y dollars but ignores the engineering effort to maintain and build it. Half the time I suspect it’s just so people have something to work on, rather than it being some crit…

Spot on. $6M/annually is not much of a saving for a company like Uber ($130B market cap). It only makes sense if it's also more efficient and actually improves the app.

Yeah in one quarter from a quick google it looks like Uber profits $1.6 Billion. Basically why I never thought about cost savings projects after I learned to put things into perspective.

People really struggle with large numbers in business I've noticed.

Post reply on HN