Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating Uber's ledger data from DynamoDB to LedgerStore

uber.com

Post reply on HN
Print view View original

171–180 of 345 posts

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#171

Post by shermantanktop »

rapht wrote: post #116
scop wrote: post #91

Earlier quoted context omitted.

That was where my mind went to when I saw the headline. Granted that while I’m not on the Finance side of things and am in fact a developer, “six million” didn’t seem like much at all considering engineer salaries. It’s certainly an achievement, but at what short and long term salaried cost?

A primer on valuation: in many financial contexts, $1 of operating savings may be worth much more than $1 of investment. That is because an investment is a one-off, so it's actually worth $1, but the savings are recurring, so they are worth the same number of years that a company's profits are valued. Depending on sector and investors' beliefs in the future of companies, this factor is typically in the 5-20x range. T…

In an ideal org, perhaps. In many places, forming that team starts a process where it continually finds reasons to still exist, so your $1m is yearly until a reorg.

Sigh.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#172

Post by scarface_74 »

gtirloni wrote: post #89
washywashy wrote: post #67

I pretty much never see engineering salaries factored into these types of savings projects. I assume because engineers are already viewed as a sunk cost or maybe it’s just because it’s way less tangible. Have seen many designs describe how X saves Y dollars but ignores the engineering effort to maintain and build it. Half the time I suspect it’s just so people have something to work on, rather than it being some crit…

If anything, it reduces Uber's exposure to AWS' proprietary technology. I don't know how to measure how much that's worth but they probably do.

This usually comes from people who have never done a mass migration at scale.

You’re always dependent on your infrastructure. Even if you have nothing but everything hosted on a bunch of VMs, it can take years and millions of dollars to migrate.

No, just use Terraform and Kubernetes is not the answer.

The typical enterprise is dependent on depending on the source between 80 - 120 SaaS products - ie outside vendors.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#173

Post by _xnmw »

khaki54 wrote: post #103
_xnmw wrote: post #18

I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...

The value proposition of commercial loud isn't cost savings unless you manage to quantify all of the ancillary and extrinsic factors such as security risk, HVAC, datacenter personnel, and hardware lifecycle. Any well capitalized and organized company could build their own cloud much more cheaply, but really a significant portion of the calculation is outsourcing the risk components.

The problem with outsourcing the risk components is that you don't know for sure whether they are properly taken care of. Major cloud providers have been caught "oopsing" your data, and bam it is gone. Furthermore, they have no incentive to be more efficient about it, they could easily be using 10x the amount of resources necessary, and you wouldn't even have a clue, you're just paying for evermore expensive crap that becomes less reliable over time.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#174

Post by appplication »

hackernewds wrote: post #164
dboreham wrote: post #143

Earlier quoted context omitted.

$6M/y is something like 20 heads (depending on where they are, could be more). So probably it's a win. Hard to see that this could take more than about 5. Add cost of hay and water of course.

is it? if you consider the value 20 engineers could drive instead in that time if you assume they wouldn't have had anything else meaningful to work on during that time to save money, then you have a different problem in the company. $6M seems like the value 1 engineer can drive in a company at the scale of Uber

You don’t need to consider the cost they could drive during that time. You have a direct and tangible savings for engineering time invested. That possible value they could otherwise derive is moot and hypothetical, this is the real deal!

But if we’re being honest, there isn’t actually any meaningful quantification of engineering time to understand return on investments at this level (not to say there’s none, but it sure does get wish washy). Corporate and engineering strategy isn’t so carefully weighed, and to believe otherwise is to fall victim to the pseudoscience that is software estimation. You just have to estimate directionally if a given proposal has you heading in a better direction in the long term, pursue that, and course correct along the way.

Put another way, the end state justifies the means and resourcing. It’s rarely possible to fully understand either the costs or benefits with much accuracy up front. You slowly put more resources into projects that show promise, and revoke them if the projects do not appear to be heading in a value add direction.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#175

Post by shermantanktop »

cdchn wrote: post #157
davedx wrote: post #148

Earlier quoted context omitted.

Is accounting really a core part of Uber's business? They're a transportation company not a bank. I kind of question the premise really

Uber is a technology company that tracks 'rides' between drivers that are contractors and customers, and accounts for taking money from one and giving it to another. I wouldn't just call it a core part, I'd go so far as it say it is the intrinsic essence of their business. They're not a bank, but they're not running a branch with tellers taking cash and running ATMs, either.

They are in the transportation market serving transportation needs for a transportation-seeking customer base. How they accomplish that is obviously interesting, but their attempts to move laterally haven’t been amazing from what I can tell (though I don’t follow them closely).

They are structured and run like a tech company but imo they don’t produce a tech product.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#176

Post by victor106 »

bjornsing wrote: post #7

I’m working on a specialized data store[1] that would be perfect for this kind of use case (large “cold” storage with indexing). But I’m having trouble finding potential customers. I’ve tried Google search ads but got 99% spam and 1% potential investors, but 0% potential customers. If anybody has any ideas I’m all ears. 1. https://www.haystackdb.dev/

I would seriously consider an open source business model with an appropriate licensing model. I see lot of companies are open to experimenting with open source db's.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#177

Post by robocat »

mbesto wrote: post #141
Rastonbury wrote: post #108

Earlier quoted context omitted.

$6m across 2.5 years is like $15.5m, how many engineers man-months to breakeven, I'm pretty sure it was worth the work.

$6m in perpetuity is like infinite and engineers can be fired.

It really isn't. The first years dominate the value and later years are worth nothing due to inflation. Google a calculator and use a reasonable discount rate and I suspect you will find that todays value for an infinite perpetuity is a lot less than your intuition might guess. It always surprises me.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#178

Post by vsnf »

barfbagginus wrote: post #144
beanjuiceII wrote: post #129

Earlier quoted context omitted.

i mean it could use a few "blazing fast" sprinkled about

And you can't have blazing fast without rust, and a little kvetching about lifetimes

While your broader point is well taken, isn’t Uber a famous Go shop?

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#179

Post by ndriscoll »

kondro wrote: post #29
_xnmw wrote: post #18

I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...

Given 30.7TB SSD’s are about $5500 each and you’d need 56 to to get to 1.7PB (with no redundancy). Not to mention that SQLite’s maximum DB size is 140TB. I don’t think you’d be able to fit this much storage into a single machine, especially not for a few thousand a month and SQLite wouldn’t be appropriate for this use-case.

There are 61.44 TB NVMe drives (best price I've seen right now is ~6200. They were ~4800 earlier this year). You can have a 1U server with 32 E1.L slots so you should be able to fit ~1.9PB raw storage into 1U for a little over $200k. Don't know how business financing works, but at 8% interest with a 5 year amortization, that's a bit over $4k/month.

Re: Migrating Uber's ledger data from DynamoDB to LedgerStore

#180

Post by nvr219 »

IanCal wrote: post #132
intunderflow wrote: post #41

Earlier quoted context omitted.

Seems to happen with all our blog posts that appear on here (I work at Uber) - I don't get why the originals don't get upvoted but these rehashes do - are our titles just not as good?

Other than the comments about titles, the entire blogpost doesn't show for me with ublock. So I'll open it, see a picture of some birds, scroll around for a bit then give up.

Loads fine for me with ublock. Perhaps you have a custom rule blocking something?
Post reply on HN