121–130 of 345 posts
I’m working on a specialized data store[1] that would be perfect for this kind of use case (large “cold” storage with indexing). But I’m having trouble finding potential customers. I’ve tried Google search ads but got 99% spam and 1% potential investors, but 0% potential customers. If anybody has any ideas I’m all ears. 1. https://www.haystackdb.dev/
Earlier quoted context omitted.
Given 30.7TB SSD’s are about $5500 each and you’d need 56 to to get to 1.7PB (with no redundancy). Not to mention that SQLite’s maximum DB size is 140TB. I don’t think you’d be able to fit this much storage into a single machine, especially not for a few thousand a month and SQLite wouldn’t be appropriate for this use-case.
At the moment they're just paying someone else to buy $5000 SSD's and run a database on them at many X markup.
Earlier quoted context omitted.
With a disclaimer that I have no formal nor practical background in marketing, here are some ideas: 1. It is a bit unclear to me when I would use Haystack. The main advantage seems to be cost cutting. It would be nice to see some realized examples of this. 2. When competing for price, you may look like the cheap, and thereby untrusted alternative. There is a risky business paradox here, for which I am sure a fellow H…
1. Good point, thanks. 2. True. One reason I haven’t priced it ridiculously cheap is to avoid this judgement, and fate. With this pricing I won’t necessarily have a smaller profit margin than competitors. The cost advantage comes from a smarter architecture. Any ideas on how I can communicate that would be greatly appreciated. 3. I used to work for one that needed it. I’ve also interviewed at one that had the same pr…
Companies generally have to be suffering pretty badly to take a risk on changing their tech stack to something unproven. And the risk for you at that point is that they choose to spend 10x on consultants to implement some existing system instead.
The CTO needs to trade off the opportunity cost of developing new features/existing maintenance against integrating an unproven product. How can you de-risk this for them? (Even just showing that you recognize that this is the case can help)
Maybe this is a time to "do things that don't scale". ie: offer to integrate it into their system for them (for at least some small part/pain point), and likely in parallel so that they can evaluate it without taking down the existing system.
Just my two cents.
Earlier quoted context omitted.
1T "records". Any given transaction can have N records. I'm assuming this includes Uber Eats as well.
Still, they have 10B rides in 2023 including Eats, say 75-100B since inception. What would be a record such that each transaction needs 10-15 on average?
Earlier quoted context omitted.
> I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance I'm guessing they know a lot about their costs, and you know very little. There's little value in insulting the team members like this.
> I'm guessing they know a lot about their costs, and you know very little. I'm curious what makes you believe the OP doesn't know about cost? They might be director-level at a large tech company with 20+ years experience for all you know... > There's little value in insulting the team members like this. I'd argue it's not insulting to question a claim (i.e. 'we saved $6MM') that is offered with little explanation.
I wonder if 1.7 petabytes of data (1T indexed records) could fit on a single (very) beefy baremetal server for under a couple thousand dollars a month, served by SQLite. Like this: https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
The original[1][2] articles are a better read IMO. The link is just a summary of the two with added spelling and grammatical errors that materially impact the meaning. 1. https://www.uber.com/blog/how-ledgerstore-supports-trillions... 2. https://www.uber.com/blog/migrating-from-dynamodb-to-ledgers...
Seems to happen with all our blog posts that appear on here (I work at Uber) - I don't get why the originals don't get upvoted but these rehashes do - are our titles just not as good?
Congrats to anyone who worked on it! However, I'm guessing the cost of just running this team be quite large and not significantly different from the savings (6M), and add on top of it the overhead of maintenance. Payments would not likely be a long-term bet as well, so kind of interesting why teams take up such projects ? Is it some kind of sunk-cost with the engineering teams you already have?
Developing and maintaining a totally bespoke DB system with that kind of volume even for $5m/yr, spitball you could get yourself 25 top-notch engineers without AI PhDs and have another mil left over for metal. Sounds plenty feasible to have a nice tailored suit for a core part of your business.
This is using existing features of Docstore which is Uber's own DynamoDB (sharded MySQL) which they seem to be using for almost everything.